[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:15:38.876179 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:39889
I20260709 12:15:38.877733 30820 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:15:38.878818 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:38.895656 30827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:38.896615 30826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:38.898523 30820 server_base.cc:1061] running on GCE node
W20260709 12:15:38.895988 30829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:40.106278 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:40.106601 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:40.106827 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599340106799 us; error 0 us; skew 500 ppm
I20260709 12:15:40.107708 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:40.113147 30820 webserver.cc:533] Webserver started at http://127.30.25.62:42029/ using document root <none> and password file <none>
I20260709 12:15:40.114015 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:40.114248 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:40.114770 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:40.120662 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "7e8eb82cbd01496eae600e58f04c31d9"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.128688 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.009s	sys 0.000s
I20260709 12:15:40.134125 30835 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.135519 30820 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:15:40.135952 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "7e8eb82cbd01496eae600e58f04c31d9"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.136309 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:40.199707 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:40.201422 30820 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:15:40.201977 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:40.276623 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:39889
I20260709 12:15:40.276682 30886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:39889 every 8 connection(s)
I20260709 12:15:40.285023 30887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:40.307215 30887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9: Bootstrap starting.
I20260709 12:15:40.313114 30887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:40.315033 30887 log.cc:826] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9: Log is configured to *not* fsync() on all Append() calls
I20260709 12:15:40.319545 30887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9: No bootstrap required, opened a new log
I20260709 12:15:40.339949 30887 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8eb82cbd01496eae600e58f04c31d9" member_type: VOTER }
I20260709 12:15:40.340591 30887 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:40.340868 30887 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e8eb82cbd01496eae600e58f04c31d9, State: Initialized, Role: FOLLOWER
I20260709 12:15:40.341912 30887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8eb82cbd01496eae600e58f04c31d9" member_type: VOTER }
I20260709 12:15:40.342466 30887 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:40.342769 30887 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:40.343127 30887 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:40.347328 30887 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8eb82cbd01496eae600e58f04c31d9" member_type: VOTER }
I20260709 12:15:40.348080 30887 leader_election.cc:304] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e8eb82cbd01496eae600e58f04c31d9; no voters: 
I20260709 12:15:40.349800 30887 leader_election.cc:290] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:40.350184 30890 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:40.352993 30890 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7e8eb82cbd01496eae600e58f04c31d9, State: Running, Role: LEADER
I20260709 12:15:40.353889 30890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8eb82cbd01496eae600e58f04c31d9" member_type: VOTER }
I20260709 12:15:40.355036 30887 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:40.364308 30891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e8eb82cbd01496eae600e58f04c31d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8eb82cbd01496eae600e58f04c31d9" member_type: VOTER } }
I20260709 12:15:40.365046 30892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e8eb82cbd01496eae600e58f04c31d9. Latest consensus state: current_term: 1 leader_uuid: "7e8eb82cbd01496eae600e58f04c31d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8eb82cbd01496eae600e58f04c31d9" member_type: VOTER } }
I20260709 12:15:40.365249 30891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:40.365667 30892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:40.368396 30898 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:40.380043 30898 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:40.384801 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:40.399042 30898 catalog_manager.cc:1383] Generated new cluster ID: 0b5bcfcd8e134b9d9b3d57378a3f7fe5
I20260709 12:15:40.399348 30898 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:40.421895 30898 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:40.423529 30898 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:40.444154 30898 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9: Generated new TSK 0
I20260709 12:15:40.445101 30898 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:40.451787 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:40.461810 30909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:40.465920 30910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:40.466635 30912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:40.466809 30820 server_base.cc:1061] running on GCE node
I20260709 12:15:40.467792 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:40.468062 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:40.468277 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599340468258 us; error 0 us; skew 500 ppm
I20260709 12:15:40.469079 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:40.473261 30820 webserver.cc:533] Webserver started at http://127.30.25.1:45109/ using document root <none> and password file <none>
I20260709 12:15:40.473984 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:40.474247 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:40.474640 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:40.476152 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "22d14ec1ba314b3296094fb24350578d"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.483122 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:15:40.488327 30917 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.489154 30820 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.002s
I20260709 12:15:40.489503 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "22d14ec1ba314b3296094fb24350578d"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.489909 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:40.510346 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:40.512179 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:40.514979 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:40.520399 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:40.520728 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.521037 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:40.521270 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.611882 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:36887
I20260709 12:15:40.611935 30979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:36887 every 8 connection(s)
I20260709 12:15:40.619727 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:40.632406 30984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:40.638839 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:40.642609 30987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:40.643074 30820 server_base.cc:1061] running on GCE node
I20260709 12:15:40.643957 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:40.644239 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:40.644434 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599340644416 us; error 0 us; skew 500 ppm
I20260709 12:15:40.645045 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:40.653503 30980 heartbeater.cc:344] Connected to a master server at 127.30.25.62:39889
I20260709 12:15:40.654016 30820 webserver.cc:533] Webserver started at http://127.30.25.2:41227/ using document root <none> and password file <none>
I20260709 12:15:40.654091 30980 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:40.654825 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:40.655118 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:40.655419 30980 heartbeater.cc:507] Master 127.30.25.62:39889 requested a full tablet report, sending...
I20260709 12:15:40.655454 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:40.657177 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "0761f3b2907f426189bc51a65f60bec1"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.659250 30852 ts_manager.cc:194] Registered new tserver with Master: 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887)
I20260709 12:15:40.662400 30852 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54552
I20260709 12:15:40.663264 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.006s
I20260709 12:15:40.668871 30992 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.669767 30820 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:40.670059 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "0761f3b2907f426189bc51a65f60bec1"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.670353 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:40.691674 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:40.693090 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:40.695408 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:40.698902 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:40.699132 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.699370 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:40.699541 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.796653 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:42661
I20260709 12:15:40.796777 31054 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:42661 every 8 connection(s)
I20260709 12:15:40.803072 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:40.811914 31058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:40.817903 31059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:40.818776 31055 heartbeater.cc:344] Connected to a master server at 127.30.25.62:39889
I20260709 12:15:40.819180 31055 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:40.820194 31055 heartbeater.cc:507] Master 127.30.25.62:39889 requested a full tablet report, sending...
W20260709 12:15:40.821194 31061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:40.822826 30852 ts_manager.cc:194] Registered new tserver with Master: 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:40.824057 30820 server_base.cc:1061] running on GCE node
I20260709 12:15:40.824658 30852 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43502
I20260709 12:15:40.825117 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:40.825407 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:40.825601 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599340825585 us; error 0 us; skew 500 ppm
I20260709 12:15:40.826360 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:40.829111 30820 webserver.cc:533] Webserver started at http://127.30.25.3:44193/ using document root <none> and password file <none>
I20260709 12:15:40.829704 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:40.829942 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:40.830255 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:40.831758 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "1b6b1f8f62214e779396d4da64de242d"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.837385 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:15:40.841486 31066 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.842388 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:15:40.842713 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "1b6b1f8f62214e779396d4da64de242d"
format_stamp: "Formatted at 2026-07-09 12:15:40 on dist-test-slave-rldg"
I20260709 12:15:40.843077 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:40.866402 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:40.868069 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:40.869714 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:40.872560 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:40.872844 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.873135 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:40.873343 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:40.957012 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:36065
I20260709 12:15:40.957099 31128 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:36065 every 8 connection(s)
I20260709 12:15:40.973716 31129 heartbeater.cc:344] Connected to a master server at 127.30.25.62:39889
I20260709 12:15:40.974095 31129 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:40.974980 31129 heartbeater.cc:507] Master 127.30.25.62:39889 requested a full tablet report, sending...
I20260709 12:15:40.977023 30852 ts_manager.cc:194] Registered new tserver with Master: 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:40.978497 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013199173s
I20260709 12:15:40.978772 30852 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43506
I20260709 12:15:41.038487 30852 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43516:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:41.043535 30852 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:41.156328 31093 tablet_service.cc:1511] Processing CreateTablet for tablet 37f44d95ff9f4981ae7f60b2fec9e713 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:41.157532 31090 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff03333b641443a84a40390b03de4f2 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:41.158108 31093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44d95ff9f4981ae7f60b2fec9e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.158918 31090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff03333b641443a84a40390b03de4f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.158794 31087 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6976d88606429d928286415c52bec5 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:41.160565 31088 tablet_service.cc:1511] Processing CreateTablet for tablet d6eafbfc73a149949ff328170fd420d9 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:41.161170 31091 tablet_service.cc:1511] Processing CreateTablet for tablet c18c4394c9804ff5a2701e1d9fbb6bc2 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:41.161952 31088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6eafbfc73a149949ff328170fd420d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.163640 31089 tablet_service.cc:1511] Processing CreateTablet for tablet ea866411680f4522a261bbc1dfd7f180 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:41.168675 31092 tablet_service.cc:1511] Processing CreateTablet for tablet 578ec24f79e14c028d88586a3cf00fc7 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:41.156328 31094 tablet_service.cc:1511] Processing CreateTablet for tablet 5b0e3bffcac84cb3a22d923f6bc3204f (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:41.171267 31092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578ec24f79e14c028d88586a3cf00fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.171972 31087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6976d88606429d928286415c52bec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.173964 31091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c18c4394c9804ff5a2701e1d9fbb6bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.179371 31089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea866411680f4522a261bbc1dfd7f180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.182173 31094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0e3bffcac84cb3a22d923f6bc3204f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.226670 31018 tablet_service.cc:1511] Processing CreateTablet for tablet 578ec24f79e14c028d88586a3cf00fc7 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:41.234751 31020 tablet_service.cc:1511] Processing CreateTablet for tablet 5b0e3bffcac84cb3a22d923f6bc3204f (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:41.238818 31016 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff03333b641443a84a40390b03de4f2 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:41.235287 31018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578ec24f79e14c028d88586a3cf00fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.240857 31019 tablet_service.cc:1511] Processing CreateTablet for tablet 37f44d95ff9f4981ae7f60b2fec9e713 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:41.250609 31017 tablet_service.cc:1511] Processing CreateTablet for tablet c18c4394c9804ff5a2701e1d9fbb6bc2 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:41.255312 31016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff03333b641443a84a40390b03de4f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.258152 31020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0e3bffcac84cb3a22d923f6bc3204f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.265130 31019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44d95ff9f4981ae7f60b2fec9e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.265066 31015 tablet_service.cc:1511] Processing CreateTablet for tablet ea866411680f4522a261bbc1dfd7f180 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:41.259902 31086 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4e730ee7c64029952254b76a7a7a01 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:41.268481 31017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c18c4394c9804ff5a2701e1d9fbb6bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.269390 31014 tablet_service.cc:1511] Processing CreateTablet for tablet d6eafbfc73a149949ff328170fd420d9 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:41.273890 31013 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6976d88606429d928286415c52bec5 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:41.275395 31013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6976d88606429d928286415c52bec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.276432 31015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea866411680f4522a261bbc1dfd7f180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.278259 31147 tablet_bootstrap.cc:492] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:41.281777 31014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6eafbfc73a149949ff328170fd420d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.293341 31086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4e730ee7c64029952254b76a7a7a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.305928 30941 tablet_service.cc:1511] Processing CreateTablet for tablet 0ff03333b641443a84a40390b03de4f2 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:41.304181 30940 tablet_service.cc:1511] Processing CreateTablet for tablet ea866411680f4522a261bbc1dfd7f180 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:41.330871 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4e730ee7c64029952254b76a7a7a01 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:41.338776 31012 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4e730ee7c64029952254b76a7a7a01 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:41.306991 30943 tablet_service.cc:1511] Processing CreateTablet for tablet 578ec24f79e14c028d88586a3cf00fc7 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:41.335744 30939 tablet_service.cc:1511] Processing CreateTablet for tablet d6eafbfc73a149949ff328170fd420d9 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:41.334398 30942 tablet_service.cc:1511] Processing CreateTablet for tablet c18c4394c9804ff5a2701e1d9fbb6bc2 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:41.348084 30941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff03333b641443a84a40390b03de4f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.338917 30945 tablet_service.cc:1511] Processing CreateTablet for tablet 5b0e3bffcac84cb3a22d923f6bc3204f (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:41.352176 30944 tablet_service.cc:1511] Processing CreateTablet for tablet 37f44d95ff9f4981ae7f60b2fec9e713 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:41.348359 30938 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6976d88606429d928286415c52bec5 (DEFAULT_TABLE table=test-workload [id=da4e1c53da564e2db62d329c4edd3942]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:41.370388 31012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4e730ee7c64029952254b76a7a7a01. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:41.396241 30900 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:41.397111 30943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578ec24f79e14c028d88586a3cf00fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.398684 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4e730ee7c64029952254b76a7a7a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.400076 30939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6eafbfc73a149949ff328170fd420d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.401507 30945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0e3bffcac84cb3a22d923f6bc3204f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.408812 30942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c18c4394c9804ff5a2701e1d9fbb6bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.410285 30944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f44d95ff9f4981ae7f60b2fec9e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.411108 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6976d88606429d928286415c52bec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.415885 30940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea866411680f4522a261bbc1dfd7f180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:41.454272 31147 tablet_bootstrap.cc:654] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.535290 31149 tablet_bootstrap.cc:492] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
W20260709 12:15:41.666695 30839 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.30.25.2:42661, user_credentials={real_user=slave}} blocked reactor thread for 127501us
I20260709 12:15:41.668921 30980 heartbeater.cc:499] Master 127.30.25.62:39889 was elected leader, sending a full tablet report...
I20260709 12:15:41.680480 31147 tablet_bootstrap.cc:492] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:41.680997 31147 ts_tablet_manager.cc:1403] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.419s	user 0.003s	sys 0.029s
I20260709 12:15:41.707259 31149 tablet_bootstrap.cc:654] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.707181 31147 raft_consensus.cc:359] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.708091 31147 raft_consensus.cc:385] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.708410 31147 raft_consensus.cc:740] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.709885 31147 consensus_queue.cc:260] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.727041 31129 heartbeater.cc:499] Master 127.30.25.62:39889 was elected leader, sending a full tablet report...
I20260709 12:15:41.740185 31147 ts_tablet_manager.cc:1434] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.059s	user 0.014s	sys 0.006s
I20260709 12:15:41.748091 31147 tablet_bootstrap.cc:492] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:41.767544 31150 tablet_bootstrap.cc:492] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:41.771234 31149 tablet_bootstrap.cc:492] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:41.772122 31149 ts_tablet_manager.cc:1403] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.237s	user 0.025s	sys 0.029s
I20260709 12:15:41.775879 31149 raft_consensus.cc:359] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.776782 31149 raft_consensus.cc:385] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.777098 31149 raft_consensus.cc:740] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.777818 31149 consensus_queue.cc:260] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.785487 31149 ts_tablet_manager.cc:1434] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 12:15:41.786147 31055 heartbeater.cc:499] Master 127.30.25.62:39889 was elected leader, sending a full tablet report...
I20260709 12:15:41.786405 31149 tablet_bootstrap.cc:492] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:41.801318 31147 tablet_bootstrap.cc:654] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.801318 31149 tablet_bootstrap.cc:654] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:41.804922 31056 tablet.cc:2406] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:15:41.801429 31150 tablet_bootstrap.cc:654] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.818266 31150 tablet_bootstrap.cc:492] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:41.818773 31150 ts_tablet_manager.cc:1403] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.002s
I20260709 12:15:41.818933 31149 tablet_bootstrap.cc:492] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:41.819855 31149 ts_tablet_manager.cc:1403] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.009s
I20260709 12:15:41.823225 31147 tablet_bootstrap.cc:492] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:41.823889 31147 ts_tablet_manager.cc:1403] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.026s
I20260709 12:15:41.823226 31150 raft_consensus.cc:359] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.824347 31150 raft_consensus.cc:385] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.823966 31149 raft_consensus.cc:359] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.824656 31150 raft_consensus.cc:740] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.825021 31149 raft_consensus.cc:385] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.825600 31149 raft_consensus.cc:740] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.825409 31150 consensus_queue.cc:260] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.826640 31149 consensus_queue.cc:260] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.826975 31147 raft_consensus.cc:359] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.827662 31147 raft_consensus.cc:385] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.827946 31147 raft_consensus.cc:740] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.828698 31147 consensus_queue.cc:260] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.829583 31149 ts_tablet_manager.cc:1434] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:15:41.830610 31149 tablet_bootstrap.cc:492] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:41.831037 31147 ts_tablet_manager.cc:1434] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:15:41.831908 31147 tablet_bootstrap.cc:492] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:41.837540 31149 tablet_bootstrap.cc:654] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.838408 31147 tablet_bootstrap.cc:654] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.844331 31150 ts_tablet_manager.cc:1434] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.003s
I20260709 12:15:41.845291 31150 tablet_bootstrap.cc:492] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:41.850559 31149 tablet_bootstrap.cc:492] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:41.850991 31149 ts_tablet_manager.cc:1403] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.006s
I20260709 12:15:41.851625 31150 tablet_bootstrap.cc:654] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.853777 31149 raft_consensus.cc:359] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.855302 31149 raft_consensus.cc:385] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.855588 31149 raft_consensus.cc:740] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.856302 31149 consensus_queue.cc:260] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.858556 31147 tablet_bootstrap.cc:492] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:41.859110 31147 ts_tablet_manager.cc:1403] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.005s
I20260709 12:15:41.859391 31149 ts_tablet_manager.cc:1434] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:15:41.860323 31149 tablet_bootstrap.cc:492] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:41.860522 31152 raft_consensus.cc:493] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.860982 31152 raft_consensus.cc:515] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.864104 31152 leader_election.cc:290] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:41.868709 31150 tablet_bootstrap.cc:492] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:41.869084 31150 ts_tablet_manager.cc:1403] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.005s
I20260709 12:15:41.871253 31147 raft_consensus.cc:359] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.872375 31147 raft_consensus.cc:385] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.872303 31150 raft_consensus.cc:359] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.873128 31150 raft_consensus.cc:385] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.872926 31147 raft_consensus.cc:740] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.873488 31150 raft_consensus.cc:740] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.874236 31150 consensus_queue.cc:260] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.874498 31147 consensus_queue.cc:260] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.878296 31150 ts_tablet_manager.cc:1434] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:15:41.879262 31150 tablet_bootstrap.cc:492] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:41.883229 31153 raft_consensus.cc:493] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.883747 31153 raft_consensus.cc:515] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.887562 31153 leader_election.cc:290] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:41.890235 31150 tablet_bootstrap.cc:654] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.891741 31149 tablet_bootstrap.cc:654] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.885468 31147 ts_tablet_manager.cc:1434] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.003s
I20260709 12:15:41.906689 31147 tablet_bootstrap.cc:492] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:41.919728 31147 tablet_bootstrap.cc:654] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.932544 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
W20260709 12:15:41.934796 30996 leader_election.cc:343] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:41.936372 31150 tablet_bootstrap.cc:492] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:41.936884 31150 ts_tablet_manager.cc:1403] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.020s
I20260709 12:15:41.940119 31150 raft_consensus.cc:359] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.941037 31150 raft_consensus.cc:385] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.941413 31150 raft_consensus.cc:740] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.942090 31150 consensus_queue.cc:260] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.945185 31149 tablet_bootstrap.cc:492] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:41.945684 31149 ts_tablet_manager.cc:1403] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.086s	user 0.001s	sys 0.015s
I20260709 12:15:41.946924 31150 ts_tablet_manager.cc:1434] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:15:41.947923 31150 tablet_bootstrap.cc:492] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:41.948670 31149 raft_consensus.cc:359] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.949394 31149 raft_consensus.cc:385] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:41.949784 31149 raft_consensus.cc:740] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:41.954667 31149 consensus_queue.cc:260] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.959424 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
W20260709 12:15:41.962280 31070 leader_election.cc:343] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:41.965885 31149 ts_tablet_manager.cc:1434] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:15:41.967152 31149 tablet_bootstrap.cc:492] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:41.968621 31153 raft_consensus.cc:493] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.969096 31153 raft_consensus.cc:515] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.971225 31153 leader_election.cc:290] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:41.972183 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:41.973464 30955 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 0.
I20260709 12:15:41.975150 30996 leader_election.cc:304] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:41.976078 31150 tablet_bootstrap.cc:654] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.976895 31153 raft_consensus.cc:2804] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:41.977254 31153 raft_consensus.cc:493] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.977826 31153 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:41.994773 31147 tablet_bootstrap.cc:492] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:41.994942 31153 raft_consensus.cc:515] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:41.995846 31147 ts_tablet_manager.cc:1403] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.018s
I20260709 12:15:41.996215 31150 tablet_bootstrap.cc:492] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:41.996917 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:41.997740 31150 ts_tablet_manager.cc:1403] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.005s
W20260709 12:15:41.998821 31070 leader_election.cc:343] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:42.000066 31070 leader_election.cc:304] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d; no voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.000908 31152 raft_consensus.cc:2749] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:42.001000 31147 raft_consensus.cc:359] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.001772 31147 raft_consensus.cc:385] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.001875 31149 tablet_bootstrap.cc:654] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:41.999086 31153 leader_election.cc:290] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:41.999573 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.003150 30955 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.005302 31150 raft_consensus.cc:359] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.006002 31150 raft_consensus.cc:385] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.006283 31150 raft_consensus.cc:740] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.002104 31147 raft_consensus.cc:740] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.007002 31104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.007627 31147 consensus_queue.cc:260] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.009903 31147 ts_tablet_manager.cc:1434] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:15:42.010864 31147 tablet_bootstrap.cc:492] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:42.014712 31104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.017066 31104 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 0.
I20260709 12:15:42.017309 31147 tablet_bootstrap.cc:654] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.015707 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.018450 31103 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.007035 31150 consensus_queue.cc:260] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.020650 31150 ts_tablet_manager.cc:1434] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:15:42.021584 31150 tablet_bootstrap.cc:492] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
W20260709 12:15:42.016244 30993 leader_election.cc:343] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:42.022828 30993 leader_election.cc:304] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1; no voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.023607 31153 raft_consensus.cc:2749] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:42.027284 31103 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 1.
I20260709 12:15:42.028745 31150 tablet_bootstrap.cc:654] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.029321 30993 leader_election.cc:304] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.030113 31153 raft_consensus.cc:2804] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.014297 30955 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 1.
I20260709 12:15:42.034673 31153 raft_consensus.cc:697] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:42.035897 31149 tablet_bootstrap.cc:492] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:42.035529 31153 consensus_queue.cc:237] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.036360 31149 ts_tablet_manager.cc:1403] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.001s
I20260709 12:15:42.038676 31149 raft_consensus.cc:359] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.039203 31149 raft_consensus.cc:385] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.039454 31149 raft_consensus.cc:740] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.040078 31149 consensus_queue.cc:260] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.042227 31174 raft_consensus.cc:493] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.042706 31174 raft_consensus.cc:515] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.046121 31147 tablet_bootstrap.cc:492] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:42.046620 31147 ts_tablet_manager.cc:1403] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.036s	user 0.002s	sys 0.010s
I20260709 12:15:42.047438 31174 leader_election.cc:290] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.048872 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eafbfc73a149949ff328170fd420d9" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.049642 30955 raft_consensus.cc:2468] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 0.
I20260709 12:15:42.049998 31147 raft_consensus.cc:359] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.050779 31147 raft_consensus.cc:385] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.051079 31147 raft_consensus.cc:740] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.051254 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eafbfc73a149949ff328170fd420d9" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.052030 31103 raft_consensus.cc:2468] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 0.
I20260709 12:15:42.051815 31147 consensus_queue.cc:260] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.053671 30993 leader_election.cc:304] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.054425 31147 ts_tablet_manager.cc:1434] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:15:42.061951 31150 tablet_bootstrap.cc:492] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:42.062496 31150 ts_tablet_manager.cc:1403] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.003s
I20260709 12:15:42.062135 31178 raft_consensus.cc:2804] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.062963 31178 raft_consensus.cc:493] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.065590 31150 raft_consensus.cc:359] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.066354 31150 raft_consensus.cc:385] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.066686 31150 raft_consensus.cc:740] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.065996 31178 raft_consensus.cc:3060] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.067449 31150 consensus_queue.cc:260] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.055383 31147 tablet_bootstrap.cc:492] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:42.069550 31149 ts_tablet_manager.cc:1434] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.004s
I20260709 12:15:42.071099 31149 tablet_bootstrap.cc:492] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:42.073104 31150 ts_tablet_manager.cc:1434] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:15:42.074075 31150 tablet_bootstrap.cc:492] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:42.077427 30851 catalog_manager.cc:5697] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.080899 31147 tablet_bootstrap.cc:654] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.086382 31149 tablet_bootstrap.cc:654] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.096349 31150 tablet_bootstrap.cc:654] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.097599 31178 raft_consensus.cc:515] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.099682 31149 tablet_bootstrap.cc:492] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:42.100412 31149 ts_tablet_manager.cc:1403] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.030s	user 0.004s	sys 0.010s
I20260709 12:15:42.100836 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eafbfc73a149949ff328170fd420d9" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.101446 30955 raft_consensus.cc:3060] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.108215 30955 raft_consensus.cc:2468] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 1.
I20260709 12:15:42.108040 31149 raft_consensus.cc:359] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.109144 31149 raft_consensus.cc:385] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.109582 31149 raft_consensus.cc:740] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.109696 30996 leader_election.cc:304] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.110569 31174 raft_consensus.cc:2804] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.111100 31174 raft_consensus.cc:697] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:42.110703 31149 consensus_queue.cc:260] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.111963 31174 consensus_queue.cc:237] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.113171 31149 ts_tablet_manager.cc:1434] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:15:42.116315 31149 tablet_bootstrap.cc:492] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:42.118567 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eafbfc73a149949ff328170fd420d9" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.119144 31103 raft_consensus.cc:3060] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.125809 31103 raft_consensus.cc:2468] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 1.
I20260709 12:15:42.126144 31149 tablet_bootstrap.cc:654] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.134268 31178 leader_election.cc:290] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.139389 31147 tablet_bootstrap.cc:492] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:42.140026 31147 ts_tablet_manager.cc:1403] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.020s
I20260709 12:15:42.141402 31150 tablet_bootstrap.cc:492] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:42.141891 31150 ts_tablet_manager.cc:1403] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.013s
I20260709 12:15:42.143693 31147 raft_consensus.cc:359] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.144616 31147 raft_consensus.cc:385] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.144919 31147 raft_consensus.cc:740] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.145459 31150 raft_consensus.cc:359] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.146186 31150 raft_consensus.cc:385] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.145817 31147 consensus_queue.cc:260] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.146512 31150 raft_consensus.cc:740] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.147308 31150 consensus_queue.cc:260] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.150568 31150 ts_tablet_manager.cc:1434] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:15:42.151541 31150 tablet_bootstrap.cc:492] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:42.155756 31147 ts_tablet_manager.cc:1434] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:15:42.156909 31147 tablet_bootstrap.cc:492] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:42.158247 31149 tablet_bootstrap.cc:492] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:42.158746 31149 ts_tablet_manager.cc:1403] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.007s
I20260709 12:15:42.161759 31149 raft_consensus.cc:359] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.162503 31149 raft_consensus.cc:385] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.162803 31149 raft_consensus.cc:740] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.163450 31147 tablet_bootstrap.cc:654] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.163550 31149 consensus_queue.cc:260] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.165930 31149 ts_tablet_manager.cc:1434] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:15:42.166863 31149 tablet_bootstrap.cc:492] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:42.174232 31150 tablet_bootstrap.cc:654] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.178901 31149 tablet_bootstrap.cc:654] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.183094 30852 catalog_manager.cc:5697] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.189286 31150 tablet_bootstrap.cc:492] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:42.189775 31150 ts_tablet_manager.cc:1403] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.003s
I20260709 12:15:42.192324 31147 tablet_bootstrap.cc:492] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:42.196902 31149 tablet_bootstrap.cc:492] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:42.197362 31149 ts_tablet_manager.cc:1403] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 12:15:42.192626 31150 raft_consensus.cc:359] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.198764 31150 raft_consensus.cc:385] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.199137 31150 raft_consensus.cc:740] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.199976 31150 consensus_queue.cc:260] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.192775 31147 ts_tablet_manager.cc:1403] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.008s
I20260709 12:15:42.202970 31149 raft_consensus.cc:359] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.203519 31149 raft_consensus.cc:385] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.203750 31149 raft_consensus.cc:740] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.204303 31149 consensus_queue.cc:260] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.204988 31152 raft_consensus.cc:493] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.204618 31147 raft_consensus.cc:359] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.205672 31147 raft_consensus.cc:385] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.205453 31152 raft_consensus.cc:515] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.206005 31147 raft_consensus.cc:740] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.207000 31147 consensus_queue.cc:260] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.207463 31152 leader_election.cc:290] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.208400 31150 ts_tablet_manager.cc:1434] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:15:42.209393 31150 tablet_bootstrap.cc:492] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:42.209596 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.209914 31147 ts_tablet_manager.cc:1434] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:15:42.210184 31030 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.210855 31147 tablet_bootstrap.cc:492] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:42.215906 31150 tablet_bootstrap.cc:654] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.220507 31070 leader_election.cc:304] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.217015 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.222734 30955 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.226868 31147 tablet_bootstrap.cc:654] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.229771 31152 raft_consensus.cc:2804] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.237594 31152 raft_consensus.cc:493] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.238065 31152 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.234921 31150 tablet_bootstrap.cc:492] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:42.242337 31150 ts_tablet_manager.cc:1403] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.006s
I20260709 12:15:42.243606 31193 raft_consensus.cc:493] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.244191 31193 raft_consensus.cc:515] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.245376 31150 raft_consensus.cc:359] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.246042 31150 raft_consensus.cc:385] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.246346 31150 raft_consensus.cc:740] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.247022 31150 consensus_queue.cc:260] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.247604 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.249600 31150 ts_tablet_manager.cc:1434] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:15:42.251114 31157 raft_consensus.cc:493] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.253878 31157 raft_consensus.cc:515] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.265328 31147 tablet_bootstrap.cc:492] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:42.265627 31157 leader_election.cc:290] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.265828 31147 ts_tablet_manager.cc:1403] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.009s
I20260709 12:15:42.267760 31150 tablet_bootstrap.cc:492] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: Bootstrap starting.
I20260709 12:15:42.268373 31147 raft_consensus.cc:359] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.269042 31147 raft_consensus.cc:385] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.269531 31147 raft_consensus.cc:740] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.270212 31147 consensus_queue.cc:260] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.277527 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.278904 31030 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.281219 31195 raft_consensus.cc:493] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.281734 31195 raft_consensus.cc:515] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.283102 31070 leader_election.cc:304] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.285698 31157 raft_consensus.cc:493] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.286131 31157 raft_consensus.cc:515] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.288138 31157 leader_election.cc:290] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.289403 31198 raft_consensus.cc:2804] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.289795 31198 raft_consensus.cc:493] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.292742 31147 ts_tablet_manager.cc:1434] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.016s
I20260709 12:15:42.293605 31147 tablet_bootstrap.cc:492] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d: Bootstrap starting.
I20260709 12:15:42.296376 31150 tablet_bootstrap.cc:654] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.300614 31147 tablet_bootstrap.cc:654] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.290140 31198 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.308180 30955 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.310415 31193 leader_election.cc:290] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.206120 31149 ts_tablet_manager.cc:1434] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:15:42.312915 31195 leader_election.cc:290] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887)
I20260709 12:15:42.315485 31149 tablet_bootstrap.cc:492] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: Bootstrap starting.
I20260709 12:15:42.319712 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.320428 31030 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 0.
I20260709 12:15:42.322389 30921 leader_election.cc:304] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.323347 31157 raft_consensus.cc:2804] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.323717 31157 raft_consensus.cc:493] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.324069 31157 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.323860 31152 raft_consensus.cc:515] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.325887 31152 leader_election.cc:290] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.328167 31174 raft_consensus.cc:493] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.328634 31174 raft_consensus.cc:515] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.330837 31174 leader_election.cc:290] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.330261 31198 raft_consensus.cc:515] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.331880 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.332574 31030 raft_consensus.cc:2468] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 0.
I20260709 12:15:42.333851 30921 leader_election.cc:304] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.337584 31157 raft_consensus.cc:515] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.343791 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.345422 30955 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 0.
I20260709 12:15:42.346644 30996 leader_election.cc:304] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.346854 31157 leader_election.cc:290] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.347486 31157 raft_consensus.cc:2804] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.347517 31174 raft_consensus.cc:2804] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.348064 31174 raft_consensus.cc:493] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.348052 31157 raft_consensus.cc:493] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.348696 31174 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.349656 31198 leader_election.cc:290] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.351789 31152 raft_consensus.cc:493] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.352427 31152 raft_consensus.cc:515] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.350461 31157 raft_consensus.cc:3060] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:42.350760 31070 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:42661, user_credentials={real_user=slave}} blocked reactor thread for 67428.3us
I20260709 12:15:42.357331 31150 tablet_bootstrap.cc:492] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: No bootstrap required, opened a new log
I20260709 12:15:42.357980 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.358798 31030 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.362704 31150 ts_tablet_manager.cc:1403] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: Time spent bootstrapping tablet: real 0.112s	user 0.020s	sys 0.001s
I20260709 12:15:42.364351 31152 leader_election.cc:290] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.365882 31174 raft_consensus.cc:515] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.367446 31149 tablet_bootstrap.cc:654] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:42.374780 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.368599 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.370661 31157 raft_consensus.cc:515] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.377324 31147 tablet_bootstrap.cc:492] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d: No bootstrap required, opened a new log
I20260709 12:15:42.377703 31147 ts_tablet_manager.cc:1403] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d: Time spent bootstrapping tablet: real 0.084s	user 0.027s	sys 0.027s
I20260709 12:15:42.379271 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.379801 31030 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.381343 31147 raft_consensus.cc:359] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.381974 31147 raft_consensus.cc:385] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.378296 30954 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.382757 31157 leader_election.cc:290] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.382848 31174 leader_election.cc:290] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.381775 31029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.382351 31147 raft_consensus.cc:740] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.387274 31029 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.386787 31147 consensus_queue.cc:260] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.387713 30955 raft_consensus.cc:2393] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b6b1f8f62214e779396d4da64de242d in current term 1: Already voted for candidate 22d14ec1ba314b3296094fb24350578d in this term.
I20260709 12:15:42.384346 31157 raft_consensus.cc:493] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.387274 30953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.410707 30953 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.410638 31157 raft_consensus.cc:515] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
W20260709 12:15:42.399413 30900 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:42.400568 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.413391 31157 leader_election.cc:290] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.413802 31028 raft_consensus.cc:3060] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.397923 30952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.403982 31150 raft_consensus.cc:359] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.397415 30954 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 1.
I20260709 12:15:42.414529 31150 raft_consensus.cc:385] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.414963 31150 raft_consensus.cc:740] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.406164 31147 ts_tablet_manager.cc:1434] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.006s
I20260709 12:15:42.414561 30952 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.415691 31150 consensus_queue.cc:260] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.415943 31027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.416635 31027 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.417716 31150 ts_tablet_manager.cc:1434] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.002s
I20260709 12:15:42.417716 31104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.418511 31104 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.418659 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.421324 31152 raft_consensus.cc:493] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.421759 31152 raft_consensus.cc:515] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.423578 31027 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 1.
I20260709 12:15:42.425154 31104 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 1.
I20260709 12:15:42.425758 31149 tablet_bootstrap.cc:492] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: No bootstrap required, opened a new log
I20260709 12:15:42.426187 31149 ts_tablet_manager.cc:1403] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.029s
I20260709 12:15:42.426888 31070 leader_election.cc:304] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.427791 31199 raft_consensus.cc:2804] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.428277 31199 raft_consensus.cc:697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:42.428886 31149 raft_consensus.cc:359] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.429097 31199 consensus_queue.cc:237] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.429756 31149 raft_consensus.cc:385] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:42.430038 31149 raft_consensus.cc:740] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Initialized, Role: FOLLOWER
I20260709 12:15:42.423643 31152 leader_election.cc:290] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.409533 31030 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 1.
I20260709 12:15:42.430876 31149 consensus_queue.cc:260] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.434984 31149 ts_tablet_manager.cc:1434] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:15:42.437157 31027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.437764 31027 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 0.
I20260709 12:15:42.440035 30921 leader_election.cc:304] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.440395 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.440847 31157 raft_consensus.cc:2804] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.441205 31157 raft_consensus.cc:493] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.441524 31157 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.443066 31026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.448138 31026 raft_consensus.cc:2393] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b6b1f8f62214e779396d4da64de242d in current term 1: Already voted for candidate 0761f3b2907f426189bc51a65f60bec1 in this term.
I20260709 12:15:42.442858 31028 raft_consensus.cc:2468] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 1.
I20260709 12:15:42.452960 30952 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 1.
I20260709 12:15:42.452960 31029 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 1.
I20260709 12:15:42.455767 31157 raft_consensus.cc:515] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.459187 31029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.459805 31029 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.460187 30952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d" is_pre_election: true
I20260709 12:15:42.460832 30952 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.461467 30953 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 1.
I20260709 12:15:42.462814 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1" is_pre_election: true
I20260709 12:15:42.463186 30921 leader_election.cc:304] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.463438 31028 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 0.
I20260709 12:15:42.462467 30993 leader_election.cc:304] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.464079 30921 leader_election.cc:304] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.464116 31202 raft_consensus.cc:2804] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.464918 31104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.465533 31174 raft_consensus.cc:2804] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.465860 31102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.466166 31174 raft_consensus.cc:697] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:42.466663 31100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.467317 31100 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 0.
I20260709 12:15:42.467378 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.467137 31174 consensus_queue.cc:237] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.467971 31101 raft_consensus.cc:3060] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.465569 31104 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 0.
I20260709 12:15:42.468830 31100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.469192 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6b1f8f62214e779396d4da64de242d" is_pre_election: true
I20260709 12:15:42.469362 31100 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.475521 31101 raft_consensus.cc:2468] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 1.
I20260709 12:15:42.476444 31100 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 1.
I20260709 12:15:42.476905 31202 raft_consensus.cc:697] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:42.478453 31202 consensus_queue.cc:237] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.477041 31206 raft_consensus.cc:2804] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.477208 31070 leader_election.cc:304] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.482441 31070 leader_election.cc:304] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.483753 31152 raft_consensus.cc:2804] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.483088 31199 raft_consensus.cc:2804] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.485306 31152 raft_consensus.cc:697] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:42.486552 31070 leader_election.cc:304] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:42.486253 31152 consensus_queue.cc:237] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.486394 30852 catalog_manager.cc:5697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 1 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:42.487442 31198 raft_consensus.cc:2804] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:42.487846 31198 raft_consensus.cc:493] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.488288 31198 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.485357 31199 raft_consensus.cc:493] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.493559 31070 leader_election.cc:304] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d; no voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.459332 31157 leader_election.cc:290] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.495661 31199 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.499449 30918 leader_election.cc:304] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.498994 31195 raft_consensus.cc:2749] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:42.483461 30851 catalog_manager.cc:5697] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.501096 31198 raft_consensus.cc:515] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.508460 31198 leader_election.cc:290] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:42.507035 31157 raft_consensus.cc:2804] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.507395 31206 raft_consensus.cc:697] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:42.511633 31157 raft_consensus.cc:697] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:42.513664 31157 consensus_queue.cc:237] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.517670 31174 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:42.512205 31199 raft_consensus.cc:515] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.514178 31029 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 1.
I20260709 12:15:42.513849 31206 consensus_queue.cc:237] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.520785 31199 leader_election.cc:290] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887)
I20260709 12:15:42.530704 31174 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:42.531860 30952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.532452 30952 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.539808 30952 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 1.
I20260709 12:15:42.541620 31029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.542166 31029 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:42.547686 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.548481 31030 raft_consensus.cc:2393] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b6b1f8f62214e779396d4da64de242d in current term 1: Already voted for candidate 22d14ec1ba314b3296094fb24350578d in this term.
I20260709 12:15:42.550410 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:42.552294 31070 leader_election.cc:304] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:42.553081 31199 raft_consensus.cc:2804] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:42.553212 31070 leader_election.cc:304] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d; no voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.553622 31199 raft_consensus.cc:697] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:42.554237 31198 raft_consensus.cc:2749] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:42.554557 31199 consensus_queue.cc:237] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.557034 30852 catalog_manager.cc:5697] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 1 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:42.559227 31029 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 1.
I20260709 12:15:42.568274 31174 consensus_queue.cc:1048] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:42.581863 30851 catalog_manager.cc:5697] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 1 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:42.641718 30851 catalog_manager.cc:5697] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 0 to 1, leader changed from <none> to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 1 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.643172 30851 catalog_manager.cc:5697] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 0 to 1, leader changed from <none> to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 1 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.644428 30851 catalog_manager.cc:5697] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 0 to 1, leader changed from <none> to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 1 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.657342 31174 consensus_queue.cc:1048] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:42.694487 31100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
mode: GRACEFUL
new_leader_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:41360
I20260709 12:15:42.695315 31100 raft_consensus.cc:606] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Received request to transfer leadership to TS 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.700623 31100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
mode: GRACEFUL
new_leader_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:41360
I20260709 12:15:42.701264 31100 raft_consensus.cc:606] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Received request to transfer leadership to TS 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.715178 31029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:51330
I20260709 12:15:42.715693 31029 raft_consensus.cc:606] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Received request to transfer leadership to TS 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:42.729871 30952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:43426
I20260709 12:15:42.730536 30952 raft_consensus.cc:606] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:42.735445 31100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:41360
I20260709 12:15:42.735965 31100 raft_consensus.cc:606] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:42.745865 30952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
mode: GRACEFUL
new_leader_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:43426
I20260709 12:15:42.746505 30952 raft_consensus.cc:606] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Received request to transfer leadership to TS 1b6b1f8f62214e779396d4da64de242d
I20260709 12:15:42.953032 31199 consensus_queue.cc:1048] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:42.955372 31199 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:42.963732 31152 raft_consensus.cc:993] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: : Instructing follower 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:42.964221 31198 raft_consensus.cc:1081] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Signalling peer 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:42.966490 30953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:43410
I20260709 12:15:42.967077 30953 raft_consensus.cc:493] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:42.967424 30953 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:42.970511 31152 raft_consensus.cc:993] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:42.970945 31195 raft_consensus.cc:1081] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:42.974586 31030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51312
I20260709 12:15:42.975122 31030 raft_consensus.cc:493] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:42.975468 31030 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:42.975814 30953 raft_consensus.cc:515] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.978547 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:42.978721 30953 leader_election.cc:290] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 2 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:42.979148 31028 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:42.979719 31174 consensus_queue.cc:1048] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:42.986550 31100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:42.987151 31100 raft_consensus.cc:3055] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:42.987491 31100 raft_consensus.cc:740] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:42.988299 31100 consensus_queue.cc:260] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:42.989324 31100 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:42.992012 31028 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 2.
I20260709 12:15:42.994706 31029 raft_consensus.cc:1240] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Rejecting Update request from peer 1b6b1f8f62214e779396d4da64de242d for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:42.995735 31198 consensus_queue.cc:1048] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:42.998067 31100 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 2.
I20260709 12:15:43.000818 31157 consensus_queue.cc:1048] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.003135 31157 consensus_queue.cc:1048] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.004429 31030 raft_consensus.cc:515] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.004113 30918 leader_election.cc:304] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:43.005589 31206 raft_consensus.cc:2804] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:43.005846 31026 raft_consensus.cc:1240] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Rejecting Update request from peer 1b6b1f8f62214e779396d4da64de242d for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:43.006045 31206 raft_consensus.cc:697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:43.007570 31206 consensus_queue.cc:237] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.011036 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:43.011730 30954 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.022727 31178 raft_consensus.cc:993] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:43.023198 31178 raft_consensus.cc:1081] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Signalling peer 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:43.023710 30952 raft_consensus.cc:1240] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Rejecting Update request from peer 1b6b1f8f62214e779396d4da64de242d for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:43.028657 30952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:43424
I20260709 12:15:43.029292 30952 raft_consensus.cc:493] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:43.029536 30954 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 2.
I20260709 12:15:43.029742 30952 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.030380 30955 raft_consensus.cc:1240] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Rejecting Update request from peer 1b6b1f8f62214e779396d4da64de242d for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:15:43.031883 31206 consensus_queue.cc:1048] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.033295 31199 consensus_queue.cc:1059] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.034087 31198 raft_consensus.cc:3055] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:43.034219 30996 leader_election.cc:304] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:43.034475 31198 raft_consensus.cc:740] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:43.034626 31030 leader_election.cc:290] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 2 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:43.035110 31153 raft_consensus.cc:2804] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:43.035346 31198 consensus_queue.cc:260] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.035697 31153 raft_consensus.cc:697] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:43.036196 31198 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.036576 31153 consensus_queue.cc:237] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.045346 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:43.051200 30852 catalog_manager.cc:5697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 1 to 2, leader changed from 1b6b1f8f62214e779396d4da64de242d (127.30.25.3) to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 2 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:43.052507 31103 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 2.
I20260709 12:15:43.054508 31153 consensus_queue.cc:1048] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.056460 31198 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:43.069350 31202 consensus_queue.cc:1048] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:43.069742 31206 consensus_queue.cc:1048] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.103479 31206 consensus_queue.cc:1048] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:43.108419 30952 raft_consensus.cc:515] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.110450 31202 raft_consensus.cc:993] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:43.110926 31202 raft_consensus.cc:1081] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:43.115231 31030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51328
I20260709 12:15:43.115782 31030 raft_consensus.cc:493] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:43.116148 31030 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.122785 31030 raft_consensus.cc:515] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.123826 31027 raft_consensus.cc:1240] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Rejecting Update request from peer 22d14ec1ba314b3296094fb24350578d for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:43.126060 31202 consensus_queue.cc:1059] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.126921 31202 raft_consensus.cc:3055] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:43.127240 31202 raft_consensus.cc:740] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:43.128024 31202 consensus_queue.cc:260] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.128916 31202 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.134793 31195 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.143669 31030 leader_election.cc:290] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 2 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:43.144038 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:43.159341 30954 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 2.
I20260709 12:15:43.162586 30996 leader_election.cc:304] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:43.163594 31178 raft_consensus.cc:2804] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:43.164080 31178 raft_consensus.cc:697] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:43.164227 31100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:43.164815 31100 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.164921 31178 consensus_queue.cc:237] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.172147 31100 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 2.
I20260709 12:15:43.105522 30851 catalog_manager.cc:5697] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 1 to 2, leader changed from 1b6b1f8f62214e779396d4da64de242d (127.30.25.3) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:43.175295 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:43.176038 31030 raft_consensus.cc:3055] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:43.176411 31030 raft_consensus.cc:740] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:43.180591 30952 leader_election.cc:290] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 2 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:43.182991 31199 raft_consensus.cc:993] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: : Instructing follower 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:43.183418 31152 raft_consensus.cc:1081] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Signalling peer 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:43.185801 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:43.186393 31101 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.185961 30952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:43410
I20260709 12:15:43.187100 30952 raft_consensus.cc:493] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:43.187427 31206 raft_consensus.cc:993] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: : Instructing follower 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:43.188643 30952 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.188766 31206 raft_consensus.cc:1081] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Signalling peer 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:43.192804 31101 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 2.
I20260709 12:15:43.193838 30918 leader_election.cc:304] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:43.194723 31206 raft_consensus.cc:2804] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:43.195174 31206 raft_consensus.cc:697] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:43.195932 31206 consensus_queue.cc:237] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.196321 30952 raft_consensus.cc:515] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.199072 30952 leader_election.cc:290] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 2 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:43.199180 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:43.199733 31028 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.201998 31103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:41356
I20260709 12:15:43.202569 31103 raft_consensus.cc:493] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:43.202945 31103 raft_consensus.cc:3060] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.208248 31028 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 2.
I20260709 12:15:43.209383 31103 raft_consensus.cc:515] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.177345 31030 consensus_queue.cc:260] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.210592 31030 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.211395 31103 leader_election.cc:290] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 2 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:43.213490 30952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:43.214119 30952 raft_consensus.cc:3055] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:43.214546 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:43.214900 31101 raft_consensus.cc:1240] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Rejecting Update request from peer 22d14ec1ba314b3296094fb24350578d for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:43.215183 31103 raft_consensus.cc:3055] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:43.215521 31103 raft_consensus.cc:740] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
W20260709 12:15:43.214989 31178 consensus_queue.cc:1175] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1b6b1f8f62214e779396d4da64de242d" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:15:43.214541 30952 raft_consensus.cc:740] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:43.216647 30921 leader_election.cc:304] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:43.216917 30952 consensus_queue.cc:260] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.218046 30952 raft_consensus.cc:3060] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.222847 31030 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 2.
I20260709 12:15:43.225279 30952 raft_consensus.cc:2468] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 2.
I20260709 12:15:43.226454 31070 leader_election.cc:304] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:43.222518 31202 raft_consensus.cc:2804] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:43.227234 31152 raft_consensus.cc:2804] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:43.227624 31202 raft_consensus.cc:697] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:43.227726 31152 raft_consensus.cc:697] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:43.228299 30849 catalog_manager.cc:5697] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 1 to 2, leader changed from 22d14ec1ba314b3296094fb24350578d (127.30.25.1) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:43.228605 31202 consensus_queue.cc:237] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.228587 31152 consensus_queue.cc:237] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.228657 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e730ee7c64029952254b76a7a7a01" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:43.231415 31028 raft_consensus.cc:3060] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.237097 31103 consensus_queue.cc:260] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:43.238358 31103 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:43.241026 30851 catalog_manager.cc:5697] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 1 to 2, leader changed from 22d14ec1ba314b3296094fb24350578d (127.30.25.1) to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 2 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:43.244765 31103 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 2.
I20260709 12:15:43.249526 31028 raft_consensus.cc:2468] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 2.
I20260709 12:15:43.252504 30852 catalog_manager.cc:5697] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 1 to 2, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 2 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:43.266515 30852 catalog_manager.cc:5697] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 1 to 2, leader changed from 1b6b1f8f62214e779396d4da64de242d (127.30.25.3) to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 2 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:43.485909 31028 raft_consensus.cc:1275] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.487334 31202 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.500600 31103 raft_consensus.cc:1275] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.502111 31206 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.529747 31103 raft_consensus.cc:1275] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.531347 31178 consensus_queue.cc:1048] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.540652 30952 raft_consensus.cc:1275] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.542831 31153 consensus_queue.cc:1048] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.577617 31101 raft_consensus.cc:1275] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.579667 31153 consensus_queue.cc:1048] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:43.590193 30952 raft_consensus.cc:1275] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.591707 31178 consensus_queue.cc:1048] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:43.599189 31101 raft_consensus.cc:1275] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.600639 31206 consensus_queue.cc:1048] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.610507 31028 raft_consensus.cc:1275] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.611784 31206 consensus_queue.cc:1048] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.661988 31028 raft_consensus.cc:1275] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.663363 31206 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.665992 30952 raft_consensus.cc:1275] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.668177 31152 consensus_queue.cc:1048] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.674618 31101 raft_consensus.cc:1275] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.678234 31157 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:43.681666 31028 raft_consensus.cc:1275] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:43.684981 31199 consensus_queue.cc:1048] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.776140 30820 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:15:45.776904 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:45.796816 31028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c6976d88606429d928286415c52bec5"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:51346
I20260709 12:15:45.797293 31028 raft_consensus.cc:606] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Received request to transfer leadership to TS 1b6b1f8f62214e779396d4da64de242d
I20260709 12:15:45.807718 30952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
mode: GRACEFUL
new_leader_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:43440
I20260709 12:15:45.808248 30952 raft_consensus.cc:606] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Received request to transfer leadership to TS 1b6b1f8f62214e779396d4da64de242d
I20260709 12:15:45.809245 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:45.809705 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:46.168975 31243 raft_consensus.cc:993] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: : Instructing follower 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:46.169447 31238 raft_consensus.cc:1081] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Signalling peer 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:46.170796 31101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:41356
I20260709 12:15:46.171213 31101 raft_consensus.cc:493] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.171469 31101 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.176151 31101 raft_consensus.cc:515] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:46.177589 31101 leader_election.cc:290] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 2 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887)
I20260709 12:15:46.178434 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:46.178674 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:46.179028 31028 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.179224 30955 raft_consensus.cc:3055] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.179553 30955 raft_consensus.cc:740] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:46.180255 30955 consensus_queue.cc:260] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:46.181209 30955 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.184716 31028 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 2.
I20260709 12:15:46.185806 31070 leader_election.cc:304] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:46.185950 30955 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 2.
I20260709 12:15:46.186488 31237 raft_consensus.cc:2804] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.187716 31237 raft_consensus.cc:697] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:46.188566 31237 consensus_queue.cc:237] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:46.195165 30849 catalog_manager.cc:5697] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 1 to 2, leader changed from 22d14ec1ba314b3296094fb24350578d (127.30.25.1) to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 2 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:46.353681 31239 raft_consensus.cc:993] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:46.354092 31250 raft_consensus.cc:1081] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Signalling peer 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:46.355553 31101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c6976d88606429d928286415c52bec5"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:41354
I20260709 12:15:46.356104 31101 raft_consensus.cc:493] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.356498 31101 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.361621 31101 raft_consensus.cc:515] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:46.363204 31101 leader_election.cc:290] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 2 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:46.364084 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:46.364382 31028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:46.364665 30955 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.364977 31028 raft_consensus.cc:3055] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.365332 31028 raft_consensus.cc:740] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:46.366103 31028 consensus_queue.cc:260] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:46.367192 31028 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.369572 30955 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 2.
I20260709 12:15:46.370558 31070 leader_election.cc:304] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:46.371191 31237 raft_consensus.cc:2804] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.371697 31237 raft_consensus.cc:697] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:46.371742 31028 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 2.
I20260709 12:15:46.372608 31237 consensus_queue.cc:237] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:46.379791 30849 catalog_manager.cc:5697] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 1 to 2, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 2 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:46.702729 30955 raft_consensus.cc:1275] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.704062 31237 consensus_queue.cc:1048] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.714417 31028 raft_consensus.cc:1275] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.716059 31237 consensus_queue.cc:1048] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.795442 30955 raft_consensus.cc:1275] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.796607 31254 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.804988 31028 raft_consensus.cc:1275] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.807677 31244 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.810716 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:46.818958 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:46.819463 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:47.834039 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:47.843416 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:47.844187 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:15:48.766147 30883 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 30823
W20260709 12:15:48.766937 30883 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 30886
W20260709 12:15:48.767747 30883 debug-util.cc:398] Leaking SignalData structure 0x7b0800158b60 after lost signal to thread 30979
W20260709 12:15:48.768512 30883 debug-util.cc:398] Leaking SignalData structure 0x7b08001f3500 after lost signal to thread 31054
W20260709 12:15:48.769183 30883 debug-util.cc:398] Leaking SignalData structure 0x7b080017eaa0 after lost signal to thread 31128
I20260709 12:15:48.845424 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:48.851361 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:48.851954 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:49.853124 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:49.863623 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:49.864194 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:51.029904 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:51.035355 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:51.035780 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:15:51.519708 31125 debug-util.cc:398] Leaking SignalData structure 0x7b080016d700 after lost signal to thread 30823
W20260709 12:15:51.520576 31125 debug-util.cc:398] Leaking SignalData structure 0x7b080016dde0 after lost signal to thread 30886
W20260709 12:15:51.521428 31125 debug-util.cc:398] Leaking SignalData structure 0x7b080015b2c0 after lost signal to thread 30979
W20260709 12:15:51.522203 31125 debug-util.cc:398] Leaking SignalData structure 0x7b08000cdbe0 after lost signal to thread 31054
W20260709 12:15:51.522989 31125 debug-util.cc:398] Leaking SignalData structure 0x7b08001eeec0 after lost signal to thread 31128
I20260709 12:15:52.036674 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:52.041386 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:52.041806 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:53.042732 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:53.047873 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:53.048290 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:54.049212 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:54.054095 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:54.054535 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:55.055497 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:55.061000 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:55.061450 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:56.090055 30820 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:15:56.095053 31103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:41360
I20260709 12:15:56.095747 31103 raft_consensus.cc:606] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:56.101079 30952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:43426
I20260709 12:15:56.101655 30952 raft_consensus.cc:606] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:56.107371 30952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:43426
I20260709 12:15:56.108022 30952 raft_consensus.cc:606] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:56.112989 30952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:43426
I20260709 12:15:56.113483 30952 raft_consensus.cc:606] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:56.125207 31103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c6976d88606429d928286415c52bec5"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
mode: GRACEFUL
new_leader_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:41360
I20260709 12:15:56.125718 31103 raft_consensus.cc:606] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Received request to transfer leadership to TS 0761f3b2907f426189bc51a65f60bec1
I20260709 12:15:56.409091 31277 raft_consensus.cc:993] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.409601 31273 raft_consensus.cc:1081] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.411065 31028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c6976d88606429d928286415c52bec5"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51312
I20260709 12:15:56.411635 31028 raft_consensus.cc:493] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:56.412051 31028 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.414252 31277 raft_consensus.cc:993] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.414717 31273 raft_consensus.cc:1081] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.416350 31030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51312
I20260709 12:15:56.416873 31030 raft_consensus.cc:493] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:56.417269 31030 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.418216 31028 raft_consensus.cc:515] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.420467 31028 leader_election.cc:290] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:56.421237 30955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:56.421882 31103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:56.421977 30955 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.422389 31103 raft_consensus.cc:3055] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:56.422642 31103 raft_consensus.cc:740] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:56.423273 31103 consensus_queue.cc:260] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.424117 31103 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.423810 31030 raft_consensus.cc:515] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.426828 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:56.427335 30954 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.428272 31030 leader_election.cc:290] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:56.428313 30955 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.429788 30996 leader_election.cc:304] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:56.430437 31103 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.430787 31276 raft_consensus.cc:2804] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:56.430846 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0e3bffcac84cb3a22d923f6bc3204f" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:56.431533 31101 raft_consensus.cc:3055] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:56.431828 31101 raft_consensus.cc:740] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:56.432546 31276 raft_consensus.cc:697] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:56.432659 31101 consensus_queue.cc:260] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.433847 31101 raft_consensus.cc:3060] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.433684 31276 consensus_queue.cc:237] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.434273 30954 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.435338 30996 leader_election.cc:304] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:56.436223 31278 raft_consensus.cc:2804] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:56.437330 31278 raft_consensus.cc:697] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:56.438210 31278 consensus_queue.cc:237] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.440862 31101 raft_consensus.cc:2468] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.446264 30849 catalog_manager.cc:5697] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 2 to 3, leader changed from 1b6b1f8f62214e779396d4da64de242d (127.30.25.3) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.461870 30849 catalog_manager.cc:5697] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 2 to 3, leader changed from 1b6b1f8f62214e779396d4da64de242d (127.30.25.3) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.476284 31275 raft_consensus.cc:993] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.476696 31275 raft_consensus.cc:1081] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.478024 31030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51328
I20260709 12:15:56.478493 31030 raft_consensus.cc:493] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:56.478776 31030 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.483084 31030 raft_consensus.cc:515] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.484833 31030 leader_election.cc:290] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:56.485648 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:56.485885 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:56.486267 30954 raft_consensus.cc:3055] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:56.486459 31101 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.486660 30954 raft_consensus.cc:740] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:56.487479 30954 consensus_queue.cc:260] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.488770 30954 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.493336 31101 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.494526 30993 leader_election.cc:304] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:56.495102 30954 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.495165 31278 raft_consensus.cc:2804] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:56.495744 31278 raft_consensus.cc:697] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:56.496484 31278 consensus_queue.cc:237] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.503106 30849 catalog_manager.cc:5697] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 2 to 3, leader changed from 22d14ec1ba314b3296094fb24350578d (127.30.25.1) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.534904 31286 raft_consensus.cc:993] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.535372 31286 raft_consensus.cc:1081] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.536738 31030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51328
I20260709 12:15:56.537226 31030 raft_consensus.cc:493] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:56.537565 31030 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.541923 31030 raft_consensus.cc:515] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.543427 31030 leader_election.cc:290] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:56.544257 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:56.544343 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:56.545037 31101 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.545013 30954 raft_consensus.cc:3055] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:56.545549 30954 raft_consensus.cc:740] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:56.546396 30954 consensus_queue.cc:260] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.547613 30954 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.551563 31101 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.552703 30993 leader_election.cc:304] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:56.553406 31278 raft_consensus.cc:2804] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:56.553783 30954 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.553917 31278 raft_consensus.cc:697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:56.554867 31278 consensus_queue.cc:237] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.561678 30849 catalog_manager.cc:5697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 2 to 3, leader changed from 22d14ec1ba314b3296094fb24350578d (127.30.25.1) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.715766 31275 raft_consensus.cc:993] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: : Instructing follower 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.716202 31286 raft_consensus.cc:1081] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Signalling peer 0761f3b2907f426189bc51a65f60bec1 to start an election
I20260709 12:15:56.717639 31030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
 from {username='slave'} at 127.0.0.1:51328
I20260709 12:15:56.718163 31030 raft_consensus.cc:493] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:56.718487 31030 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.722940 31030 raft_consensus.cc:515] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.724741 31030 leader_election.cc:290] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:56.725636 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:56.725577 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c18c4394c9804ff5a2701e1d9fbb6bc2" candidate_uuid: "0761f3b2907f426189bc51a65f60bec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:56.726245 31101 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.726380 30954 raft_consensus.cc:3055] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:56.726708 30954 raft_consensus.cc:740] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:56.727445 30954 consensus_queue.cc:260] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.728562 30954 raft_consensus.cc:3060] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:56.732441 31101 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.733470 30993 leader_election.cc:304] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0761f3b2907f426189bc51a65f60bec1, 1b6b1f8f62214e779396d4da64de242d; no voters: 
I20260709 12:15:56.734110 31278 raft_consensus.cc:2804] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:56.734587 31278 raft_consensus.cc:697] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Leader. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:56.734808 30954 raft_consensus.cc:2468] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0761f3b2907f426189bc51a65f60bec1 in term 3.
I20260709 12:15:56.735388 31278 consensus_queue.cc:237] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:56.741528 30849 catalog_manager.cc:5697] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 reported cstate change: term changed from 2 to 3, leader changed from 22d14ec1ba314b3296094fb24350578d (127.30.25.1) to 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "0761f3b2907f426189bc51a65f60bec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.896339 31101 raft_consensus.cc:1275] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:56.897490 31278 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:56.906994 30954 raft_consensus.cc:1275] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:56.908286 30955 raft_consensus.cc:1275] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:56.909554 31279 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:56.909576 31278 consensus_queue.cc:1048] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:15:56.921165 31101 raft_consensus.cc:1275] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:56.922887 31278 consensus_queue.cc:1048] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:56.945143 31101 raft_consensus.cc:1275] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:56.946273 31278 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:56.954582 30954 raft_consensus.cc:1275] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:56.956017 31279 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:57.002250 30954 raft_consensus.cc:1275] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:57.003571 31279 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:57.012125 31101 raft_consensus.cc:1275] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:57.013708 31279 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:57.170648 31101 raft_consensus.cc:1275] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:57.171826 31279 consensus_queue.cc:1048] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:57.180374 30954 raft_consensus.cc:1275] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Refusing update from remote peer 0761f3b2907f426189bc51a65f60bec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:57.181767 31278 consensus_queue.cc:1048] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:59.158106 30820 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:15:59.158944 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:59.166421 31030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:51346
I20260709 12:15:59.167475 31030 raft_consensus.cc:606] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Received request to transfer leadership to TS 1b6b1f8f62214e779396d4da64de242d
I20260709 12:15:59.170964 31030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:51346
I20260709 12:15:59.171579 31030 raft_consensus.cc:606] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Received request to transfer leadership to TS 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:59.173949 31030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c6976d88606429d928286415c52bec5"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:51346
I20260709 12:15:59.174464 31030 raft_consensus.cc:606] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Received request to transfer leadership to TS 1b6b1f8f62214e779396d4da64de242d
I20260709 12:15:59.176523 31030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:51346
I20260709 12:15:59.176985 31030 raft_consensus.cc:606] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Received request to transfer leadership to TS 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:59.179033 31030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180"
dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
mode: GRACEFUL
new_leader_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:51346
I20260709 12:15:59.179479 31030 raft_consensus.cc:606] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Received request to transfer leadership to TS 22d14ec1ba314b3296094fb24350578d
I20260709 12:15:59.180478 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:59.180939 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:59.338002 31306 raft_consensus.cc:993] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:59.338558 31306 raft_consensus.cc:1081] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Signalling peer 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:59.340070 30954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:43424
I20260709 12:15:59.340631 30954 raft_consensus.cc:493] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.340972 30954 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.346755 30954 raft_consensus.cc:515] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.349189 30954 leader_election.cc:290] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 3 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:59.350356 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:59.350232 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea866411680f4522a261bbc1dfd7f180" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:59.351109 31101 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.351169 31030 raft_consensus.cc:3055] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:59.351620 31030 raft_consensus.cc:740] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:59.352502 31030 consensus_queue.cc:260] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.353909 31030 raft_consensus.cc:3060] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.358741 31101 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 3.
I20260709 12:15:59.360229 30918 leader_election.cc:304] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:59.361624 31030 raft_consensus.cc:2468] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 3.
I20260709 12:15:59.361986 31307 raft_consensus.cc:2804] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:59.363387 31307 raft_consensus.cc:697] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 3 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:59.364305 31307 consensus_queue.cc:237] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.370842 30852 catalog_manager.cc:5697] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 2 to 3, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 3 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.485805 31278 raft_consensus.cc:993] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:59.486234 31278 raft_consensus.cc:1081] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Signalling peer 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:59.487668 31101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:41354
I20260709 12:15:59.488106 31101 raft_consensus.cc:493] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.488381 31101 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.492434 31101 raft_consensus.cc:515] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.494216 31101 leader_election.cc:290] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 3 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:59.495163 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:59.495419 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ff03333b641443a84a40390b03de4f2" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:59.495795 30954 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.495952 31030 raft_consensus.cc:3055] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:59.496299 31030 raft_consensus.cc:740] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:59.496923 31030 consensus_queue.cc:260] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.497821 31030 raft_consensus.cc:3060] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.500985 30954 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 3.
I20260709 12:15:59.502223 31070 leader_election.cc:304] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:59.503118 31304 raft_consensus.cc:2804] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:59.503542 31030 raft_consensus.cc:2468] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 3.
I20260709 12:15:59.504483 31304 raft_consensus.cc:697] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 3 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:59.505393 31304 consensus_queue.cc:237] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.512198 30852 catalog_manager.cc:5697] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 2 to 3, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 3 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:59.524777 31306 raft_consensus.cc:993] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:59.525164 31306 raft_consensus.cc:1081] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Signalling peer 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:59.526595 30954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:43424
I20260709 12:15:59.527057 30954 raft_consensus.cc:493] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.527400 30954 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.531823 30954 raft_consensus.cc:515] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.533422 30954 leader_election.cc:290] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 4 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:59.534235 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:59.534502 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37f44d95ff9f4981ae7f60b2fec9e713" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:59.534930 31030 raft_consensus.cc:3055] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:59.535050 31101 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.535254 31030 raft_consensus.cc:740] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:59.536090 31030 consensus_queue.cc:260] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.536991 31030 raft_consensus.cc:3060] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.539809 31101 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 4.
I20260709 12:15:59.540804 30918 leader_election.cc:304] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:59.541510 31307 raft_consensus.cc:2804] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:59.541972 31307 raft_consensus.cc:697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 4 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:59.542750 31030 raft_consensus.cc:2468] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 4.
I20260709 12:15:59.542904 31307 consensus_queue.cc:237] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.550199 30852 catalog_manager.cc:5697] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 3 to 4, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 4 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.605233 31278 raft_consensus.cc:993] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:59.605705 31306 raft_consensus.cc:1081] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Signalling peer 1b6b1f8f62214e779396d4da64de242d to start an election
I20260709 12:15:59.607434 31101 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c6976d88606429d928286415c52bec5"
dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
 from {username='slave'} at 127.0.0.1:41354
I20260709 12:15:59.607909 31101 raft_consensus.cc:493] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.608196 31101 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.612322 31101 raft_consensus.cc:515] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.613808 31101 leader_election.cc:290] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 4 election: Requested vote from peers 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887), 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661)
I20260709 12:15:59.614687 30954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "22d14ec1ba314b3296094fb24350578d"
I20260709 12:15:59.614977 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6976d88606429d928286415c52bec5" candidate_uuid: "1b6b1f8f62214e779396d4da64de242d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:59.615303 30954 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.615548 31030 raft_consensus.cc:3055] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:59.615850 31030 raft_consensus.cc:740] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:59.616648 31030 consensus_queue.cc:260] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.617832 31030 raft_consensus.cc:3060] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.621490 30954 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 4.
I20260709 12:15:59.622640 31070 leader_election.cc:304] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:59.623402 31304 raft_consensus.cc:2804] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:59.623889 31304 raft_consensus.cc:697] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 4 LEADER]: Becoming Leader. State: Replica: 1b6b1f8f62214e779396d4da64de242d, State: Running, Role: LEADER
I20260709 12:15:59.623986 31030 raft_consensus.cc:2468] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6b1f8f62214e779396d4da64de242d in term 4.
I20260709 12:15:59.624814 31304 consensus_queue.cc:237] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.631868 30852 catalog_manager.cc:5697] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d reported cstate change: term changed from 3 to 4, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 1b6b1f8f62214e779396d4da64de242d (127.30.25.3). New cstate: current_term: 4 leader_uuid: "1b6b1f8f62214e779396d4da64de242d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:59.686650 31278 raft_consensus.cc:993] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: : Instructing follower 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:59.687098 31278 raft_consensus.cc:1081] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Signalling peer 22d14ec1ba314b3296094fb24350578d to start an election
I20260709 12:15:59.688572 30954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7"
dest_uuid: "22d14ec1ba314b3296094fb24350578d"
 from {username='slave'} at 127.0.0.1:43424
I20260709 12:15:59.689075 30954 raft_consensus.cc:493] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.689410 30954 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.693441 30954 raft_consensus.cc:515] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.695051 30954 leader_election.cc:290] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 4 election: Requested vote from peers 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2:42661), 1b6b1f8f62214e779396d4da64de242d (127.30.25.3:36065)
I20260709 12:15:59.695827 31030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0761f3b2907f426189bc51a65f60bec1"
I20260709 12:15:59.696077 31101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578ec24f79e14c028d88586a3cf00fc7" candidate_uuid: "22d14ec1ba314b3296094fb24350578d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1b6b1f8f62214e779396d4da64de242d"
I20260709 12:15:59.696300 31030 raft_consensus.cc:3055] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:59.696599 31101 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.696633 31030 raft_consensus.cc:740] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0761f3b2907f426189bc51a65f60bec1, State: Running, Role: LEADER
I20260709 12:15:59.697693 31030 consensus_queue.cc:260] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.698586 31030 raft_consensus.cc:3060] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.700999 31101 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 4.
I20260709 12:15:59.702091 30918 leader_election.cc:304] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6b1f8f62214e779396d4da64de242d, 22d14ec1ba314b3296094fb24350578d; no voters: 
I20260709 12:15:59.702608 31030 raft_consensus.cc:2468] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22d14ec1ba314b3296094fb24350578d in term 4.
I20260709 12:15:59.703006 31307 raft_consensus.cc:2804] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:59.703523 31307 raft_consensus.cc:697] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 4 LEADER]: Becoming Leader. State: Replica: 22d14ec1ba314b3296094fb24350578d, State: Running, Role: LEADER
I20260709 12:15:59.704253 31307 consensus_queue.cc:237] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } }
I20260709 12:15:59.711120 30852 catalog_manager.cc:5697] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d reported cstate change: term changed from 3 to 4, leader changed from 0761f3b2907f426189bc51a65f60bec1 (127.30.25.2) to 22d14ec1ba314b3296094fb24350578d (127.30.25.1). New cstate: current_term: 4 leader_uuid: "22d14ec1ba314b3296094fb24350578d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.915652 31101 raft_consensus.cc:1275] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:59.916767 31307 consensus_queue.cc:1048] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:59.928617 31030 raft_consensus.cc:1275] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:59.930125 31308 consensus_queue.cc:1048] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:59.938143 31030 raft_consensus.cc:1275] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:59.939714 31304 consensus_queue.cc:1048] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:59.948786 30954 raft_consensus.cc:1275] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:59.950209 31304 consensus_queue.cc:1048] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:59.992120 31101 raft_consensus.cc:1275] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:59.993348 31308 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.001677 31030 raft_consensus.cc:1275] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.002965 31307 consensus_queue.cc:1048] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.090410 31103 raft_consensus.cc:1275] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.091662 31307 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6b1f8f62214e779396d4da64de242d" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.101322 31030 raft_consensus.cc:1275] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Refusing update from remote peer 22d14ec1ba314b3296094fb24350578d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.102921 31307 consensus_queue.cc:1048] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.105916 30954 raft_consensus.cc:1275] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.107298 31304 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22d14ec1ba314b3296094fb24350578d" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.120193 31030 raft_consensus.cc:1275] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Refusing update from remote peer 1b6b1f8f62214e779396d4da64de242d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.121804 31325 consensus_queue.cc:1048] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0761f3b2907f426189bc51a65f60bec1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:00.182103 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:00.187134 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:00.187551 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:01.188541 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:01.193593 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:01.194031 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:02.195036 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:02.199888 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:02.200317 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:03.201476 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:03.206692 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:03.207120 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:04.208180 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:04.213398 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:04.213837 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:05.215071 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:05.220082 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:05.220533 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:06.221616 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:06.226435 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:06.226871 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:07.227943 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:07.232781 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:07.233256 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:08.234571 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:08.240180 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:08.240643 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:09.269021 30820 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:16:09.269730 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
I20260709 12:16:09.295898 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:09.296805 30820 tablet_replica.cc:333] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.297582 30820 raft_consensus.cc:2243] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.298739 30820 raft_consensus.cc:2272] T 578ec24f79e14c028d88586a3cf00fc7 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.300951 30820 tablet_replica.cc:333] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.301476 30820 raft_consensus.cc:2243] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.301936 30820 raft_consensus.cc:2272] T 0ff03333b641443a84a40390b03de4f2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.303601 30820 tablet_replica.cc:333] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.304090 30820 raft_consensus.cc:2243] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.304498 30820 raft_consensus.cc:2272] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.306203 30820 tablet_replica.cc:333] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.306746 30820 raft_consensus.cc:2243] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.307662 30820 raft_consensus.cc:2272] T 37f44d95ff9f4981ae7f60b2fec9e713 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.309288 30820 tablet_replica.cc:333] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.309746 30820 raft_consensus.cc:2243] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.310173 30820 raft_consensus.cc:2272] T d6eafbfc73a149949ff328170fd420d9 P 22d14ec1ba314b3296094fb24350578d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.311738 30820 tablet_replica.cc:333] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.312194 30820 raft_consensus.cc:2243] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.312919 30820 raft_consensus.cc:2272] T ea866411680f4522a261bbc1dfd7f180 P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.314628 30820 tablet_replica.cc:333] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.315090 30820 raft_consensus.cc:2243] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.315472 30820 raft_consensus.cc:2272] T 5b0e3bffcac84cb3a22d923f6bc3204f P 22d14ec1ba314b3296094fb24350578d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.317039 30820 tablet_replica.cc:333] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.317516 30820 raft_consensus.cc:2243] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.317883 30820 raft_consensus.cc:2272] T 3d4e730ee7c64029952254b76a7a7a01 P 22d14ec1ba314b3296094fb24350578d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.319429 30820 tablet_replica.cc:333] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d: stopping tablet replica
I20260709 12:16:09.319795 30820 raft_consensus.cc:2243] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.320205 30820 raft_consensus.cc:2272] T 4c6976d88606429d928286415c52bec5 P 22d14ec1ba314b3296094fb24350578d [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:09.415234 30996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36887: connect: Connection refused (error 111)
I20260709 12:16:09.417006 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
W20260709 12:16:09.419857 30996 consensus_peers.cc:597] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 -> Peer 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887): Couldn't send request to peer 22d14ec1ba314b3296094fb24350578d. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:09.429952 31070 consensus_peers.cc:597] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d -> Peer 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887): Couldn't send request to peer 22d14ec1ba314b3296094fb24350578d. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:09.439575 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
W20260709 12:16:09.452994 31070 consensus_peers.cc:597] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d -> Peer 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887): Couldn't send request to peer 22d14ec1ba314b3296094fb24350578d. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:09.475227 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:09.476051 30820 tablet_replica.cc:333] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.476996 30820 raft_consensus.cc:2243] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.477659 30820 raft_consensus.cc:2272] T 0ff03333b641443a84a40390b03de4f2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.479450 30820 tablet_replica.cc:333] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.479982 30820 raft_consensus.cc:2243] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.480702 30820 raft_consensus.cc:2272] T 5b0e3bffcac84cb3a22d923f6bc3204f P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.482488 30820 tablet_replica.cc:333] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.482957 30820 raft_consensus.cc:2243] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.483392 30820 raft_consensus.cc:2272] T 578ec24f79e14c028d88586a3cf00fc7 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.485399 30820 tablet_replica.cc:333] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.485900 30820 raft_consensus.cc:2243] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.486917 30820 raft_consensus.cc:2272] T d6eafbfc73a149949ff328170fd420d9 P 0761f3b2907f426189bc51a65f60bec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.489390 30820 tablet_replica.cc:333] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
W20260709 12:16:09.489473 31070 consensus_peers.cc:597] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d -> Peer 22d14ec1ba314b3296094fb24350578d (127.30.25.1:36887): Couldn't send request to peer 22d14ec1ba314b3296094fb24350578d. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:09.489984 30820 raft_consensus.cc:2243] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.490500 30820 raft_consensus.cc:2272] T 37f44d95ff9f4981ae7f60b2fec9e713 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.492128 30820 tablet_replica.cc:333] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.492591 30820 raft_consensus.cc:2243] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.493345 30820 raft_consensus.cc:2272] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.495033 30820 tablet_replica.cc:333] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.495514 30820 raft_consensus.cc:2243] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.495929 30820 raft_consensus.cc:2272] T ea866411680f4522a261bbc1dfd7f180 P 0761f3b2907f426189bc51a65f60bec1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.497469 30820 tablet_replica.cc:333] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.497920 30820 raft_consensus.cc:2243] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.498459 30820 raft_consensus.cc:2272] T 3d4e730ee7c64029952254b76a7a7a01 P 0761f3b2907f426189bc51a65f60bec1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.500097 30820 tablet_replica.cc:333] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1: stopping tablet replica
I20260709 12:16:09.500555 30820 raft_consensus.cc:2243] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.500954 30820 raft_consensus.cc:2272] T 4c6976d88606429d928286415c52bec5 P 0761f3b2907f426189bc51a65f60bec1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.528188 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
I20260709 12:16:09.546427 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
I20260709 12:16:09.577039 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:09.577759 30820 tablet_replica.cc:333] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.578431 30820 raft_consensus.cc:2243] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.578927 30820 raft_consensus.cc:2272] T 578ec24f79e14c028d88586a3cf00fc7 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.580842 30820 tablet_replica.cc:333] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.581331 30820 raft_consensus.cc:2243] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.581751 30820 raft_consensus.cc:2272] T ea866411680f4522a261bbc1dfd7f180 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.583514 30820 tablet_replica.cc:333] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.584069 30820 raft_consensus.cc:2243] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.584510 30820 raft_consensus.cc:2272] T 5b0e3bffcac84cb3a22d923f6bc3204f P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.586221 30820 tablet_replica.cc:333] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.586776 30820 raft_consensus.cc:2243] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.587284 30820 raft_consensus.cc:2272] T 37f44d95ff9f4981ae7f60b2fec9e713 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.588850 30820 tablet_replica.cc:333] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.589336 30820 raft_consensus.cc:2243] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.589762 30820 raft_consensus.cc:2272] T d6eafbfc73a149949ff328170fd420d9 P 1b6b1f8f62214e779396d4da64de242d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.591464 30820 tablet_replica.cc:333] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.591928 30820 raft_consensus.cc:2243] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.592650 30820 raft_consensus.cc:2272] T 3d4e730ee7c64029952254b76a7a7a01 P 1b6b1f8f62214e779396d4da64de242d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.594221 30820 tablet_replica.cc:333] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.594774 30820 raft_consensus.cc:2243] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.595510 30820 raft_consensus.cc:2272] T 4c6976d88606429d928286415c52bec5 P 1b6b1f8f62214e779396d4da64de242d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.597070 30820 tablet_replica.cc:333] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.597529 30820 raft_consensus.cc:2243] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:09.597949 30820 raft_consensus.cc:2272] T c18c4394c9804ff5a2701e1d9fbb6bc2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.599673 30820 tablet_replica.cc:333] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d: stopping tablet replica
I20260709 12:16:09.600109 30820 raft_consensus.cc:2243] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.600833 30820 raft_consensus.cc:2272] T 0ff03333b641443a84a40390b03de4f2 P 1b6b1f8f62214e779396d4da64de242d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.646602 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
I20260709 12:16:09.664743 30820 master.cc:562] Master@127.30.25.62:39889 shutting down...
I20260709 12:16:09.801074 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:09.801819 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:09.802181 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7e8eb82cbd01496eae600e58f04c31d9: stopping tablet replica
I20260709 12:16:09.838815 30820 master.cc:584] Master@127.30.25.62:39889 shutdown complete.
0761f3b2907f426189bc51a65f60bec1  4
1b6b1f8f62214e779396d4da64de242d  1
22d14ec1ba314b3296094fb24350578d  4
0761f3b2907f426189bc51a65f60bec1  3
1b6b1f8f62214e779396d4da64de242d  3
22d14ec1ba314b3296094fb24350578d  3
0761f3b2907f426189bc51a65f60bec1  8
1b6b1f8f62214e779396d4da64de242d  1
22d14ec1ba314b3296094fb24350578d  0
0761f3b2907f426189bc51a65f60bec1  3
1b6b1f8f62214e779396d4da64de242d  3
22d14ec1ba314b3296094fb24350578d  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31007 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:16:09.880584 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:43585
I20260709 12:16:09.881717 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:09.887432 31342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:09.887468 31343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:09.889014 31345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:09.889775 30820 server_base.cc:1061] running on GCE node
I20260709 12:16:09.890834 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:09.891042 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:09.891207 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599369891184 us; error 0 us; skew 500 ppm
I20260709 12:16:09.891736 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:09.894062 30820 webserver.cc:533] Webserver started at http://127.30.25.62:40259/ using document root <none> and password file <none>
I20260709 12:16:09.894865 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:09.895123 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:09.895403 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:09.896538 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "9db0ec0090ec47c39735a1f7d07a7ca5"
format_stamp: "Formatted at 2026-07-09 12:16:09 on dist-test-slave-rldg"
I20260709 12:16:09.901022 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:09.904309 31350 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:09.905035 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:09.905313 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "9db0ec0090ec47c39735a1f7d07a7ca5"
format_stamp: "Formatted at 2026-07-09 12:16:09 on dist-test-slave-rldg"
I20260709 12:16:09.905586 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:09.917786 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:09.919152 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:09.958452 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:43585
I20260709 12:16:09.958515 31401 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:43585 every 8 connection(s)
I20260709 12:16:09.962378 31402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:09.971704 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5: Bootstrap starting.
I20260709 12:16:09.976346 31402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:09.980311 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5: No bootstrap required, opened a new log
I20260709 12:16:09.982378 31402 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" member_type: VOTER }
I20260709 12:16:09.982808 31402 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:09.982985 31402 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db0ec0090ec47c39735a1f7d07a7ca5, State: Initialized, Role: FOLLOWER
I20260709 12:16:09.983460 31402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" member_type: VOTER }
I20260709 12:16:09.983881 31402 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:09.984071 31402 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:09.984282 31402 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:09.988353 31402 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" member_type: VOTER }
I20260709 12:16:09.988857 31402 leader_election.cc:304] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9db0ec0090ec47c39735a1f7d07a7ca5; no voters: 
I20260709 12:16:09.989943 31402 leader_election.cc:290] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:09.990480 31405 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:09.991788 31405 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 1 LEADER]: Becoming Leader. State: Replica: 9db0ec0090ec47c39735a1f7d07a7ca5, State: Running, Role: LEADER
I20260709 12:16:09.992561 31405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" member_type: VOTER }
I20260709 12:16:09.993104 31402 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:09.997960 31407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9db0ec0090ec47c39735a1f7d07a7ca5. Latest consensus state: current_term: 1 leader_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" member_type: VOTER } }
I20260709 12:16:09.998543 31407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:09.998148 31406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db0ec0090ec47c39735a1f7d07a7ca5" member_type: VOTER } }
I20260709 12:16:09.998937 31406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:10.001847 31412 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:10.006958 31412 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:10.010842 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:10.015980 31412 catalog_manager.cc:1383] Generated new cluster ID: 55e7a524343640e7b8acb1041e0db7a0
I20260709 12:16:10.016284 31412 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:10.035195 31412 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:10.036394 31412 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:10.058439 31412 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5: Generated new TSK 0
I20260709 12:16:10.059047 31412 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:10.077726 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:10.083901 31423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.085101 31424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.086716 31426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:10.086896 30820 server_base.cc:1061] running on GCE node
I20260709 12:16:10.087930 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:10.088160 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:10.088335 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599370088318 us; error 0 us; skew 500 ppm
I20260709 12:16:10.088889 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:10.091502 30820 webserver.cc:533] Webserver started at http://127.30.25.1:35805/ using document root <none> and password file <none>
I20260709 12:16:10.092008 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:10.092222 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:10.092495 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:10.093531 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "f6b44428e7184197b458180e0cbb111e"
format_stamp: "Formatted at 2026-07-09 12:16:10 on dist-test-slave-rldg"
I20260709 12:16:10.098057 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:16:10.101414 31431 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.102185 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:16:10.102502 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "f6b44428e7184197b458180e0cbb111e"
format_stamp: "Formatted at 2026-07-09 12:16:10 on dist-test-slave-rldg"
I20260709 12:16:10.102784 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:10.123353 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:10.124549 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:10.125991 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:10.128315 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:10.128532 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.128780 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:10.128969 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.169771 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:40751
I20260709 12:16:10.169865 31493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:40751 every 8 connection(s)
I20260709 12:16:10.174779 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:10.181990 31498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.182109 31499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.185961 31501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:10.186995 30820 server_base.cc:1061] running on GCE node
I20260709 12:16:10.188412 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:10.188712 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:10.188957 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599370188919 us; error 0 us; skew 500 ppm
I20260709 12:16:10.189726 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:10.191908 31494 heartbeater.cc:344] Connected to a master server at 127.30.25.62:43585
I20260709 12:16:10.192301 31494 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:10.192867 30820 webserver.cc:533] Webserver started at http://127.30.25.2:43205/ using document root <none> and password file <none>
I20260709 12:16:10.193017 31494 heartbeater.cc:507] Master 127.30.25.62:43585 requested a full tablet report, sending...
I20260709 12:16:10.193511 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:10.193753 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:10.194077 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:10.195312 31367 ts_manager.cc:194] Registered new tserver with Master: f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751)
I20260709 12:16:10.195405 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
format_stamp: "Formatted at 2026-07-09 12:16:10 on dist-test-slave-rldg"
I20260709 12:16:10.197271 31367 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53650
I20260709 12:16:10.200723 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:16:10.204293 31506 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.205188 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:10.205423 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
format_stamp: "Formatted at 2026-07-09 12:16:10 on dist-test-slave-rldg"
I20260709 12:16:10.205650 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:10.245393 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:10.246615 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:10.247985 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:10.250093 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:10.250263 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.250589 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:10.250728 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.289556 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:45883
I20260709 12:16:10.289659 31568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:45883 every 8 connection(s)
I20260709 12:16:10.294252 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:10.302892 31573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.303659 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:10.304885 30820 server_base.cc:1061] running on GCE node
W20260709 12:16:10.305176 31575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:10.306358 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:10.306635 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:10.306824 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599370306793 us; error 0 us; skew 500 ppm
I20260709 12:16:10.307487 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:10.309366 31569 heartbeater.cc:344] Connected to a master server at 127.30.25.62:43585
I20260709 12:16:10.309724 31569 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:10.310554 30820 webserver.cc:533] Webserver started at http://127.30.25.3:38959/ using document root <none> and password file <none>
I20260709 12:16:10.310560 31569 heartbeater.cc:507] Master 127.30.25.62:43585 requested a full tablet report, sending...
I20260709 12:16:10.311147 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:10.311364 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:10.311633 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:10.312697 31367 ts_manager.cc:194] Registered new tserver with Master: 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:10.312819 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "c471ee20d86e43cb99730b9886096a0e"
format_stamp: "Formatted at 2026-07-09 12:16:10 on dist-test-slave-rldg"
I20260709 12:16:10.313993 31367 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53662
I20260709 12:16:10.317305 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:16:10.320451 31580 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.321197 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:10.321497 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "c471ee20d86e43cb99730b9886096a0e"
format_stamp: "Formatted at 2026-07-09 12:16:10 on dist-test-slave-rldg"
I20260709 12:16:10.321764 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:10.332674 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:10.333930 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:10.335435 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:10.337539 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:10.337764 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.337998 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:10.338169 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.377481 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:42637
I20260709 12:16:10.377602 31642 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:42637 every 8 connection(s)
I20260709 12:16:10.391215 31643 heartbeater.cc:344] Connected to a master server at 127.30.25.62:43585
I20260709 12:16:10.391604 31643 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:10.392370 31643 heartbeater.cc:507] Master 127.30.25.62:43585 requested a full tablet report, sending...
I20260709 12:16:10.394445 31367 ts_manager.cc:194] Registered new tserver with Master: c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:10.395298 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014784357s
I20260709 12:16:10.396200 31367 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53664
I20260709 12:16:10.420974 31367 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53666:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:10.424036 31367 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:10.499302 31608 tablet_service.cc:1511] Processing CreateTablet for tablet ff39cf78a2904032b2294817a342e358 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:10.499847 31607 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa3aab5bbf44189918b0cc20a13b6a8 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:10.501873 31608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff39cf78a2904032b2294817a342e358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.502213 31459 tablet_service.cc:1511] Processing CreateTablet for tablet ff39cf78a2904032b2294817a342e358 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:10.503082 31458 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa3aab5bbf44189918b0cc20a13b6a8 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:10.503527 31459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff39cf78a2904032b2294817a342e358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.503789 31607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa3aab5bbf44189918b0cc20a13b6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.504351 31458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa3aab5bbf44189918b0cc20a13b6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.521261 31457 tablet_service.cc:1511] Processing CreateTablet for tablet bee4f9e6388544688e015d1c780bbe6b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:10.522666 31457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee4f9e6388544688e015d1c780bbe6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.525962 31606 tablet_service.cc:1511] Processing CreateTablet for tablet bee4f9e6388544688e015d1c780bbe6b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:10.527859 31606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee4f9e6388544688e015d1c780bbe6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.527788 31456 tablet_service.cc:1511] Processing CreateTablet for tablet de9867050e944946bf85cdb1d973350b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:10.529150 31456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de9867050e944946bf85cdb1d973350b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.532012 31605 tablet_service.cc:1511] Processing CreateTablet for tablet de9867050e944946bf85cdb1d973350b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:10.533334 31605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de9867050e944946bf85cdb1d973350b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.547982 31455 tablet_service.cc:1511] Processing CreateTablet for tablet 419d1781a76e4c3f9d3c6e0e960f2b32 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:10.549424 31455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 419d1781a76e4c3f9d3c6e0e960f2b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.550365 31534 tablet_service.cc:1511] Processing CreateTablet for tablet ff39cf78a2904032b2294817a342e358 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:10.552078 31534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff39cf78a2904032b2294817a342e358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.555931 31533 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa3aab5bbf44189918b0cc20a13b6a8 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:10.557722 31533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa3aab5bbf44189918b0cc20a13b6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.557945 31658 tablet_bootstrap.cc:492] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.564428 31458 tablet_service.cc:1511] Processing CreateTablet for tablet 2336e77544314a939418eb7325c0c23b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:10.565750 31458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2336e77544314a939418eb7325c0c23b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.572305 31658 tablet_bootstrap.cc:654] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.559249 31659 tablet_bootstrap.cc:492] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:10.559346 31532 tablet_service.cc:1511] Processing CreateTablet for tablet bee4f9e6388544688e015d1c780bbe6b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:10.575466 31532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee4f9e6388544688e015d1c780bbe6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.560775 31531 tablet_service.cc:1511] Processing CreateTablet for tablet de9867050e944946bf85cdb1d973350b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:10.579447 31531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de9867050e944946bf85cdb1d973350b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.562618 31530 tablet_service.cc:1511] Processing CreateTablet for tablet 419d1781a76e4c3f9d3c6e0e960f2b32 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:10.562700 31607 tablet_service.cc:1511] Processing CreateTablet for tablet 419d1781a76e4c3f9d3c6e0e960f2b32 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:10.583173 31604 tablet_service.cc:1511] Processing CreateTablet for tablet 2336e77544314a939418eb7325c0c23b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:10.583189 31530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 419d1781a76e4c3f9d3c6e0e960f2b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.583884 31607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 419d1781a76e4c3f9d3c6e0e960f2b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.596095 31528 tablet_service.cc:1511] Processing CreateTablet for tablet dbae48e291ac4ed5a3f172eaae5a93fe (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:10.597165 31604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2336e77544314a939418eb7325c0c23b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.597461 31528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbae48e291ac4ed5a3f172eaae5a93fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.598773 31455 tablet_service.cc:1511] Processing CreateTablet for tablet f80bf20b2b2942a3b7d0c76fdd769900 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:10.600072 31455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80bf20b2b2942a3b7d0c76fdd769900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.605345 31659 tablet_bootstrap.cc:654] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.605984 31605 tablet_service.cc:1511] Processing CreateTablet for tablet dbae48e291ac4ed5a3f172eaae5a93fe (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:10.607836 31605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbae48e291ac4ed5a3f172eaae5a93fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.609002 31527 tablet_service.cc:1511] Processing CreateTablet for tablet f80bf20b2b2942a3b7d0c76fdd769900 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:10.610272 31527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80bf20b2b2942a3b7d0c76fdd769900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.581918 31529 tablet_service.cc:1511] Processing CreateTablet for tablet 2336e77544314a939418eb7325c0c23b (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:10.639595 31529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2336e77544314a939418eb7325c0c23b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.585131 31457 tablet_service.cc:1511] Processing CreateTablet for tablet dbae48e291ac4ed5a3f172eaae5a93fe (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:10.644461 31661 tablet_bootstrap.cc:492] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:10.644775 31457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbae48e291ac4ed5a3f172eaae5a93fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.662873 31658 tablet_bootstrap.cc:492] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:10.663395 31658 ts_tablet_manager.cc:1403] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.106s	user 0.024s	sys 0.032s
I20260709 12:16:10.663616 31661 tablet_bootstrap.cc:654] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.672658 31604 tablet_service.cc:1511] Processing CreateTablet for tablet f80bf20b2b2942a3b7d0c76fdd769900 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:10.675182 31604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80bf20b2b2942a3b7d0c76fdd769900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.666298 31658 raft_consensus.cc:359] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.690450 31658 raft_consensus.cc:385] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.690825 31658 raft_consensus.cc:740] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.693058 31658 consensus_queue.cc:260] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.697949 31494 heartbeater.cc:499] Master 127.30.25.62:43585 was elected leader, sending a full tablet report...
I20260709 12:16:10.699196 31659 tablet_bootstrap.cc:492] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:10.699734 31659 ts_tablet_manager.cc:1403] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.141s	user 0.006s	sys 0.011s
I20260709 12:16:10.700340 31658 ts_tablet_manager.cc:1434] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.003s
I20260709 12:16:10.702597 31658 tablet_bootstrap.cc:492] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.704823 31659 raft_consensus.cc:359] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.705730 31659 raft_consensus.cc:385] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.706139 31659 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.707223 31659 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.709529 31658 tablet_bootstrap.cc:654] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.709270 31457 tablet_service.cc:1511] Processing CreateTablet for tablet 29e14bbf2a0e4ffda1b17577c770cbf3 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:10.712162 31661 tablet_bootstrap.cc:492] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:10.712734 31661 ts_tablet_manager.cc:1403] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.021s
I20260709 12:16:10.713491 31529 tablet_service.cc:1511] Processing CreateTablet for tablet 29e14bbf2a0e4ffda1b17577c770cbf3 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:10.714943 31529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29e14bbf2a0e4ffda1b17577c770cbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.722162 31661 raft_consensus.cc:359] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.723110 31661 raft_consensus.cc:385] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.723506 31661 raft_consensus.cc:740] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.724601 31661 consensus_queue.cc:260] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.730422 31457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29e14bbf2a0e4ffda1b17577c770cbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.732697 31455 tablet_service.cc:1511] Processing CreateTablet for tablet b6b1a66a8e974614b59a81ab2b446c0e (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:10.734037 31455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b1a66a8e974614b59a81ab2b446c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.735860 31643 heartbeater.cc:499] Master 127.30.25.62:43585 was elected leader, sending a full tablet report...
I20260709 12:16:10.735769 31528 tablet_service.cc:1511] Processing CreateTablet for tablet b6b1a66a8e974614b59a81ab2b446c0e (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:10.737018 31528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b1a66a8e974614b59a81ab2b446c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.739003 31607 tablet_service.cc:1511] Processing CreateTablet for tablet b6b1a66a8e974614b59a81ab2b446c0e (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:10.740315 31607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b1a66a8e974614b59a81ab2b446c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.743705 31659 ts_tablet_manager.cc:1434] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 12:16:10.745150 31659 tablet_bootstrap.cc:492] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:10.752421 31658 tablet_bootstrap.cc:492] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:10.752902 31658 ts_tablet_manager.cc:1403] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.017s
I20260709 12:16:10.729733 31604 tablet_service.cc:1511] Processing CreateTablet for tablet 29e14bbf2a0e4ffda1b17577c770cbf3 (DEFAULT_TABLE table=test-workload [id=9c28c0a64ff848979cd370f5d785c3ad]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:10.756331 31658 raft_consensus.cc:359] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.757050 31658 raft_consensus.cc:385] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.757371 31658 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.758539 31604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29e14bbf2a0e4ffda1b17577c770cbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:10.758159 31658 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.761648 31658 ts_tablet_manager.cc:1434] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:10.762614 31658 tablet_bootstrap.cc:492] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.766836 31659 tablet_bootstrap.cc:654] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.768800 31658 tablet_bootstrap.cc:654] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.771744 31569 heartbeater.cc:499] Master 127.30.25.62:43585 was elected leader, sending a full tablet report...
I20260709 12:16:10.780151 31658 tablet_bootstrap.cc:492] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:10.780705 31658 ts_tablet_manager.cc:1403] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:16:10.785869 31668 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.787340 31658 raft_consensus.cc:359] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.786748 31668 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.788059 31658 raft_consensus.cc:385] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.788486 31658 raft_consensus.cc:740] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.789542 31665 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.789367 31658 consensus_queue.cc:260] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.790002 31665 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.792285 31661 ts_tablet_manager.cc:1434] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.004s
I20260709 12:16:10.792354 31664 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.792917 31664 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.793499 31661 tablet_bootstrap.cc:492] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:10.796940 31668 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:10.803267 31665 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:10.811919 31664 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:10.848785 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
W20260709 12:16:10.850459 31509 leader_election.cc:343] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:10.846648 31661 tablet_bootstrap.cc:654] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.853333 31659 tablet_bootstrap.cc:492] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:10.862869 31659 ts_tablet_manager.cc:1403] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.005s
I20260709 12:16:10.866006 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:10.866892 31469 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:10.867102 31659 raft_consensus.cc:359] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.867863 31659 raft_consensus.cc:385] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.868187 31659 raft_consensus.cc:740] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.868721 31583 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:10.871433 31665 raft_consensus.cc:2804] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:10.871838 31665 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.872207 31665 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:10.876873 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
W20260709 12:16:10.880746 31582 leader_election.cc:343] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:10.880841 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:10.883605 31658 ts_tablet_manager.cc:1434] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.103s	user 0.011s	sys 0.023s
I20260709 12:16:10.886798 31677 raft_consensus.cc:493] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.870000 31659 consensus_queue.cc:260] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.890223 31665 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.891300 31658 tablet_bootstrap.cc:492] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.891784 31618 raft_consensus.cc:2393] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f6b44428e7184197b458180e0cbb111e in current term 1: Already voted for candidate c471ee20d86e43cb99730b9886096a0e in this term.
I20260709 12:16:10.893781 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:10.894137 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:10.894392 31469 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:10.895543 31582 leader_election.cc:343] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:10.896097 31665 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:10.897110 31677 raft_consensus.cc:515] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.899148 31658 tablet_bootstrap.cc:654] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.900143 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:10.900801 31618 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:10.902105 31435 leader_election.cc:304] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:10.903134 31664 raft_consensus.cc:2804] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:10.903520 31664 raft_consensus.cc:493] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.903877 31664 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:10.903765 31469 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
I20260709 12:16:10.905378 31583 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:10.905473 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:10.906697 31665 raft_consensus.cc:2804] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:10.907320 31665 raft_consensus.cc:697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:10.908223 31665 consensus_queue.cc:237] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
W20260709 12:16:10.909384 31510 leader_election.cc:343] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:10.909869 31510 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863; no voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e
I20260709 12:16:10.910703 31668 raft_consensus.cc:2749] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:10.911242 31664 raft_consensus.cc:515] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.911808 31658 tablet_bootstrap.cc:492] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:10.912312 31658 ts_tablet_manager.cc:1403] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
I20260709 12:16:10.913592 31661 tablet_bootstrap.cc:492] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:10.914058 31661 ts_tablet_manager.cc:1403] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.121s	user 0.015s	sys 0.015s
I20260709 12:16:10.913100 31677 leader_election.cc:290] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:10.916303 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:10.916541 31658 raft_consensus.cc:359] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.916973 31661 raft_consensus.cc:359] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.917527 31658 raft_consensus.cc:385] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.917698 31661 raft_consensus.cc:385] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:10.917766 31433 leader_election.cc:343] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:10.918113 31661 raft_consensus.cc:740] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.918254 31433 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6b44428e7184197b458180e0cbb111e; no voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:10.918875 31661 consensus_queue.cc:260] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.919585 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:10.919435 31677 raft_consensus.cc:2749] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:10.920084 31658 raft_consensus.cc:740] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.920192 31618 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:10.920966 31658 consensus_queue.cc:260] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.922891 31661 ts_tablet_manager.cc:1434] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:16:10.923153 31658 ts_tablet_manager.cc:1434] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:16:10.923913 31661 tablet_bootstrap.cc:492] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:10.924160 31658 tablet_bootstrap.cc:492] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.919775 31664 leader_election.cc:290] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Requested vote from peers c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:10.927542 31618 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:10.928680 31435 leader_election.cc:304] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:10.929610 31664 raft_consensus.cc:2804] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:10.930084 31664 raft_consensus.cc:697] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:10.930954 31664 consensus_queue.cc:237] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:10.931803 31658 tablet_bootstrap.cc:654] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.932976 31367 catalog_manager.cc:5697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 0 to 1, leader changed from <none> to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 1 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:10.935449 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:10.937004 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:10.937999 31659 ts_tablet_manager.cc:1434] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.075s	user 0.013s	sys 0.005s
I20260709 12:16:10.939354 31659 tablet_bootstrap.cc:492] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
W20260709 12:16:10.941051 31433 leader_election.cc:343] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:10.942265 31433 leader_election.cc:343] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:10.943552 31661 tablet_bootstrap.cc:654] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.950414 31658 tablet_bootstrap.cc:492] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:10.950944 31658 ts_tablet_manager.cc:1403] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.004s
I20260709 12:16:10.953862 31658 raft_consensus.cc:359] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.954679 31658 raft_consensus.cc:385] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.954990 31658 raft_consensus.cc:740] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.955863 31658 consensus_queue.cc:260] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.958160 31659 tablet_bootstrap.cc:654] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.958384 31658 ts_tablet_manager.cc:1434] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:16:10.965691 31658 tablet_bootstrap.cc:492] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.968838 31367 catalog_manager.cc:5697] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:10.971868 31658 tablet_bootstrap.cc:654] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.977767 31661 tablet_bootstrap.cc:492] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:10.978372 31661 ts_tablet_manager.cc:1403] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.055s	user 0.026s	sys 0.014s
I20260709 12:16:10.980713 31658 tablet_bootstrap.cc:492] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:10.980742 31659 tablet_bootstrap.cc:492] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:10.981438 31658 ts_tablet_manager.cc:1403] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.003s
I20260709 12:16:10.981189 31661 raft_consensus.cc:359] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.981956 31661 raft_consensus.cc:385] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.981478 31659 ts_tablet_manager.cc:1403] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.011s
I20260709 12:16:10.982281 31661 raft_consensus.cc:740] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.983299 31661 consensus_queue.cc:260] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.984568 31658 raft_consensus.cc:359] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.985299 31658 raft_consensus.cc:385] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.985658 31658 raft_consensus.cc:740] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.985370 31659 raft_consensus.cc:359] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.986179 31659 raft_consensus.cc:385] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:10.986506 31659 raft_consensus.cc:740] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:10.986476 31658 consensus_queue.cc:260] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.987497 31659 consensus_queue.cc:260] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:10.988209 31661 ts_tablet_manager.cc:1434] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.004s
I20260709 12:16:10.988719 31658 ts_tablet_manager.cc:1434] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:10.989135 31661 tablet_bootstrap.cc:492] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:10.989650 31658 tablet_bootstrap.cc:492] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:10.991480 31659 ts_tablet_manager.cc:1434] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:16:10.992408 31659 tablet_bootstrap.cc:492] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:10.996083 31658 tablet_bootstrap.cc:654] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.996084 31661 tablet_bootstrap.cc:654] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:10.998764 31659 tablet_bootstrap.cc:654] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:11.007922 31415 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:11.009610 31661 tablet_bootstrap.cc:492] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.010299 31661 ts_tablet_manager.cc:1403] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.001s
I20260709 12:16:11.012427 31658 tablet_bootstrap.cc:492] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:11.012894 31658 ts_tablet_manager.cc:1403] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.008s
I20260709 12:16:11.013826 31659 tablet_bootstrap.cc:492] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.014084 31664 raft_consensus.cc:493] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.014273 31659 ts_tablet_manager.cc:1403] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.003s
I20260709 12:16:11.014595 31664 raft_consensus.cc:515] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.016335 31664 leader_election.cc:290] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.017045 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9867050e944946bf85cdb1d973350b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.017262 31661 raft_consensus.cc:359] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.017675 31544 raft_consensus.cc:2468] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.017863 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9867050e944946bf85cdb1d973350b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:11.017992 31661 raft_consensus.cc:385] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.018487 31661 raft_consensus.cc:740] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.018544 31618 raft_consensus.cc:2468] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.019002 31433 leader_election.cc:304] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.019340 31661 consensus_queue.cc:260] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.020289 31664 raft_consensus.cc:2804] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.020651 31664 raft_consensus.cc:493] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.020977 31664 raft_consensus.cc:3060] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.021119 31659 raft_consensus.cc:359] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.021585 31661 ts_tablet_manager.cc:1434] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:16:11.021890 31659 raft_consensus.cc:385] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.022215 31659 raft_consensus.cc:740] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.023015 31661 tablet_bootstrap.cc:492] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:11.022981 31659 consensus_queue.cc:260] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.024825 31658 raft_consensus.cc:359] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.025513 31658 raft_consensus.cc:385] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.025818 31658 raft_consensus.cc:740] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.026549 31658 consensus_queue.cc:260] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.029003 31659 ts_tablet_manager.cc:1434] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:11.029986 31659 tablet_bootstrap.cc:492] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:11.030454 31661 tablet_bootstrap.cc:654] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.036671 31659 tablet_bootstrap.cc:654] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.037703 31658 ts_tablet_manager.cc:1434] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.004s
I20260709 12:16:11.038203 31661 tablet_bootstrap.cc:492] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.038676 31661 ts_tablet_manager.cc:1403] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.010s
I20260709 12:16:11.038795 31658 tablet_bootstrap.cc:492] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:11.041487 31661 raft_consensus.cc:359] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:11.042236 31661 raft_consensus.cc:385] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.042572 31661 raft_consensus.cc:740] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.043498 31661 consensus_queue.cc:260] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:11.045821 31658 tablet_bootstrap.cc:654] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.048585 31661 ts_tablet_manager.cc:1434] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:16:11.049827 31661 tablet_bootstrap.cc:492] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:11.053314 31677 raft_consensus.cc:493] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.053960 31677 raft_consensus.cc:515] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.056730 31677 leader_election.cc:290] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.057497 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee4f9e6388544688e015d1c780bbe6b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.058117 31661 tablet_bootstrap.cc:654] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.058143 31544 raft_consensus.cc:2468] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.058282 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee4f9e6388544688e015d1c780bbe6b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:11.059679 31433 leader_election.cc:304] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.060499 31677 raft_consensus.cc:2804] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:11.066411 31435 leader_election.cc:343] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:11.060878 31677 raft_consensus.cc:493] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.069204 31677 raft_consensus.cc:3060] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.069123 31664 raft_consensus.cc:515] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.076246 31677 raft_consensus.cc:515] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.079228 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9867050e944946bf85cdb1d973350b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.079665 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee4f9e6388544688e015d1c780bbe6b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.079185 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee4f9e6388544688e015d1c780bbe6b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.080237 31543 raft_consensus.cc:3060] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.081751 31664 leader_election.cc:290] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.079828 31544 raft_consensus.cc:3060] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.084792 31658 tablet_bootstrap.cc:492] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:11.085289 31658 ts_tablet_manager.cc:1403] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.035s
I20260709 12:16:11.085402 31661 tablet_bootstrap.cc:492] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.085902 31661 ts_tablet_manager.cc:1403] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.015s
I20260709 12:16:11.087455 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9867050e944946bf85cdb1d973350b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.087986 31618 raft_consensus.cc:3060] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.087972 31658 raft_consensus.cc:359] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.088701 31658 raft_consensus.cc:385] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.088996 31658 raft_consensus.cc:740] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.089216 31544 raft_consensus.cc:2468] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.088888 31661 raft_consensus.cc:359] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.089751 31661 raft_consensus.cc:385] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.090094 31661 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.089845 31658 consensus_queue.cc:260] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.090485 31433 leader_election.cc:304] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.090843 31661 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.091878 31664 raft_consensus.cc:2804] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.092362 31664 raft_consensus.cc:697] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.093254 31661 ts_tablet_manager.cc:1434] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:16:11.093192 31664 consensus_queue.cc:237] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.094355 31661 tablet_bootstrap.cc:492] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:11.096434 31658 ts_tablet_manager.cc:1434] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
W20260709 12:16:11.097043 31435 leader_election.cc:343] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:11.097299 31658 tablet_bootstrap.cc:492] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:11.100843 31661 tablet_bootstrap.cc:654] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.104410 31543 raft_consensus.cc:2468] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.105931 31658 tablet_bootstrap.cc:654] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.107172 31618 raft_consensus.cc:2468] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.111872 31659 tablet_bootstrap.cc:492] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.112416 31659 ts_tablet_manager.cc:1403] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.083s	user 0.028s	sys 0.000s
I20260709 12:16:11.115221 31659 raft_consensus.cc:359] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.115772 31659 raft_consensus.cc:385] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.116029 31659 raft_consensus.cc:740] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.116645 31659 consensus_queue.cc:260] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.118300 31659 ts_tablet_manager.cc:1434] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 12:16:11.118713 31677 leader_election.cc:290] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.119509 31433 leader_election.cc:304] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:11.119562 31659 tablet_bootstrap.cc:492] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:11.120639 31695 raft_consensus.cc:2804] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.121116 31695 raft_consensus.cc:697] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.121923 31695 consensus_queue.cc:237] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.129374 31659 tablet_bootstrap.cc:654] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.135417 31677 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.137457 31677 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.141390 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.142264 31544 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.143968 31433 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.145190 31664 raft_consensus.cc:2804] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.144485 31364 catalog_manager.cc:5697] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.145681 31664 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.146288 31664 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.147204 31677 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.152093 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:11.152727 31661 tablet_bootstrap.cc:492] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.153209 31661 ts_tablet_manager.cc:1403] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.059s	user 0.026s	sys 0.025s
W20260709 12:16:11.153847 31435 leader_election.cc:343] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:11.155541 31658 tablet_bootstrap.cc:492] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:11.156039 31658 ts_tablet_manager.cc:1403] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.018s
I20260709 12:16:11.156299 31661 raft_consensus.cc:359] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.157011 31661 raft_consensus.cc:385] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.157305 31661 raft_consensus.cc:740] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.158106 31661 consensus_queue.cc:260] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.159237 31664 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.160459 31661 ts_tablet_manager.cc:1434] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:11.160296 31658 raft_consensus.cc:359] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.161590 31661 tablet_bootstrap.cc:492] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:11.161943 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.162838 31544 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.166898 31664 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.162112 31658 raft_consensus.cc:385] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.167647 31658 raft_consensus.cc:740] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.168407 31658 consensus_queue.cc:260] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.169001 31661 tablet_bootstrap.cc:654] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.170045 31544 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.173779 31668 raft_consensus.cc:493] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.174060 31659 tablet_bootstrap.cc:492] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.174264 31668 raft_consensus.cc:515] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:11.174821 31658 ts_tablet_manager.cc:1434] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:16:11.174898 31659 ts_tablet_manager.cc:1403] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.007s
I20260709 12:16:11.175709 31658 tablet_bootstrap.cc:492] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e: Bootstrap starting.
I20260709 12:16:11.176342 31668 leader_election.cc:290] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637), f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751)
I20260709 12:16:11.177933 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:11.179936 31661 tablet_bootstrap.cc:492] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.179574 31659 raft_consensus.cc:359] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.180343 31661 ts_tablet_manager.cc:1403] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:16:11.180383 31659 raft_consensus.cc:385] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.180734 31433 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.180869 31659 raft_consensus.cc:740] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.181879 31659 consensus_queue.cc:260] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.182833 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:11.183023 31661 raft_consensus.cc:359] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.183568 31618 raft_consensus.cc:2393] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in current term 1: Already voted for candidate f6b44428e7184197b458180e0cbb111e in this term.
I20260709 12:16:11.183668 31661 raft_consensus.cc:385] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.184018 31661 raft_consensus.cc:740] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.184579 31658 tablet_bootstrap.cc:654] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.184890 31510 leader_election.cc:304] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863; no voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e
I20260709 12:16:11.184623 31661 consensus_queue.cc:260] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.185909 31667 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.187342 31659 ts_tablet_manager.cc:1434] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:16:11.188403 31659 tablet_bootstrap.cc:492] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:11.193611 31667 raft_consensus.cc:2749] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:11.194866 31659 tablet_bootstrap.cc:654] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.195631 31661 ts_tablet_manager.cc:1434] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:16:11.196247 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.196681 31661 tablet_bootstrap.cc:492] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
W20260709 12:16:11.198159 31435 leader_election.cc:343] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:11.199769 31695 raft_consensus.cc:2804] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.202206 31695 raft_consensus.cc:697] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.203415 31695 consensus_queue.cc:237] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.209210 31661 tablet_bootstrap.cc:654] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.218142 31661 tablet_bootstrap.cc:492] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.218652 31661 ts_tablet_manager.cc:1403] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.007s
I20260709 12:16:11.220923 31659 tablet_bootstrap.cc:492] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.221486 31659 ts_tablet_manager.cc:1403] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.007s
I20260709 12:16:11.221169 31661 raft_consensus.cc:359] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.221954 31661 raft_consensus.cc:385] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.222267 31661 raft_consensus.cc:740] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.223075 31661 consensus_queue.cc:260] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.225332 31661 ts_tablet_manager.cc:1434] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:11.224942 31659 raft_consensus.cc:359] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.225759 31659 raft_consensus.cc:385] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.226114 31659 raft_consensus.cc:740] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.226262 31661 tablet_bootstrap.cc:492] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863: Bootstrap starting.
I20260709 12:16:11.227159 31659 consensus_queue.cc:260] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.228399 31664 raft_consensus.cc:493] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.228852 31664 raft_consensus.cc:515] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.230943 31664 leader_election.cc:290] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.232478 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.232534 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:11.233122 31544 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.233394 31618 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.234613 31433 leader_election.cc:304] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.232046 31659 ts_tablet_manager.cc:1434] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:11.235473 31664 raft_consensus.cc:2804] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.235879 31664 raft_consensus.cc:493] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.236220 31664 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.236692 31659 tablet_bootstrap.cc:492] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:11.243850 31659 tablet_bootstrap.cc:654] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.245450 31661 tablet_bootstrap.cc:654] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.243440 31664 raft_consensus.cc:515] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.251282 31664 leader_election.cc:290] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.251221 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.251860 31544 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.254815 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.255584 31618 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.258620 31544 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.260403 31433 leader_election.cc:304] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.261489 31664 raft_consensus.cc:2804] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.262348 31664 raft_consensus.cc:697] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.263422 31664 consensus_queue.cc:237] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.270805 31661 tablet_bootstrap.cc:492] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863: No bootstrap required, opened a new log
I20260709 12:16:11.271447 31661 ts_tablet_manager.cc:1403] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.000s
I20260709 12:16:11.272359 31659 tablet_bootstrap.cc:492] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.273662 31680 raft_consensus.cc:493] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.274538 31661 raft_consensus.cc:359] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.274437 31680 raft_consensus.cc:515] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.275331 31661 raft_consensus.cc:385] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.275619 31661 raft_consensus.cc:740] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.276270 31661 consensus_queue.cc:260] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.276927 31680 leader_election.cc:290] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.277918 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:11.278014 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.278726 31544 raft_consensus.cc:2468] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:11.279507 31659 ts_tablet_manager.cc:1403] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.002s
I20260709 12:16:11.280265 31582 leader_election.cc:304] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:11.278224 31661 ts_tablet_manager.cc:1434] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:11.281756 31680 raft_consensus.cc:2804] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.282078 31680 raft_consensus.cc:493] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.278702 31469 raft_consensus.cc:2468] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:11.282434 31680 raft_consensus.cc:3060] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.287676 31680 raft_consensus.cc:515] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.291116 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.291205 31659 raft_consensus.cc:359] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.291775 31469 raft_consensus.cc:3060] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.291844 31659 raft_consensus.cc:385] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.292173 31659 raft_consensus.cc:740] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.294824 31618 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.294021 31364 catalog_manager.cc:5697] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.295800 31659 consensus_queue.cc:260] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.297848 31659 ts_tablet_manager.cc:1434] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:16:11.298811 31659 tablet_bootstrap.cc:492] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:11.300690 31469 raft_consensus.cc:2468] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
I20260709 12:16:11.300745 31544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.302675 31544 raft_consensus.cc:3060] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.303051 31364 catalog_manager.cc:5697] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.305667 31659 tablet_bootstrap.cc:654] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.302057 31665 raft_consensus.cc:493] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.306926 31665 raft_consensus.cc:515] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.309478 31665 leader_election.cc:290] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.311643 31658 tablet_bootstrap.cc:492] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e: No bootstrap required, opened a new log
I20260709 12:16:11.312093 31658 ts_tablet_manager.cc:1403] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e: Time spent bootstrapping tablet: real 0.137s	user 0.051s	sys 0.060s
I20260709 12:16:11.312438 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:11.312819 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.313462 31543 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:11.314721 31658 raft_consensus.cc:359] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.315361 31658 raft_consensus.cc:385] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.315623 31658 raft_consensus.cc:740] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.316256 31658 consensus_queue.cc:260] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.313357 31468 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:11.318250 31583 leader_election.cc:304] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.310067 31665 raft_consensus.cc:493] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.318188 31658 ts_tablet_manager.cc:1434] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:11.319396 31665 raft_consensus.cc:515] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.330328 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee4f9e6388544688e015d1c780bbe6b" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:11.333060 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee4f9e6388544688e015d1c780bbe6b" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.334151 31543 raft_consensus.cc:2393] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c471ee20d86e43cb99730b9886096a0e in current term 1: Already voted for candidate f6b44428e7184197b458180e0cbb111e in this term.
I20260709 12:16:11.335750 31582 leader_election.cc:304] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e; no voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e
I20260709 12:16:11.339207 31367 catalog_manager.cc:5697] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.347505 31665 leader_election.cc:290] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.348371 31665 raft_consensus.cc:2804] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.352371 31665 raft_consensus.cc:493] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.353977 31665 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.301939 31680 leader_election.cc:290] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.360632 31680 raft_consensus.cc:3060] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.362142 31583 leader_election.cc:304] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.351373 31544 raft_consensus.cc:2468] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
I20260709 12:16:11.378005 31695 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:11.387560 31709 raft_consensus.cc:2804] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.388099 31709 raft_consensus.cc:697] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:11.388748 31709 consensus_queue.cc:237] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.393146 31665 raft_consensus.cc:515] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.401427 31680 raft_consensus.cc:2749] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:11.403132 31582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:45883, user_credentials={real_user=slave}} blocked reactor thread for 67501.3us
I20260709 12:16:11.407395 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.407737 31543 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.408138 31469 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.412959 31665 leader_election.cc:290] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.421983 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.422639 31659 tablet_bootstrap.cc:492] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.423003 31542 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.423255 31659 ts_tablet_manager.cc:1403] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.125s	user 0.047s	sys 0.049s
I20260709 12:16:11.425436 31469 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
I20260709 12:16:11.426677 31583 leader_election.cc:304] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.427840 31665 raft_consensus.cc:2804] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.428587 31665 raft_consensus.cc:697] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:11.429644 31665 consensus_queue.cc:237] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.438158 31677 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.444101 31659 raft_consensus.cc:359] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.445369 31659 raft_consensus.cc:385] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.444882 31367 catalog_manager.cc:5697] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 0 to 1, leader changed from <none> to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 1 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.445689 31659 raft_consensus.cc:740] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.446434 31659 consensus_queue.cc:260] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.447434 31677 raft_consensus.cc:493] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.448328 31677 raft_consensus.cc:515] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.453114 31711 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:11.457221 31659 ts_tablet_manager.cc:1434] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:16:11.453092 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.458241 31659 tablet_bootstrap.cc:492] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e: Bootstrap starting.
I20260709 12:16:11.459142 31617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e" is_pre_election: true
I20260709 12:16:11.465233 31542 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
W20260709 12:16:11.465932 31435 leader_election.cc:343] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:11.468128 31712 raft_consensus.cc:493] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.468632 31712 raft_consensus.cc:515] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.471055 31712 leader_election.cc:290] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.472880 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b1a66a8e974614b59a81ab2b446c0e" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.473795 31542 raft_consensus.cc:2468] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:11.458407 31543 raft_consensus.cc:2468] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 0.
I20260709 12:16:11.474639 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b1a66a8e974614b59a81ab2b446c0e" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:11.477234 31469 raft_consensus.cc:2468] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 0.
I20260709 12:16:11.463716 31677 leader_election.cc:290] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.479180 31433 leader_election.cc:304] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:11.480036 31582 leader_election.cc:304] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:11.482709 31709 raft_consensus.cc:2804] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.483085 31709 raft_consensus.cc:493] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.483438 31709 raft_consensus.cc:3060] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.488822 31367 catalog_manager.cc:5697] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 0 to 1, leader changed from <none> to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 1 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.491194 31677 raft_consensus.cc:2804] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:11.491596 31659 tablet_bootstrap.cc:654] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:11.491628 31677 raft_consensus.cc:493] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.492148 31677 raft_consensus.cc:3060] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.496833 31709 raft_consensus.cc:515] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.500155 31709 leader_election.cc:290] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.500029 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b1a66a8e974614b59a81ab2b446c0e" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.501794 31469 raft_consensus.cc:3060] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.503319 31659 tablet_bootstrap.cc:492] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e: No bootstrap required, opened a new log
I20260709 12:16:11.503760 31659 ts_tablet_manager.cc:1403] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 12:16:11.504426 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b1a66a8e974614b59a81ab2b446c0e" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.504988 31542 raft_consensus.cc:3060] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.510057 31711 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.510965 31659 raft_consensus.cc:359] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.511601 31659 raft_consensus.cc:385] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:11.511866 31659 raft_consensus.cc:740] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.512611 31659 consensus_queue.cc:260] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.514552 31659 ts_tablet_manager.cc:1434] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:16:11.515401 31469 raft_consensus.cc:2468] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
I20260709 12:16:11.516223 31542 raft_consensus.cc:2468] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 1.
I20260709 12:16:11.516528 31583 leader_election.cc:304] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.518177 31711 raft_consensus.cc:2804] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.518507 31677 raft_consensus.cc:515] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.518887 31711 raft_consensus.cc:697] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:11.519706 31711 consensus_queue.cc:237] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.521495 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.522050 31618 raft_consensus.cc:3060] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.526839 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:11.527570 31542 raft_consensus.cc:3060] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.530213 31618 raft_consensus.cc:2468] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.530674 31677 leader_election.cc:290] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.531729 31435 leader_election.cc:304] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.532980 31677 raft_consensus.cc:2804] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:11.533761 31677 raft_consensus.cc:697] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.534839 31677 consensus_queue.cc:237] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.537807 31542 raft_consensus.cc:2468] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 1.
I20260709 12:16:11.548060 31367 catalog_manager.cc:5697] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 0 to 1, leader changed from <none> to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 1 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.555070 31365 catalog_manager.cc:5697] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.606284 31677 consensus_queue.cc:1048] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.609896 31677 consensus_queue.cc:1048] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.622535 31617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 12:16:11.623584 31617 raft_consensus.cc:606] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:11.629014 31730 consensus_queue.cc:1048] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:11.640612 31711 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.641111 31711 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.642611 31695 consensus_queue.cc:1048] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.643272 31711 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:11.645556 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" is_pre_election: true
I20260709 12:16:11.645833 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b44428e7184197b458180e0cbb111e" is_pre_election: true
I20260709 12:16:11.646406 31542 raft_consensus.cc:2393] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c471ee20d86e43cb99730b9886096a0e in current term 1: Already voted for candidate f6b44428e7184197b458180e0cbb111e in this term.
I20260709 12:16:11.647506 31582 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e; no voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e
I20260709 12:16:11.648571 31709 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:11.650568 31618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 12:16:11.651995 31618 raft_consensus.cc:606] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:11.656005 31709 raft_consensus.cc:2749] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:11.659396 31618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2336e77544314a939418eb7325c0c23b"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
mode: GRACEFUL
new_leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:46398
I20260709 12:16:11.659902 31618 raft_consensus.cc:606] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Received request to transfer leadership to TS 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:11.672186 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:44032
I20260709 12:16:11.672623 31468 raft_consensus.cc:606] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Received request to transfer leadership to TS 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:11.677165 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:44032
I20260709 12:16:11.677613 31468 raft_consensus.cc:606] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Received request to transfer leadership to TS 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:11.682204 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:44032
I20260709 12:16:11.682741 31468 raft_consensus.cc:606] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Received request to transfer leadership to TS 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:11.699518 31695 consensus_queue.cc:1048] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.709990 31677 consensus_queue.cc:1048] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.717234 31695 raft_consensus.cc:993] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: : Instructing follower 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.717644 31677 raft_consensus.cc:1081] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Signalling peer 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.719007 31543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:16:11.719552 31543 raft_consensus.cc:493] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:11.719866 31543 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.724273 31543 raft_consensus.cc:515] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.725870 31543 leader_election.cc:290] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.726529 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.726979 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.727089 31468 raft_consensus.cc:3055] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:11.727594 31468 raft_consensus.cc:740] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.727582 31618 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.728500 31468 consensus_queue.cc:260] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.729400 31468 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.733788 31618 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.734570 31677 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.734858 31510 leader_election.cc:304] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:11.735656 31667 raft_consensus.cc:2804] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.735783 31468 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.736851 31667 raft_consensus.cc:697] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Leader. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:11.737615 31667 consensus_queue.cc:237] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.744625 31365 catalog_manager.cc:5697] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 reported cstate change: term changed from 1 to 2, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.753283 31695 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.760097 31730 raft_consensus.cc:993] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: : Instructing follower 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.760543 31695 raft_consensus.cc:1081] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Signalling peer 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.761941 31543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:16:11.762425 31543 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:11.762712 31543 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.767393 31543 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.769223 31543 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.769732 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.770402 31468 raft_consensus.cc:3055] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:11.770658 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.770758 31468 raft_consensus.cc:740] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.771301 31618 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.771727 31468 consensus_queue.cc:260] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.773000 31468 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.776373 31618 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.777307 31510 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:11.777989 31667 raft_consensus.cc:2804] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.778132 31468 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.778468 31667 raft_consensus.cc:697] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Leader. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:11.779381 31667 consensus_queue.cc:237] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.785355 31364 catalog_manager.cc:5697] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 reported cstate change: term changed from 1 to 2, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.844771 31709 consensus_queue.cc:1048] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.851377 31711 raft_consensus.cc:993] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: : Instructing follower 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.851842 31680 raft_consensus.cc:1081] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Signalling peer 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.853538 31542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2336e77544314a939418eb7325c0c23b"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:56048
I20260709 12:16:11.855578 31542 raft_consensus.cc:493] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:11.855966 31542 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.861370 31709 consensus_queue.cc:1048] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.863111 31542 raft_consensus.cc:515] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.869160 31542 leader_election.cc:290] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.869859 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.870604 31469 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.871176 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.871590 31618 raft_consensus.cc:3055] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:11.871790 31618 raft_consensus.cc:740] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:11.872394 31618 consensus_queue.cc:260] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.873139 31618 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.877419 31469 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.877970 31618 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.878664 31509 leader_election.cc:304] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:11.879524 31667 raft_consensus.cc:2804] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.879937 31667 raft_consensus.cc:697] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Leader. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:11.880683 31667 consensus_queue.cc:237] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.890725 31364 catalog_manager.cc:5697] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 reported cstate change: term changed from 1 to 2, leader changed from c471ee20d86e43cb99730b9886096a0e (127.30.25.3) to 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.946287 31695 consensus_queue.cc:1048] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.956669 31730 consensus_queue.cc:1048] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:11.964885 31664 raft_consensus.cc:993] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: : Instructing follower 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.965346 31730 raft_consensus.cc:1081] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Signalling peer 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:11.966900 31542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:16:11.967480 31542 raft_consensus.cc:493] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:11.967850 31542 raft_consensus.cc:3060] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.972508 31542 raft_consensus.cc:515] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.974512 31542 leader_election.cc:290] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:11.975417 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:11.975670 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e14bbf2a0e4ffda1b17577c770cbf3" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:11.976068 31469 raft_consensus.cc:3055] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:11.976234 31618 raft_consensus.cc:3060] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.976462 31469 raft_consensus.cc:740] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:11.977298 31469 consensus_queue.cc:260] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.978299 31469 raft_consensus.cc:3060] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.981906 31618 raft_consensus.cc:2468] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.983100 31510 leader_election.cc:304] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:11.983681 31469 raft_consensus.cc:2468] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:11.983830 31667 raft_consensus.cc:2804] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.984397 31667 raft_consensus.cc:697] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Leader. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:11.985281 31667 consensus_queue.cc:237] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:11.991281 31367 catalog_manager.cc:5697] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 reported cstate change: term changed from 1 to 2, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.999955 31711 raft_consensus.cc:993] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:12.000438 31711 raft_consensus.cc:1081] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:12.001878 31469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44020
I20260709 12:16:12.002460 31469 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:12.002823 31469 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.005322 31711 consensus_queue.cc:1048] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:12.009186 31469 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:12.011703 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:12.012272 31542 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.017158 31542 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 2.
I20260709 12:16:12.018384 31469 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 2 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:12.018457 31433 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:12.019836 31730 raft_consensus.cc:2804] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:12.020476 31730 raft_consensus.cc:697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:12.021531 31730 consensus_queue.cc:237] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:12.022966 31711 consensus_queue.cc:1048] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:12.018985 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:12.024446 31618 raft_consensus.cc:3055] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:12.024837 31618 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:12.025625 31618 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:12.032351 31618 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.034824 31367 catalog_manager.cc:5697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 1 to 2, leader changed from c471ee20d86e43cb99730b9886096a0e (127.30.25.3) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 2 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:12.045365 31618 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 2.
I20260709 12:16:12.048936 31744 raft_consensus.cc:993] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:12.049458 31665 raft_consensus.cc:1081] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:12.051139 31469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44020
I20260709 12:16:12.051626 31469 raft_consensus.cc:493] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:12.051913 31469 raft_consensus.cc:3060] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.056561 31469 raft_consensus.cc:515] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:12.058599 31469 leader_election.cc:290] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 2 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:12.059185 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:12.059690 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419d1781a76e4c3f9d3c6e0e960f2b32" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:12.059789 31542 raft_consensus.cc:3060] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.060202 31618 raft_consensus.cc:3055] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:12.060516 31618 raft_consensus.cc:740] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:12.061115 31618 consensus_queue.cc:260] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:12.062057 31618 raft_consensus.cc:3060] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.064488 31542 raft_consensus.cc:2468] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 2.
I20260709 12:16:12.065371 31433 leader_election.cc:304] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:12.065991 31730 raft_consensus.cc:2804] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:12.066490 31730 raft_consensus.cc:697] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:12.066582 31618 raft_consensus.cc:2468] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 2.
I20260709 12:16:12.067303 31730 consensus_queue.cc:237] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:12.072790 31367 catalog_manager.cc:5697] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 1 to 2, leader changed from c471ee20d86e43cb99730b9886096a0e (127.30.25.3) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 2 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:12.114985 31665 consensus_queue.cc:1048] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:12.125798 31744 consensus_queue.cc:1048] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:12.219601 31469 raft_consensus.cc:1275] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.220819 31667 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.228366 31618 raft_consensus.cc:1275] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.229914 31736 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.237679 31618 raft_consensus.cc:1275] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.239012 31736 consensus_queue.cc:1048] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.247745 31469 raft_consensus.cc:1275] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.249001 31748 consensus_queue.cc:1048] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.313519 31618 raft_consensus.cc:1275] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.314761 31748 consensus_queue.cc:1048] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.323205 31469 raft_consensus.cc:1275] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.324719 31748 consensus_queue.cc:1048] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.476744 31542 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.478371 31730 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.488324 31618 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.490142 31730 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.558557 31618 raft_consensus.cc:1275] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.559755 31748 consensus_queue.cc:1048] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.566668 31469 raft_consensus.cc:1275] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.568233 31748 consensus_queue.cc:1048] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.586429 31618 raft_consensus.cc:1275] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.587512 31730 consensus_queue.cc:1048] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.594112 31542 raft_consensus.cc:1275] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.595463 31664 consensus_queue.cc:1048] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:14.717394 30820 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:16:14.718071 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:14.733790 31469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:44044
I20260709 12:16:14.734357 31469 raft_consensus.cc:606] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Received request to transfer leadership to TS c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:14.736806 31469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff39cf78a2904032b2294817a342e358"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:44044
I20260709 12:16:14.737277 31469 raft_consensus.cc:606] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Received request to transfer leadership to TS c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:14.738353 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:14.738759 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:14.907802 31677 raft_consensus.cc:993] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: : Instructing follower c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:14.908283 31753 raft_consensus.cc:1081] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Signalling peer c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:14.909631 31617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:46384
I20260709 12:16:14.910041 31617 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:14.910405 31617 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:14.914928 31617 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:14.916657 31617 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 3 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:14.917435 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:14.917761 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:14.918035 31469 raft_consensus.cc:3055] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:14.918311 31542 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:14.918378 31469 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:14.919129 31469 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:14.920076 31469 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:14.922996 31542 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 3.
I20260709 12:16:14.924209 31582 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:14.924582 31469 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 3.
I20260709 12:16:14.924893 31751 raft_consensus.cc:2804] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:14.926131 31751 raft_consensus.cc:697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:14.926882 31751 consensus_queue.cc:237] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:14.934279 31364 catalog_manager.cc:5697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 2 to 3, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 3 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.153578 31677 raft_consensus.cc:993] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: : Instructing follower c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:15.154103 31753 raft_consensus.cc:1081] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Signalling peer c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:15.155560 31617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff39cf78a2904032b2294817a342e358"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:46384
I20260709 12:16:15.156131 31617 raft_consensus.cc:493] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.156433 31617 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.160888 31617 raft_consensus.cc:515] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:15.162539 31617 leader_election.cc:290] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 2 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:15.163368 31469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:15.163686 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:15.163985 31469 raft_consensus.cc:3055] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.164216 31542 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.164311 31469 raft_consensus.cc:740] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:15.165166 31469 consensus_queue.cc:260] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:15.166348 31469 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.170883 31542 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 2.
I20260709 12:16:15.171974 31582 leader_election.cc:304] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:15.172381 31469 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 2.
I20260709 12:16:15.172636 31751 raft_consensus.cc:2804] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.173138 31751 raft_consensus.cc:697] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:15.173882 31751 consensus_queue.cc:237] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:15.180119 31364 catalog_manager.cc:5697] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 1 to 2, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 2 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.348366 31542 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:15.349517 31751 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:15.366175 31469 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:15.367954 31765 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:15.607517 31469 raft_consensus.cc:1275] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:15.609067 31765 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:15.618670 31542 raft_consensus.cc:1275] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:15.619897 31765 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:15.739647 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:15.745189 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:15.745623 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:16.746793 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:16.754099 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:16.754492 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:17.755326 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:17.760473 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:17.760850 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:18.761718 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:18.767716 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:18.768144 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:19.769320 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:19.775575 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:19.775995 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:20.777174 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:20.782920 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:20.783383 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:21.784409 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:21.789834 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:21.790345 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:22.791352 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:22.796680 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:22.797128 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:23.797951 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:23.803107 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:23.803519 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:24.832175 30820 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:16:24.837112 31617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff39cf78a2904032b2294817a342e358"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 12:16:24.838076 31617 raft_consensus.cc:606] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:24.842787 31617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 12:16:24.843242 31617 raft_consensus.cc:606] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:24.864877 31542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2336e77544314a939418eb7325c0c23b"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:38444
I20260709 12:16:24.865362 31542 raft_consensus.cc:606] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:24.869694 31542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:38444
I20260709 12:16:24.870128 31542 raft_consensus.cc:606] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:24.874178 31542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
mode: GRACEFUL
new_leader_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:38444
I20260709 12:16:24.874626 31542 raft_consensus.cc:606] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Received request to transfer leadership to TS f6b44428e7184197b458180e0cbb111e
I20260709 12:16:25.076071 31752 raft_consensus.cc:993] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.076565 31780 raft_consensus.cc:1081] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.078063 31468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44018
I20260709 12:16:25.078603 31468 raft_consensus.cc:493] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.078948 31468 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.084760 31468 raft_consensus.cc:515] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.086792 31468 leader_election.cc:290] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:25.087560 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:25.087848 31617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbae48e291ac4ed5a3f172eaae5a93fe" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:25.088310 31542 raft_consensus.cc:3055] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:25.088430 31617 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.088619 31542 raft_consensus.cc:740] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:25.089490 31542 consensus_queue.cc:260] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.090428 31542 raft_consensus.cc:3060] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.093410 31617 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.094655 31435 leader_election.cc:304] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:25.095145 31542 raft_consensus.cc:2468] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.095504 31753 raft_consensus.cc:2804] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:25.096441 31753 raft_consensus.cc:697] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:25.097195 31753 consensus_queue.cc:237] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.104287 31364 catalog_manager.cc:5697] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 2 to 3, leader changed from 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 3 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:25.147215 31752 raft_consensus.cc:993] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.147720 31752 raft_consensus.cc:1081] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.149322 31469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44018
I20260709 12:16:25.149834 31469 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.150177 31469 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.152578 31777 raft_consensus.cc:993] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.153151 31787 raft_consensus.cc:1081] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.155030 31468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44020
I20260709 12:16:25.155638 31468 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.156018 31468 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:25.156682 31469 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.158247 31469 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:25.158893 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:25.159309 31617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:25.159430 31542 raft_consensus.cc:3055] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:25.159845 31617 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.159818 31542 raft_consensus.cc:740] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:25.160856 31542 consensus_queue.cc:260] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.162015 31542 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.161928 31468 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.164494 31468 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 4 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:25.164544 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:25.165361 31543 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:25.166811 31617 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.168368 31435 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:25.169119 31753 raft_consensus.cc:2804] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:25.169515 31753 raft_consensus.cc:697] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:25.169536 31542 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.164775 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:25.171542 31618 raft_consensus.cc:3055] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:25.171844 31618 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:25.171633 31753 consensus_queue.cc:237] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.172650 31618 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.173224 31787 raft_consensus.cc:993] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.173626 31787 raft_consensus.cc:1081] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.173851 31618 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:25.175319 31468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff39cf78a2904032b2294817a342e358"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44020
I20260709 12:16:25.175853 31468 raft_consensus.cc:493] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.176218 31468 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.181609 31543 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 4.
I20260709 12:16:25.182953 31468 raft_consensus.cc:515] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:25.185448 31433 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:25.186141 31468 leader_election.cc:290] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Requested vote from peers c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:25.186831 31781 raft_consensus.cc:2804] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:25.187270 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:25.187777 31542 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.184171 31618 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 4.
I20260709 12:16:25.190721 31617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:25.191550 31617 raft_consensus.cc:3055] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:25.192042 31617 raft_consensus.cc:740] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:25.191043 31364 catalog_manager.cc:5697] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 2 to 3, leader changed from 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 3 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:25.193245 31617 consensus_queue.cc:260] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:25.194950 31617 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.193904 31542 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.196940 31433 leader_election.cc:304] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:25.197546 31753 raft_consensus.cc:2804] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:25.197940 31753 raft_consensus.cc:697] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:25.198668 31753 consensus_queue.cc:237] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:25.202808 31781 raft_consensus.cc:697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:25.203570 31617 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.203850 31781 consensus_queue.cc:237] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.212118 31364 catalog_manager.cc:5697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 3 to 4, leader changed from c471ee20d86e43cb99730b9886096a0e (127.30.25.3) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:25.223841 31364 catalog_manager.cc:5697] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 2 to 3, leader changed from c471ee20d86e43cb99730b9886096a0e (127.30.25.3) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 3 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:25.457723 31780 raft_consensus.cc:993] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: : Instructing follower f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.458160 31752 raft_consensus.cc:1081] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Signalling peer f6b44428e7184197b458180e0cbb111e to start an election
I20260709 12:16:25.459745 31468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2336e77544314a939418eb7325c0c23b"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
 from {username='slave'} at 127.0.0.1:44018
I20260709 12:16:25.460306 31468 raft_consensus.cc:493] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.460680 31468 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.465471 31468 raft_consensus.cc:515] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.466905 31468 leader_election.cc:290] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Requested vote from peers 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:25.467818 31617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:25.468384 31617 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.468403 31542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2336e77544314a939418eb7325c0c23b" candidate_uuid: "f6b44428e7184197b458180e0cbb111e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:25.469007 31542 raft_consensus.cc:3055] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:25.469218 31542 raft_consensus.cc:740] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:25.469852 31542 consensus_queue.cc:260] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.470659 31542 raft_consensus.cc:3060] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:25.473102 31617 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.474100 31435 leader_election.cc:304] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c471ee20d86e43cb99730b9886096a0e, f6b44428e7184197b458180e0cbb111e; no voters: 
I20260709 12:16:25.474658 31542 raft_consensus.cc:2468] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b44428e7184197b458180e0cbb111e in term 3.
I20260709 12:16:25.474830 31781 raft_consensus.cc:2804] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:25.475282 31781 raft_consensus.cc:697] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Becoming Leader. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:25.476065 31781 consensus_queue.cc:237] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:25.482853 31364 catalog_manager.cc:5697] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e reported cstate change: term changed from 2 to 3, leader changed from 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2) to f6b44428e7184197b458180e0cbb111e (127.30.25.1). New cstate: current_term: 3 leader_uuid: "f6b44428e7184197b458180e0cbb111e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:25.500432 31542 raft_consensus.cc:1275] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.501456 31781 consensus_queue.cc:1048] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.510085 31617 raft_consensus.cc:1275] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.511605 31753 consensus_queue.cc:1048] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.577833 31617 raft_consensus.cc:1275] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.579089 31753 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.587730 31542 raft_consensus.cc:1275] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.589104 31781 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.688330 31542 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:25.689438 31781 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:25.697938 31617 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:25.699676 31753 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:25.711011 31542 raft_consensus.cc:1275] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.712297 31753 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.721556 31617 raft_consensus.cc:1275] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.723160 31789 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.913667 31542 raft_consensus.cc:1275] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.914800 31789 consensus_queue.cc:1048] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:25.925179 31617 raft_consensus.cc:1275] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Refusing update from remote peer f6b44428e7184197b458180e0cbb111e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.926476 31753 consensus_queue.cc:1048] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.910610 30820 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:16:27.911259 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.918201 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:44044
I20260709 12:16:27.918807 31468 raft_consensus.cc:606] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 LEADER]: Received request to transfer leadership to TS c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:27.921141 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de9867050e944946bf85cdb1d973350b"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:44044
I20260709 12:16:27.921586 31468 raft_consensus.cc:606] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Received request to transfer leadership to TS 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:27.923799 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:44044
I20260709 12:16:27.924212 31468 raft_consensus.cc:606] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Received request to transfer leadership to TS c471ee20d86e43cb99730b9886096a0e
I20260709 12:16:27.926167 31468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff39cf78a2904032b2294817a342e358"
dest_uuid: "f6b44428e7184197b458180e0cbb111e"
mode: GRACEFUL
new_leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:44044
I20260709 12:16:27.926602 31468 raft_consensus.cc:606] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Received request to transfer leadership to TS 3a049e04c4234cdfbfa0b0ad6f51c863
I20260709 12:16:27.927477 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:27.927912 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:28.116647 31753 raft_consensus.cc:993] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: : Instructing follower c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:28.117074 31806 raft_consensus.cc:1081] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 LEADER]: Signalling peer c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:28.118371 31617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:46384
I20260709 12:16:28.118788 31617 raft_consensus.cc:493] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.119046 31617 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:28.122977 31617 raft_consensus.cc:515] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.124997 31617 leader_election.cc:290] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 5 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:28.125777 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:28.126003 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa3aab5bbf44189918b0cc20a13b6a8" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:28.126390 31468 raft_consensus.cc:3055] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:28.126557 31543 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:28.126718 31468 raft_consensus.cc:740] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:28.127513 31468 consensus_queue.cc:260] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.128656 31468 raft_consensus.cc:3060] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:28.133612 31543 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 5.
I20260709 12:16:28.135003 31468 raft_consensus.cc:2468] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 5.
I20260709 12:16:28.135097 31582 leader_election.cc:304] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:28.136025 31787 raft_consensus.cc:2804] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:28.136926 31787 raft_consensus.cc:697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 5 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:28.137723 31787 consensus_queue.cc:237] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.143740 31364 catalog_manager.cc:5697] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 4 to 5, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 5 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:28.351055 31806 raft_consensus.cc:993] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: : Instructing follower 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:28.351478 31753 raft_consensus.cc:1081] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Signalling peer 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:28.352689 31543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff39cf78a2904032b2294817a342e358"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:16:28.353087 31543 raft_consensus.cc:493] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.353333 31543 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.357693 31543 raft_consensus.cc:515] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:28.359738 31543 leader_election.cc:290] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 4 election: Requested vote from peers c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637), f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751)
I20260709 12:16:28.360378 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:28.360704 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff39cf78a2904032b2294817a342e358" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:28.360936 31618 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.361282 31468 raft_consensus.cc:3055] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:28.361616 31468 raft_consensus.cc:740] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:28.362341 31468 consensus_queue.cc:260] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:28.363421 31468 raft_consensus.cc:3060] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.365820 31618 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 4.
I20260709 12:16:28.366849 31510 leader_election.cc:304] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:28.367426 31752 raft_consensus.cc:2804] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:28.368520 31752 raft_consensus.cc:697] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 LEADER]: Becoming Leader. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:28.368875 31468 raft_consensus.cc:2468] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 4.
I20260709 12:16:28.369328 31752 consensus_queue.cc:237] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } }
I20260709 12:16:28.375423 31364 catalog_manager.cc:5697] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 reported cstate change: term changed from 3 to 4, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2). New cstate: current_term: 4 leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:28.485601 31806 raft_consensus.cc:993] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: : Instructing follower c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:28.486081 31753 raft_consensus.cc:1081] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Signalling peer c471ee20d86e43cb99730b9886096a0e to start an election
I20260709 12:16:28.487408 31618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900"
dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
 from {username='slave'} at 127.0.0.1:46384
I20260709 12:16:28.487804 31618 raft_consensus.cc:493] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.488036 31618 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.492059 31618 raft_consensus.cc:515] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.493469 31618 leader_election.cc:290] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 4 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883)
I20260709 12:16:28.494223 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:28.494539 31543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80bf20b2b2942a3b7d0c76fdd769900" candidate_uuid: "c471ee20d86e43cb99730b9886096a0e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
I20260709 12:16:28.494740 31468 raft_consensus.cc:3055] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:28.495052 31543 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.495106 31468 raft_consensus.cc:740] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:28.495812 31468 consensus_queue.cc:260] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.496672 31468 raft_consensus.cc:3060] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.499912 31543 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 4.
I20260709 12:16:28.500932 31582 leader_election.cc:304] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:28.501457 31468 raft_consensus.cc:2468] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c471ee20d86e43cb99730b9886096a0e in term 4.
I20260709 12:16:28.501669 31787 raft_consensus.cc:2804] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:28.502187 31787 raft_consensus.cc:697] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 4 LEADER]: Becoming Leader. State: Replica: c471ee20d86e43cb99730b9886096a0e, State: Running, Role: LEADER
I20260709 12:16:28.502415 31806 raft_consensus.cc:993] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: : Instructing follower 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:28.503052 31753 raft_consensus.cc:1081] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Signalling peer 3a049e04c4234cdfbfa0b0ad6f51c863 to start an election
I20260709 12:16:28.503166 31787 consensus_queue.cc:237] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.504436 31542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de9867050e944946bf85cdb1d973350b"
dest_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:16:28.504914 31542 raft_consensus.cc:493] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.505211 31542 raft_consensus.cc:3060] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:28.510694 31364 catalog_manager.cc:5697] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e reported cstate change: term changed from 3 to 4, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to c471ee20d86e43cb99730b9886096a0e (127.30.25.3). New cstate: current_term: 4 leader_uuid: "c471ee20d86e43cb99730b9886096a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:28.511160 31542 raft_consensus.cc:515] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.512696 31542 leader_election.cc:290] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Requested vote from peers f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751), c471ee20d86e43cb99730b9886096a0e (127.30.25.3:42637)
I20260709 12:16:28.513247 31468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9867050e944946bf85cdb1d973350b" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6b44428e7184197b458180e0cbb111e"
I20260709 12:16:28.513447 31618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de9867050e944946bf85cdb1d973350b" candidate_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c471ee20d86e43cb99730b9886096a0e"
I20260709 12:16:28.513845 31468 raft_consensus.cc:3055] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:28.513931 31618 raft_consensus.cc:3060] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:28.514192 31468 raft_consensus.cc:740] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f6b44428e7184197b458180e0cbb111e, State: Running, Role: LEADER
I20260709 12:16:28.514966 31468 consensus_queue.cc:260] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.515738 31468 raft_consensus.cc:3060] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:28.519234 31618 raft_consensus.cc:2468] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:28.520277 31510 leader_election.cc:304] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a049e04c4234cdfbfa0b0ad6f51c863, c471ee20d86e43cb99730b9886096a0e; no voters: 
I20260709 12:16:28.520915 31752 raft_consensus.cc:2804] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:28.521345 31752 raft_consensus.cc:697] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Becoming Leader. State: Replica: 3a049e04c4234cdfbfa0b0ad6f51c863, State: Running, Role: LEADER
I20260709 12:16:28.521669 31468 raft_consensus.cc:2468] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a049e04c4234cdfbfa0b0ad6f51c863 in term 2.
I20260709 12:16:28.522024 31752 consensus_queue.cc:237] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } }
I20260709 12:16:28.528959 31364 catalog_manager.cc:5697] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 reported cstate change: term changed from 1 to 2, leader changed from f6b44428e7184197b458180e0cbb111e (127.30.25.1) to 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:28.532579 31542 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 5 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:28.533867 31787 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:28.545976 31468 raft_consensus.cc:1275] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 5 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:28.547647 31820 consensus_queue.cc:1048] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:28.890013 31468 raft_consensus.cc:1275] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:28.891150 31752 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:28.900252 31618 raft_consensus.cc:1275] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:28.901669 31825 consensus_queue.cc:1048] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:28.909345 31617 raft_consensus.cc:1275] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.910480 31752 consensus_queue.cc:1048] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c471ee20d86e43cb99730b9886096a0e" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.918686 31468 raft_consensus.cc:1275] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Refusing update from remote peer 3a049e04c4234cdfbfa0b0ad6f51c863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.920599 31752 consensus_queue.cc:1048] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.925722 31542 raft_consensus.cc:1275] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:28.927000 31820 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a049e04c4234cdfbfa0b0ad6f51c863" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:28.929123 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:28.935941 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:28.936645 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:28.943238 31468 raft_consensus.cc:1275] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Refusing update from remote peer c471ee20d86e43cb99730b9886096a0e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:28.944710 31807 consensus_queue.cc:1048] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b44428e7184197b458180e0cbb111e" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:29.938000 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.945380 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:29.945837 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:30.946861 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:30.952111 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:30.952564 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:31.953436 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:31.958894 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:31.959306 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:32.960214 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:32.965466 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:32.965823 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:33.966743 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:33.972688 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:33.973161 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:34.974051 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:34.979427 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:34.979763 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:35.980598 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:35.985597 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:35.986088 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:36.987159 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:36.992437 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:36.992945 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:38.023869 30820 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:16:38.024369 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
I20260709 12:16:38.051918 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:38.052649 30820 tablet_replica.cc:333] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.053388 30820 raft_consensus.cc:2243] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.054239 30820 raft_consensus.cc:2272] T b6b1a66a8e974614b59a81ab2b446c0e P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:38.054576 31583 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260709 12:16:38.056574 30820 tablet_replica.cc:333] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.057334 30820 raft_consensus.cc:2243] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:16:38.057866 31583 consensus_peers.cc:597] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e -> Peer f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751): Couldn't send request to peer f6b44428e7184197b458180e0cbb111e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.058362 30820 raft_consensus.cc:2272] T dbae48e291ac4ed5a3f172eaae5a93fe P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.060211 30820 tablet_replica.cc:333] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.060738 30820 raft_consensus.cc:2243] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.061164 30820 raft_consensus.cc:2272] T f80bf20b2b2942a3b7d0c76fdd769900 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.062901 30820 tablet_replica.cc:333] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.063416 30820 raft_consensus.cc:2243] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.063838 30820 raft_consensus.cc:2272] T de9867050e944946bf85cdb1d973350b P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.065544 30820 tablet_replica.cc:333] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.066077 30820 raft_consensus.cc:2243] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.066538 30820 raft_consensus.cc:2272] T 4fa3aab5bbf44189918b0cc20a13b6a8 P f6b44428e7184197b458180e0cbb111e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.068254 30820 tablet_replica.cc:333] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.068732 30820 raft_consensus.cc:2243] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.069430 30820 raft_consensus.cc:2272] T 419d1781a76e4c3f9d3c6e0e960f2b32 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.071182 30820 tablet_replica.cc:333] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.071668 30820 raft_consensus.cc:2243] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.072095 30820 raft_consensus.cc:2272] T 29e14bbf2a0e4ffda1b17577c770cbf3 P f6b44428e7184197b458180e0cbb111e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.073995 30820 tablet_replica.cc:333] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.074661 30820 raft_consensus.cc:2243] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.075553 30820 raft_consensus.cc:2272] T bee4f9e6388544688e015d1c780bbe6b P f6b44428e7184197b458180e0cbb111e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.077520 30820 tablet_replica.cc:333] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.078111 30820 raft_consensus.cc:2243] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.078820 30820 raft_consensus.cc:2272] T 2336e77544314a939418eb7325c0c23b P f6b44428e7184197b458180e0cbb111e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.080592 30820 tablet_replica.cc:333] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e: stopping tablet replica
I20260709 12:16:38.081061 30820 raft_consensus.cc:2243] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.081446 30820 raft_consensus.cc:2272] T ff39cf78a2904032b2294817a342e358 P f6b44428e7184197b458180e0cbb111e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.111120 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
I20260709 12:16:38.132350 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
W20260709 12:16:38.137600 31509 consensus_peers.cc:597] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 -> Peer f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751): Couldn't send request to peer f6b44428e7184197b458180e0cbb111e. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.161332 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:38.161979 30820 tablet_replica.cc:333] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.162694 30820 raft_consensus.cc:2243] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.163173 30820 raft_consensus.cc:2272] T b6b1a66a8e974614b59a81ab2b446c0e P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.164870 30820 tablet_replica.cc:333] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.165375 30820 raft_consensus.cc:2243] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.165796 30820 raft_consensus.cc:2272] T dbae48e291ac4ed5a3f172eaae5a93fe P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.167500 30820 tablet_replica.cc:333] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.167996 30820 raft_consensus.cc:2243] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.168408 30820 raft_consensus.cc:2272] T 2336e77544314a939418eb7325c0c23b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.169889 30820 tablet_replica.cc:333] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.170471 30820 raft_consensus.cc:2243] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.171193 30820 raft_consensus.cc:2272] T ff39cf78a2904032b2294817a342e358 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.172773 30820 tablet_replica.cc:333] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.173264 30820 raft_consensus.cc:2243] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.173974 30820 raft_consensus.cc:2272] T de9867050e944946bf85cdb1d973350b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.175575 30820 tablet_replica.cc:333] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.176060 30820 raft_consensus.cc:2243] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.176769 30820 raft_consensus.cc:2272] T 29e14bbf2a0e4ffda1b17577c770cbf3 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.178241 30820 tablet_replica.cc:333] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.178748 30820 raft_consensus.cc:2243] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.179111 30820 raft_consensus.cc:2272] T bee4f9e6388544688e015d1c780bbe6b P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.180619 30820 tablet_replica.cc:333] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.181061 30820 raft_consensus.cc:2243] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.181517 30820 raft_consensus.cc:2272] T 4fa3aab5bbf44189918b0cc20a13b6a8 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.183077 30820 tablet_replica.cc:333] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.183525 30820 raft_consensus.cc:2243] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.183920 30820 raft_consensus.cc:2272] T 419d1781a76e4c3f9d3c6e0e960f2b32 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.185362 30820 tablet_replica.cc:333] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863: stopping tablet replica
I20260709 12:16:38.185832 30820 raft_consensus.cc:2243] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.186249 30820 raft_consensus.cc:2272] T f80bf20b2b2942a3b7d0c76fdd769900 P 3a049e04c4234cdfbfa0b0ad6f51c863 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.214217 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
W20260709 12:16:38.220577 31583 consensus_peers.cc:597] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e -> Peer f6b44428e7184197b458180e0cbb111e (127.30.25.1:40751): Couldn't send request to peer f6b44428e7184197b458180e0cbb111e. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.235337 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
W20260709 12:16:38.259960 31582 consensus_peers.cc:597] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e -> Peer 3a049e04c4234cdfbfa0b0ad6f51c863 (127.30.25.2:45883): Couldn't send request to peer 3a049e04c4234cdfbfa0b0ad6f51c863. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:45883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.261220 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:38.261971 30820 tablet_replica.cc:333] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.262603 30820 raft_consensus.cc:2243] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.263379 30820 raft_consensus.cc:2272] T b6b1a66a8e974614b59a81ab2b446c0e P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.265763 30820 tablet_replica.cc:333] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.266536 30820 raft_consensus.cc:2243] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.267405 30820 raft_consensus.cc:2272] T f80bf20b2b2942a3b7d0c76fdd769900 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.269084 30820 tablet_replica.cc:333] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.269613 30820 raft_consensus.cc:2243] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.270057 30820 raft_consensus.cc:2272] T 29e14bbf2a0e4ffda1b17577c770cbf3 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.271687 30820 tablet_replica.cc:333] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.272166 30820 raft_consensus.cc:2243] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.272583 30820 raft_consensus.cc:2272] T bee4f9e6388544688e015d1c780bbe6b P c471ee20d86e43cb99730b9886096a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.274155 30820 tablet_replica.cc:333] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.274655 30820 raft_consensus.cc:2243] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.275354 30820 raft_consensus.cc:2272] T 4fa3aab5bbf44189918b0cc20a13b6a8 P c471ee20d86e43cb99730b9886096a0e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.276963 30820 tablet_replica.cc:333] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.277490 30820 raft_consensus.cc:2243] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.277897 30820 raft_consensus.cc:2272] T 419d1781a76e4c3f9d3c6e0e960f2b32 P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.279595 30820 tablet_replica.cc:333] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.280076 30820 raft_consensus.cc:2243] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.280450 30820 raft_consensus.cc:2272] T dbae48e291ac4ed5a3f172eaae5a93fe P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.282004 30820 tablet_replica.cc:333] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.282580 30820 raft_consensus.cc:2243] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.282987 30820 raft_consensus.cc:2272] T de9867050e944946bf85cdb1d973350b P c471ee20d86e43cb99730b9886096a0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.284592 30820 tablet_replica.cc:333] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.285037 30820 raft_consensus.cc:2243] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.285456 30820 raft_consensus.cc:2272] T 2336e77544314a939418eb7325c0c23b P c471ee20d86e43cb99730b9886096a0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.287052 30820 tablet_replica.cc:333] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e: stopping tablet replica
I20260709 12:16:38.287500 30820 raft_consensus.cc:2243] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.287899 30820 raft_consensus.cc:2272] T ff39cf78a2904032b2294817a342e358 P c471ee20d86e43cb99730b9886096a0e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.316612 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
I20260709 12:16:38.336177 30820 master.cc:562] Master@127.30.25.62:43585 shutting down...
I20260709 12:16:38.361738 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.362301 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.362669 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9db0ec0090ec47c39735a1f7d07a7ca5: stopping tablet replica
I20260709 12:16:38.382453 30820 master.cc:584] Master@127.30.25.62:43585 shutdown complete.
3a049e04c4234cdfbfa0b0ad6f51c863  4
c471ee20d86e43cb99730b9886096a0e  1
f6b44428e7184197b458180e0cbb111e  5
3a049e04c4234cdfbfa0b0ad6f51c863  4
c471ee20d86e43cb99730b9886096a0e  3
f6b44428e7184197b458180e0cbb111e  3
3a049e04c4234cdfbfa0b0ad6f51c863  1
c471ee20d86e43cb99730b9886096a0e  1
f6b44428e7184197b458180e0cbb111e  8
3a049e04c4234cdfbfa0b0ad6f51c863  3
c471ee20d86e43cb99730b9886096a0e  3
f6b44428e7184197b458180e0cbb111e  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28542 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:16:38.423504 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:33695
I20260709 12:16:38.424650 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:38.430604 31845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:38.430667 31846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:38.432359 31848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:38.433166 30820 server_base.cc:1061] running on GCE node
I20260709 12:16:38.434175 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:38.434795 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:38.434926 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599398434913 us; error 0 us; skew 500 ppm
I20260709 12:16:38.435493 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:38.437928 30820 webserver.cc:533] Webserver started at http://127.30.25.62:43821/ using document root <none> and password file <none>
I20260709 12:16:38.438470 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:38.438663 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:38.438941 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:38.440088 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "a0887d9cdc8044caa86af7acdb454104"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.444516 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:16:38.447902 31853 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.448611 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:38.448923 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "a0887d9cdc8044caa86af7acdb454104"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.449184 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:38.469326 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:38.470604 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:38.508122 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:33695
I20260709 12:16:38.508220 31904 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:33695 every 8 connection(s)
I20260709 12:16:38.511876 31905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:38.521772 31905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Bootstrap starting.
I20260709 12:16:38.526365 31905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:38.530361 31905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: No bootstrap required, opened a new log
I20260709 12:16:38.532472 31905 raft_consensus.cc:359] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0887d9cdc8044caa86af7acdb454104" member_type: VOTER }
I20260709 12:16:38.532878 31905 raft_consensus.cc:385] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:38.533056 31905 raft_consensus.cc:740] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0887d9cdc8044caa86af7acdb454104, State: Initialized, Role: FOLLOWER
I20260709 12:16:38.533510 31905 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0887d9cdc8044caa86af7acdb454104" member_type: VOTER }
I20260709 12:16:38.533922 31905 raft_consensus.cc:399] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:38.534117 31905 raft_consensus.cc:493] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:38.534404 31905 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:38.538810 31905 raft_consensus.cc:515] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0887d9cdc8044caa86af7acdb454104" member_type: VOTER }
I20260709 12:16:38.539268 31905 leader_election.cc:304] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0887d9cdc8044caa86af7acdb454104; no voters: 
I20260709 12:16:38.540241 31905 leader_election.cc:290] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:38.540637 31908 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:38.542066 31908 raft_consensus.cc:697] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 1 LEADER]: Becoming Leader. State: Replica: a0887d9cdc8044caa86af7acdb454104, State: Running, Role: LEADER
I20260709 12:16:38.542769 31908 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0887d9cdc8044caa86af7acdb454104" member_type: VOTER }
I20260709 12:16:38.543241 31905 sys_catalog.cc:565] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:38.548167 31910 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0887d9cdc8044caa86af7acdb454104. Latest consensus state: current_term: 1 leader_uuid: "a0887d9cdc8044caa86af7acdb454104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0887d9cdc8044caa86af7acdb454104" member_type: VOTER } }
I20260709 12:16:38.549052 31910 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:38.548399 31909 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0887d9cdc8044caa86af7acdb454104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0887d9cdc8044caa86af7acdb454104" member_type: VOTER } }
I20260709 12:16:38.550472 31909 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:38.551267 31916 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:38.556007 31916 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:38.560487 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:38.565212 31916 catalog_manager.cc:1383] Generated new cluster ID: 399dccaf960e4eef941171f151bd4f1b
I20260709 12:16:38.565474 31916 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:38.594180 31916 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:38.595499 31916 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:38.611168 31916 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Generated new TSK 0
I20260709 12:16:38.611756 31916 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:38.627238 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:38.633819 31926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:38.635120 31927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:38.636875 30820 server_base.cc:1061] running on GCE node
W20260709 12:16:38.637141 31929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:38.638031 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:38.638278 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:38.638494 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599398638475 us; error 0 us; skew 500 ppm
I20260709 12:16:38.639024 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:38.641244 30820 webserver.cc:533] Webserver started at http://127.30.25.1:41375/ using document root <none> and password file <none>
I20260709 12:16:38.641731 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:38.641937 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:38.642215 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:38.643307 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "f05e4741d35c4cf3a31358c56e65b30c"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.648063 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:16:38.651412 31934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.652137 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:16:38.652423 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "f05e4741d35c4cf3a31358c56e65b30c"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.652734 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:38.669829 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:38.671151 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:38.672659 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:38.674942 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:38.675169 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.675402 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:38.675585 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.716420 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:40903
I20260709 12:16:38.716533 31996 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:40903 every 8 connection(s)
I20260709 12:16:38.721875 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:38.730512 32001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:38.731313 32002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:38.733161 30820 server_base.cc:1061] running on GCE node
W20260709 12:16:38.734305 32004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:38.735320 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:38.735689 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:38.735934 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599398735914 us; error 0 us; skew 500 ppm
I20260709 12:16:38.736559 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:38.739480 30820 webserver.cc:533] Webserver started at http://127.30.25.2:38371/ using document root <none> and password file <none>
I20260709 12:16:38.740200 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:38.740531 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:38.740895 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:38.742039 31997 heartbeater.cc:344] Connected to a master server at 127.30.25.62:33695
I20260709 12:16:38.742462 31997 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:38.742525 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "3836f9b5cd36488da56d4540dc9d80ec"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.743228 31997 heartbeater.cc:507] Master 127.30.25.62:33695 requested a full tablet report, sending...
I20260709 12:16:38.745718 31870 ts_manager.cc:194] Registered new tserver with Master: f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:38.748359 31870 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55344
I20260709 12:16:38.748831 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:38.752938 32009 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.753829 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:38.754149 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "3836f9b5cd36488da56d4540dc9d80ec"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.754550 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:38.775138 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:38.776481 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:38.778021 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:38.780378 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:38.780620 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.780915 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:38.781101 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.823254 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:40293
I20260709 12:16:38.823364 32071 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:40293 every 8 connection(s)
I20260709 12:16:38.828454 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:38.836432 32075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:38.837945 32076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:38.839287 32078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:38.840302 30820 server_base.cc:1061] running on GCE node
I20260709 12:16:38.841286 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:38.841509 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:38.841694 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599398841677 us; error 0 us; skew 500 ppm
I20260709 12:16:38.842097 32072 heartbeater.cc:344] Connected to a master server at 127.30.25.62:33695
I20260709 12:16:38.842415 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:38.842535 32072 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:38.843457 32072 heartbeater.cc:507] Master 127.30.25.62:33695 requested a full tablet report, sending...
I20260709 12:16:38.845288 30820 webserver.cc:533] Webserver started at http://127.30.25.3:38743/ using document root <none> and password file <none>
I20260709 12:16:38.845647 31870 ts_manager.cc:194] Registered new tserver with Master: 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:38.845803 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:38.846101 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:38.846414 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:38.847148 31870 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55356
I20260709 12:16:38.847568 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "38c927f0ee104b57972899f5b27aa99b"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.852325 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:16:38.855566 32083 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.856292 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:16:38.856577 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "38c927f0ee104b57972899f5b27aa99b"
format_stamp: "Formatted at 2026-07-09 12:16:38 on dist-test-slave-rldg"
I20260709 12:16:38.856858 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:38.868633 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:38.869692 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:38.871009 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:38.873065 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:38.873262 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.873536 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:38.873693 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:38.912883 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:35255
I20260709 12:16:38.913018 32145 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:35255 every 8 connection(s)
I20260709 12:16:38.926853 32146 heartbeater.cc:344] Connected to a master server at 127.30.25.62:33695
I20260709 12:16:38.927241 32146 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:38.927996 32146 heartbeater.cc:507] Master 127.30.25.62:33695 requested a full tablet report, sending...
I20260709 12:16:38.929775 31870 ts_manager.cc:194] Registered new tserver with Master: 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:38.929920 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014094121s
I20260709 12:16:38.931111 31870 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55370
I20260709 12:16:38.955102 31870 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55386:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:38.961194 31870 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:39.193564 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:39.193315 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 71e0cf115549454c894335e70c43843a (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:39.194769 32109 tablet_service.cc:1511] Processing CreateTablet for tablet 786db0eb7e9b431ba90c5ad880c46c9e (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:39.195746 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.196406 32109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 786db0eb7e9b431ba90c5ad880c46c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.200666 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e0cf115549454c894335e70c43843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.200881 32108 tablet_service.cc:1511] Processing CreateTablet for tablet fbb363b96dbf430998ab5641d3b03f46 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:39.202266 32108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb363b96dbf430998ab5641d3b03f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.206087 31962 tablet_service.cc:1511] Processing CreateTablet for tablet 71e0cf115549454c894335e70c43843a (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:39.207747 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e0cf115549454c894335e70c43843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.209448 32107 tablet_service.cc:1511] Processing CreateTablet for tablet ef2f2222a1ac4f6f923fdee4de1f48dc (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:39.210912 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef2f2222a1ac4f6f923fdee4de1f48dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.211611 31961 tablet_service.cc:1511] Processing CreateTablet for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:39.213088 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.215634 31960 tablet_service.cc:1511] Processing CreateTablet for tablet 786db0eb7e9b431ba90c5ad880c46c9e (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:39.217165 31960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 786db0eb7e9b431ba90c5ad880c46c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.218993 32037 tablet_service.cc:1511] Processing CreateTablet for tablet 71e0cf115549454c894335e70c43843a (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:39.219600 31959 tablet_service.cc:1511] Processing CreateTablet for tablet fbb363b96dbf430998ab5641d3b03f46 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:39.220525 32037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e0cf115549454c894335e70c43843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.221097 31959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb363b96dbf430998ab5641d3b03f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.223551 32035 tablet_service.cc:1511] Processing CreateTablet for tablet 786db0eb7e9b431ba90c5ad880c46c9e (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:39.224886 32035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 786db0eb7e9b431ba90c5ad880c46c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.225924 31958 tablet_service.cc:1511] Processing CreateTablet for tablet ef2f2222a1ac4f6f923fdee4de1f48dc (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:39.227353 31958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef2f2222a1ac4f6f923fdee4de1f48dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.227586 32034 tablet_service.cc:1511] Processing CreateTablet for tablet fbb363b96dbf430998ab5641d3b03f46 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:39.228952 32034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb363b96dbf430998ab5641d3b03f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.232100 32036 tablet_service.cc:1511] Processing CreateTablet for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:39.231582 32033 tablet_service.cc:1511] Processing CreateTablet for tablet ef2f2222a1ac4f6f923fdee4de1f48dc (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:39.237749 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 193a74284231469f9229c961dfaff799 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:39.239395 31957 tablet_service.cc:1511] Processing CreateTablet for tablet 193a74284231469f9229c961dfaff799 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:39.240170 32031 tablet_service.cc:1511] Processing CreateTablet for tablet 88a84a8ea71c4258b378603a0827a768 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:39.240741 31957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193a74284231469f9229c961dfaff799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.233475 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.245714 32106 tablet_service.cc:1511] Processing CreateTablet for tablet 193a74284231469f9229c961dfaff799 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:39.247161 32106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193a74284231469f9229c961dfaff799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.246961 32030 tablet_service.cc:1511] Processing CreateTablet for tablet b83e7300b3fe4d21a97d941dee98fa81 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:39.250416 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193a74284231469f9229c961dfaff799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.249991 32029 tablet_service.cc:1511] Processing CreateTablet for tablet 2f93a3ac5b02416a8d1d2547d9a9c726 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:39.251144 32031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a84a8ea71c4258b378603a0827a768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.251802 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b83e7300b3fe4d21a97d941dee98fa81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.254029 32029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f93a3ac5b02416a8d1d2547d9a9c726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.255004 32105 tablet_service.cc:1511] Processing CreateTablet for tablet b83e7300b3fe4d21a97d941dee98fa81 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:39.255373 32104 tablet_service.cc:1511] Processing CreateTablet for tablet 2f93a3ac5b02416a8d1d2547d9a9c726 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:39.256283 32033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef2f2222a1ac4f6f923fdee4de1f48dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.256448 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b83e7300b3fe4d21a97d941dee98fa81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.257287 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f93a3ac5b02416a8d1d2547d9a9c726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.260126 31956 tablet_service.cc:1511] Processing CreateTablet for tablet 88a84a8ea71c4258b378603a0827a768 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:39.261641 31956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a84a8ea71c4258b378603a0827a768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.261476 32103 tablet_service.cc:1511] Processing CreateTablet for tablet 12a3b65ef9024a64b336b00ad51e294b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:39.262948 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a3b65ef9024a64b336b00ad51e294b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.264138 31955 tablet_service.cc:1511] Processing CreateTablet for tablet b83e7300b3fe4d21a97d941dee98fa81 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:39.265518 31955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b83e7300b3fe4d21a97d941dee98fa81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.268046 32102 tablet_service.cc:1511] Processing CreateTablet for tablet 4633b71e43824d84895ff136e5765fb1 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:39.269455 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 12a3b65ef9024a64b336b00ad51e294b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:39.271711 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a3b65ef9024a64b336b00ad51e294b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.269397 32102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633b71e43824d84895ff136e5765fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.269096 31954 tablet_service.cc:1511] Processing CreateTablet for tablet 2f93a3ac5b02416a8d1d2547d9a9c726 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:39.276531 31954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f93a3ac5b02416a8d1d2547d9a9c726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.279425 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 12a3b65ef9024a64b336b00ad51e294b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:39.299139 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a3b65ef9024a64b336b00ad51e294b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.301191 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 88a84a8ea71c4258b378603a0827a768 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:39.312283 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a84a8ea71c4258b378603a0827a768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.356490 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 19357e2a1cf1408b9391be0c12c3b0e8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:39.357846 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19357e2a1cf1408b9391be0c12c3b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.355006 32162 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:39.371526 32164 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:39.387691 32105 tablet_service.cc:1511] Processing CreateTablet for tablet 775fd46a41044b1ea06bc796ef1b233b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:39.401424 32163 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:39.402896 32162 tablet_bootstrap.cc:654] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.406783 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775fd46a41044b1ea06bc796ef1b233b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.414826 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 4633b71e43824d84895ff136e5765fb1 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:39.421172 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633b71e43824d84895ff136e5765fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.424937 31958 tablet_service.cc:1511] Processing CreateTablet for tablet 775fd46a41044b1ea06bc796ef1b233b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:39.427361 32034 tablet_service.cc:1511] Processing CreateTablet for tablet 775fd46a41044b1ea06bc796ef1b233b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:39.428642 32034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775fd46a41044b1ea06bc796ef1b233b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.408283 31961 tablet_service.cc:1511] Processing CreateTablet for tablet 19357e2a1cf1408b9391be0c12c3b0e8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:39.432565 32164 tablet_bootstrap.cc:654] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.434913 32033 tablet_service.cc:1511] Processing CreateTablet for tablet 19357e2a1cf1408b9391be0c12c3b0e8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:39.434995 31962 tablet_service.cc:1511] Processing CreateTablet for tablet c6c31e616d2c4874ac5a8e29453cc51b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:39.436275 32031 tablet_service.cc:1511] Processing CreateTablet for tablet c6c31e616d2c4874ac5a8e29453cc51b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:39.437178 31958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775fd46a41044b1ea06bc796ef1b233b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.436959 32104 tablet_service.cc:1511] Processing CreateTablet for tablet c6c31e616d2c4874ac5a8e29453cc51b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:39.443050 32163 tablet_bootstrap.cc:654] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.444518 32162 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:39.445011 32162 ts_tablet_manager.cc:1403] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.090s	user 0.008s	sys 0.006s
I20260709 12:16:39.446287 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6c31e616d2c4874ac5a8e29453cc51b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.406949 31959 tablet_service.cc:1511] Processing CreateTablet for tablet 4633b71e43824d84895ff136e5765fb1 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:39.465411 32031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6c31e616d2c4874ac5a8e29453cc51b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.474283 32029 tablet_service.cc:1511] Processing CreateTablet for tablet f953fa305a42492da9de0fa8c18c43eb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:39.476086 32029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f953fa305a42492da9de0fa8c18c43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.475909 32162 raft_consensus.cc:359] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:39.485584 32162 raft_consensus.cc:385] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.485951 32162 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.486953 32162 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:39.490419 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6c31e616d2c4874ac5a8e29453cc51b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.490759 32033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19357e2a1cf1408b9391be0c12c3b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.502580 31956 tablet_service.cc:1511] Processing CreateTablet for tablet f953fa305a42492da9de0fa8c18c43eb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:39.503934 31956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f953fa305a42492da9de0fa8c18c43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.509629 31960 tablet_service.cc:1511] Processing CreateTablet for tablet e828882e6b1940ffb33d4c2f76b47fe9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:39.510996 32035 tablet_service.cc:1511] Processing CreateTablet for tablet e828882e6b1940ffb33d4c2f76b47fe9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:39.512300 32035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e828882e6b1940ffb33d4c2f76b47fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.515331 32102 tablet_service.cc:1511] Processing CreateTablet for tablet f953fa305a42492da9de0fa8c18c43eb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:39.516592 32102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f953fa305a42492da9de0fa8c18c43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.519413 32109 tablet_service.cc:1511] Processing CreateTablet for tablet e828882e6b1940ffb33d4c2f76b47fe9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:39.520629 32109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e828882e6b1940ffb33d4c2f76b47fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.520684 32028 tablet_service.cc:1511] Processing CreateTablet for tablet a7f1c1160177412eb547493e1f3c561f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:39.524276 32106 tablet_service.cc:1511] Processing CreateTablet for tablet 0292f21aab4a4843b2b424c3497ddced (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:39.524708 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 0292f21aab4a4843b2b424c3497ddced (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:39.525501 32106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0292f21aab4a4843b2b424c3497ddced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.525871 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0292f21aab4a4843b2b424c3497ddced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.533304 31959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633b71e43824d84895ff136e5765fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.547199 32146 heartbeater.cc:499] Master 127.30.25.62:33695 was elected leader, sending a full tablet report...
I20260709 12:16:39.548036 32162 ts_tablet_manager.cc:1434] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.103s	user 0.011s	sys 0.001s
I20260709 12:16:39.543587 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f1c1160177412eb547493e1f3c561f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.552811 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 0292f21aab4a4843b2b424c3497ddced (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:39.553316 31955 tablet_service.cc:1511] Processing CreateTablet for tablet a7f1c1160177412eb547493e1f3c561f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260709 12:16:39.556069 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:39.557574 32036 tablet_service.cc:1511] Processing CreateTablet for tablet 7c52a06827f24bb6bc1851474fc80e53 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:39.558764 31960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e828882e6b1940ffb33d4c2f76b47fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.559029 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c52a06827f24bb6bc1851474fc80e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.568004 32103 tablet_service.cc:1511] Processing CreateTablet for tablet a7f1c1160177412eb547493e1f3c561f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:39.569337 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f1c1160177412eb547493e1f3c561f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.574759 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0292f21aab4a4843b2b424c3497ddced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.576522 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19357e2a1cf1408b9391be0c12c3b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.576875 32107 tablet_service.cc:1511] Processing CreateTablet for tablet 7c52a06827f24bb6bc1851474fc80e53 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:39.578094 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c52a06827f24bb6bc1851474fc80e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.546742 32169 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.583431 32169 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:39.584430 32037 tablet_service.cc:1511] Processing CreateTablet for tablet 478b63c892a745d09e9508bc9cc5321f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:39.589725 32033 tablet_service.cc:1511] Processing CreateTablet for tablet e0bd59e23a6d4e1f9674d7662259db2f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:39.591997 32164 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:39.592521 32164 ts_tablet_manager.cc:1403] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.221s	user 0.037s	sys 0.007s
I20260709 12:16:39.593022 32169 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:39.591622 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 478b63c892a745d09e9508bc9cc5321f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:39.595110 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478b63c892a745d09e9508bc9cc5321f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.598798 31957 tablet_service.cc:1511] Processing CreateTablet for tablet 7c52a06827f24bb6bc1851474fc80e53 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:39.600204 31957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c52a06827f24bb6bc1851474fc80e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.604977 32163 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:39.604542 32164 raft_consensus.cc:359] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.605912 32163 ts_tablet_manager.cc:1403] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.205s	user 0.037s	sys 0.027s
I20260709 12:16:39.641390 32163 raft_consensus.cc:359] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.612706 32033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bd59e23a6d4e1f9674d7662259db2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.642819 32037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478b63c892a745d09e9508bc9cc5321f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.642128 32163 raft_consensus.cc:385] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.644793 32163 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.645820 32163 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.615923 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 6d02c13b00af439e946e76ef880668d9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:39.610733 32162 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:39.634618 31958 tablet_service.cc:1511] Processing CreateTablet for tablet e0bd59e23a6d4e1f9674d7662259db2f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:39.659705 32164 raft_consensus.cc:385] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.671599 31955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f1c1160177412eb547493e1f3c561f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.667426 32164 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
W20260709 12:16:39.668207 32147 tablet.cc:2406] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:39.677464 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02c13b00af439e946e76ef880668d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.684741 31954 tablet_service.cc:1511] Processing CreateTablet for tablet 6d02c13b00af439e946e76ef880668d9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:39.686183 31954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02c13b00af439e946e76ef880668d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.697862 31958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bd59e23a6d4e1f9674d7662259db2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.712239 32163 ts_tablet_manager.cc:1434] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.018s
I20260709 12:16:39.713250 32072 heartbeater.cc:499] Master 127.30.25.62:33695 was elected leader, sending a full tablet report...
I20260709 12:16:39.715710 32162 tablet_bootstrap.cc:654] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.721318 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 81129181ae3246c1bfb02954120c9b91 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:39.725360 32163 tablet_bootstrap.cc:492] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:39.727449 31956 tablet_service.cc:1511] Processing CreateTablet for tablet 81129181ae3246c1bfb02954120c9b91 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:39.728788 31956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81129181ae3246c1bfb02954120c9b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.708252 32029 tablet_service.cc:1511] Processing CreateTablet for tablet 3e4bfb2be45e4fc38e6622722ab10735 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:39.729162 31962 tablet_service.cc:1511] Processing CreateTablet for tablet 3e4bfb2be45e4fc38e6622722ab10735 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:39.732228 32163 tablet_bootstrap.cc:654] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.735443 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81129181ae3246c1bfb02954120c9b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.699004 32164 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.751765 31997 heartbeater.cc:499] Master 127.30.25.62:33695 was elected leader, sending a full tablet report...
I20260709 12:16:39.756022 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4bfb2be45e4fc38e6622722ab10735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.756809 32029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4bfb2be45e4fc38e6622722ab10735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.774712 32172 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.774821 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 02ef57fadd044f68ba212dc2fe410f83 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:39.775290 32172 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.775593 32109 tablet_service.cc:1511] Processing CreateTablet for tablet 478b63c892a745d09e9508bc9cc5321f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:39.776151 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02ef57fadd044f68ba212dc2fe410f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.776937 32109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478b63c892a745d09e9508bc9cc5321f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.779356 32104 tablet_service.cc:1511] Processing CreateTablet for tablet 3e4bfb2be45e4fc38e6622722ab10735 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:39.779963 32106 tablet_service.cc:1511] Processing CreateTablet for tablet e0bd59e23a6d4e1f9674d7662259db2f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:39.780591 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4bfb2be45e4fc38e6622722ab10735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.781169 32106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bd59e23a6d4e1f9674d7662259db2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.782548 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 81129181ae3246c1bfb02954120c9b91 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:39.789685 32103 tablet_service.cc:1511] Processing CreateTablet for tablet 6d02c13b00af439e946e76ef880668d9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:39.795514 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:39.790732 32172 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:39.804595 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:39.807461 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81129181ae3246c1bfb02954120c9b91. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:39.809360 32084 leader_election.cc:343] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.810228 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02c13b00af439e946e76ef880668d9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:39.810487 32084 leader_election.cc:343] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.811059 32084 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:39.812494 32169 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:39.835014 32102 tablet_service.cc:1511] Processing CreateTablet for tablet 02ef57fadd044f68ba212dc2fe410f83 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:39.844069 32037 tablet_service.cc:1511] Processing CreateTablet for tablet 02ef57fadd044f68ba212dc2fe410f83 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:39.845397 32037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02ef57fadd044f68ba212dc2fe410f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.846097 32179 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.846594 32179 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.847034 31955 tablet_service.cc:1511] Processing CreateTablet for tablet c50b710409be4b8986d1a0b839db7004 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:39.840281 32163 tablet_bootstrap.cc:492] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:39.850858 32163 ts_tablet_manager.cc:1403] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.126s	user 0.020s	sys 0.028s
I20260709 12:16:39.848831 31955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50b710409be4b8986d1a0b839db7004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.856148 32102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02ef57fadd044f68ba212dc2fe410f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.856640 32163 raft_consensus.cc:359] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.857439 32163 raft_consensus.cc:385] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.857834 32163 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.859347 32105 tablet_service.cc:1511] Processing CreateTablet for tablet 422b4d293d444b06bb00c3f5b66a6f25 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:39.859543 32110 tablet_service.cc:1511] Processing CreateTablet for tablet c50b710409be4b8986d1a0b839db7004 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:39.860702 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b4d293d444b06bb00c3f5b66a6f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.863507 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50b710409be4b8986d1a0b839db7004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.864945 32107 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3ed5d095c544f0abfe4e2c375b06cc (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:39.866161 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ed5d095c544f0abfe4e2c375b06cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.870690 32163 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.872515 32162 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:39.877347 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3ed5d095c544f0abfe4e2c375b06cc (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:39.878593 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ed5d095c544f0abfe4e2c375b06cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.878443 32036 tablet_service.cc:1511] Processing CreateTablet for tablet 422b4d293d444b06bb00c3f5b66a6f25 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:39.879969 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b4d293d444b06bb00c3f5b66a6f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.872442 32033 tablet_service.cc:1511] Processing CreateTablet for tablet c50b710409be4b8986d1a0b839db7004 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:39.882637 32162 ts_tablet_manager.cc:1403] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.272s	user 0.010s	sys 0.007s
I20260709 12:16:39.883265 32033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50b710409be4b8986d1a0b839db7004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.885737 32162 raft_consensus.cc:359] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.892346 32162 raft_consensus.cc:385] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.929211 31961 tablet_service.cc:1511] Processing CreateTablet for tablet 422b4d293d444b06bb00c3f5b66a6f25 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:39.929347 31957 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3ed5d095c544f0abfe4e2c375b06cc (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:39.930686 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b4d293d444b06bb00c3f5b66a6f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.896457 32179 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:39.935997 31957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ed5d095c544f0abfe4e2c375b06cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.939185 32164 ts_tablet_manager.cc:1434] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.346s	user 0.016s	sys 0.026s
I20260709 12:16:39.940135 32164 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:39.927672 32162 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.953620 32164 tablet_bootstrap.cc:654] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.968575 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:39.973064 32013 leader_election.cc:343] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.966768 32162 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:39.975185 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:39.976718 32162 ts_tablet_manager.cc:1434] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.001s
W20260709 12:16:39.977411 32010 leader_election.cc:343] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.977662 32162 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:39.977877 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:39.978888 32172 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:39.962800 31953 tablet_service.cc:1511] Processing CreateTablet for tablet ad86bf28242f4576af3db683944d53a3 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:39.981788 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad86bf28242f4576af3db683944d53a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:39.991135 32163 ts_tablet_manager.cc:1434] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.002s
I20260709 12:16:39.992259 32163 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:39.998456 32037 tablet_service.cc:1511] Processing CreateTablet for tablet ad86bf28242f4576af3db683944d53a3 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:39.999817 32037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad86bf28242f4576af3db683944d53a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.003464 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:40.004999 31935 leader_election.cc:343] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.017812 32032 tablet_service.cc:1511] Processing CreateTablet for tablet d3949314523941858ae6d6a4fd770fb8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:40.019145 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3949314523941858ae6d6a4fd770fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.018882 32110 tablet_service.cc:1511] Processing CreateTablet for tablet d3949314523941858ae6d6a4fd770fb8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:40.020828 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3949314523941858ae6d6a4fd770fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.026453 32163 tablet_bootstrap.cc:654] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.026999 31960 tablet_service.cc:1511] Processing CreateTablet for tablet 3695041c497f47468894b0baf52fcfd0 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:40.028257 31960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3695041c497f47468894b0baf52fcfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.031421 32035 tablet_service.cc:1511] Processing CreateTablet for tablet 3695041c497f47468894b0baf52fcfd0 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:40.032884 32035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3695041c497f47468894b0baf52fcfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.035136 31954 tablet_service.cc:1511] Processing CreateTablet for tablet 826cffce8bb540c8beb56cc00a702782 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:40.041085 31954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 826cffce8bb540c8beb56cc00a702782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.041317 31956 tablet_service.cc:1511] Processing CreateTablet for tablet d3949314523941858ae6d6a4fd770fb8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:40.042577 31956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3949314523941858ae6d6a4fd770fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.047313 32105 tablet_service.cc:1511] Processing CreateTablet for tablet ad86bf28242f4576af3db683944d53a3 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:40.051683 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad86bf28242f4576af3db683944d53a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.052310 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 826cffce8bb540c8beb56cc00a702782 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:40.053570 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 826cffce8bb540c8beb56cc00a702782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.061275 32102 tablet_service.cc:1511] Processing CreateTablet for tablet 3695041c497f47468894b0baf52fcfd0 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:40.062616 32102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3695041c497f47468894b0baf52fcfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.077675 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.078804 32164 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:40.079289 32164 ts_tablet_manager.cc:1403] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.139s	user 0.015s	sys 0.012s
W20260709 12:16:40.079430 31938 leader_election.cc:343] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.079887 31938 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:40.081131 32181 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:40.083946 32163 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:40.084460 32163 ts_tablet_manager.cc:1403] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.003s
I20260709 12:16:40.084654 31962 tablet_service.cc:1511] Processing CreateTablet for tablet 0d6c0fabd5a9486f95f088218e4ba65d (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:40.086014 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d6c0fabd5a9486f95f088218e4ba65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.097244 32106 tablet_service.cc:1511] Processing CreateTablet for tablet 826cffce8bb540c8beb56cc00a702782 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:40.103120 32106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 826cffce8bb540c8beb56cc00a702782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.102005 32162 tablet_bootstrap.cc:654] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.098598 32035 tablet_service.cc:1511] Processing CreateTablet for tablet 0d6c0fabd5a9486f95f088218e4ba65d (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:40.103233 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 0d6c0fabd5a9486f95f088218e4ba65d (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:40.104929 31957 tablet_service.cc:1511] Processing CreateTablet for tablet 51b3fd343e81436fa5bea82486abcc64 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:40.105182 32107 tablet_service.cc:1511] Processing CreateTablet for tablet 51b3fd343e81436fa5bea82486abcc64 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:40.106140 31957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b3fd343e81436fa5bea82486abcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.107985 32035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d6c0fabd5a9486f95f088218e4ba65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.108556 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b3fd343e81436fa5bea82486abcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.109368 32169 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.109833 32169 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.109179 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d6c0fabd5a9486f95f088218e4ba65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.113819 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 8f4287254496472398f69d3a10b48d77 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:40.115088 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4287254496472398f69d3a10b48d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.116106 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:40.118026 32084 leader_election.cc:343] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.104053 32164 raft_consensus.cc:359] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:40.119230 32169 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:40.119067 32036 tablet_service.cc:1511] Processing CreateTablet for tablet 8f4287254496472398f69d3a10b48d77 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:40.133419 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4287254496472398f69d3a10b48d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.133755 32033 tablet_service.cc:1511] Processing CreateTablet for tablet bfedf707307443deb5a0681825db83cb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:40.135066 32033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfedf707307443deb5a0681825db83cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.111483 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 51b3fd343e81436fa5bea82486abcc64 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:40.135319 32164 raft_consensus.cc:385] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.135710 32164 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.122641 32172 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.141101 32172 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.123675 32163 raft_consensus.cc:359] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.141323 32037 tablet_service.cc:1511] Processing CreateTablet for tablet 58ee45b18b714479af8291d9056afb9e (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:40.142042 32163 raft_consensus.cc:385] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.120543 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:40.142696 32037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ee45b18b714479af8291d9056afb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.143283 32172 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:16:40.143812 32084 leader_election.cc:343] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.144268 32084 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:40.145315 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b3fd343e81436fa5bea82486abcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.145252 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.145920 32163 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.141228 32164 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:40.147315 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:40.147487 32013 leader_election.cc:343] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.126271 31961 tablet_service.cc:1511] Processing CreateTablet for tablet bfedf707307443deb5a0681825db83cb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:40.145442 32169 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:40.148705 32010 leader_election.cc:343] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.149251 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfedf707307443deb5a0681825db83cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.149142 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:40.149893 32172 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:40.155745 31955 tablet_service.cc:1511] Processing CreateTablet for tablet 58ee45b18b714479af8291d9056afb9e (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:40.157797 32164 ts_tablet_manager.cc:1434] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.078s	user 0.004s	sys 0.003s
I20260709 12:16:40.154263 32105 tablet_service.cc:1511] Processing CreateTablet for tablet bfedf707307443deb5a0681825db83cb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:40.164170 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfedf707307443deb5a0681825db83cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.165377 31955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ee45b18b714479af8291d9056afb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.160369 32162 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:40.166644 32162 ts_tablet_manager.cc:1403] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.189s	user 0.010s	sys 0.005s
I20260709 12:16:40.168215 32164 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:40.169611 32162 raft_consensus.cc:359] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.162595 32163 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.172115 32163 ts_tablet_manager.cc:1434] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.002s
I20260709 12:16:40.173110 32163 tablet_bootstrap.cc:492] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:40.173391 32103 tablet_service.cc:1511] Processing CreateTablet for tablet 58ee45b18b714479af8291d9056afb9e (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:40.174676 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ee45b18b714479af8291d9056afb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.175325 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 8f4287254496472398f69d3a10b48d77 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:40.176512 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4287254496472398f69d3a10b48d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.178387 32162 raft_consensus.cc:385] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.178743 32162 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.180281 32172 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.180841 32172 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.195818 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:40.196480 31972 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:40.197716 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:40.198661 32171 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.198525 32162 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.199064 32171 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.199407 32171 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.200316 32164 tablet_bootstrap.cc:654] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.205384 32171 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.224696 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:40.225256 31972 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.225317 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 5b2dd1839ae848ae849e1ec74ce14112 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:40.226796 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2dd1839ae848ae849e1ec74ce14112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.229715 32163 tablet_bootstrap.cc:654] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.230798 32171 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.232148 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:40.233145 31972 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
W20260709 12:16:40.233532 32013 leader_election.cc:343] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.235328 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:40.236398 32171 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.234066 31961 tablet_service.cc:1511] Processing CreateTablet for tablet 5b2dd1839ae848ae849e1ec74ce14112 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:40.238090 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2dd1839ae848ae849e1ec74ce14112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.242578 32102 tablet_service.cc:1511] Processing CreateTablet for tablet 5b2dd1839ae848ae849e1ec74ce14112 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:40.206792 32162 ts_tablet_manager.cc:1434] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.002s
I20260709 12:16:40.209008 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.209995 32172 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.213351 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 15d2c5dc46254473902eca585b82958f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:40.248945 31962 tablet_service.cc:1511] Processing CreateTablet for tablet b88d8f995fcc4faea19a190f8dd669ab (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:40.250507 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88d8f995fcc4faea19a190f8dd669ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:40.251610 32013 leader_election.cc:343] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.216424 31956 tablet_service.cc:1511] Processing CreateTablet for tablet 6d27358a7bd3493a8877bd9ce8ae9055 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:40.219316 32029 tablet_service.cc:1511] Processing CreateTablet for tablet 6d27358a7bd3493a8877bd9ce8ae9055 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:40.216440 32107 tablet_service.cc:1511] Processing CreateTablet for tablet 15d2c5dc46254473902eca585b82958f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:40.264596 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d2c5dc46254473902eca585b82958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.220829 32104 tablet_service.cc:1511] Processing CreateTablet for tablet 6d27358a7bd3493a8877bd9ce8ae9055 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:40.266769 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d27358a7bd3493a8877bd9ce8ae9055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.250392 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d2c5dc46254473902eca585b82958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.272980 32029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d27358a7bd3493a8877bd9ce8ae9055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.280922 32030 tablet_service.cc:1511] Processing CreateTablet for tablet b88d8f995fcc4faea19a190f8dd669ab (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:40.282245 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88d8f995fcc4faea19a190f8dd669ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.284449 32164 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:40.214558 31957 tablet_service.cc:1511] Processing CreateTablet for tablet 15d2c5dc46254473902eca585b82958f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:40.284953 32164 ts_tablet_manager.cc:1403] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.010s
I20260709 12:16:40.288573 32164 raft_consensus.cc:359] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.289279 32164 raft_consensus.cc:385] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.289598 32164 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.291121 32162 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:40.292583 32110 tablet_service.cc:1511] Processing CreateTablet for tablet b88d8f995fcc4faea19a190f8dd669ab (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:40.293840 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88d8f995fcc4faea19a190f8dd669ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.295300 32102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2dd1839ae848ae849e1ec74ce14112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.300578 32163 tablet_bootstrap.cc:492] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:40.301044 32163 ts_tablet_manager.cc:1403] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.000s
I20260709 12:16:40.295127 31955 tablet_service.cc:1511] Processing CreateTablet for tablet 58d45de750cd4fd1a0cdd13ad4f731fb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:40.306007 32163 raft_consensus.cc:359] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.306728 32163 raft_consensus.cc:385] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.307083 31956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d27358a7bd3493a8877bd9ce8ae9055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.307751 31955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d45de750cd4fd1a0cdd13ad4f731fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.308749 31957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d2c5dc46254473902eca585b82958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.314570 32164 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.315349 32029 tablet_service.cc:1511] Processing CreateTablet for tablet 58d45de750cd4fd1a0cdd13ad4f731fb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:40.316637 32029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d45de750cd4fd1a0cdd13ad4f731fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.318424 32171 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:40.318948 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 1a06d9856d134874b3144fd2abdbfc71 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:40.319427 32171 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.320266 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a06d9856d134874b3144fd2abdbfc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.320037 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 1a06d9856d134874b3144fd2abdbfc71 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:40.322214 31960 tablet_service.cc:1511] Processing CreateTablet for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:40.323362 32037 tablet_service.cc:1511] Processing CreateTablet for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:40.325129 32106 tablet_service.cc:1511] Processing CreateTablet for tablet 1a06d9856d134874b3144fd2abdbfc71 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:40.325914 32169 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.326300 32106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a06d9856d134874b3144fd2abdbfc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.326381 32169 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.328558 32169 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:40.325253 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a06d9856d134874b3144fd2abdbfc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.330212 32037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.330297 32164 ts_tablet_manager.cc:1434] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 12:16:40.331825 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:40.333411 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:40.335347 32084 leader_election.cc:343] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:40.336031 32084 leader_election.cc:343] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.336408 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:40.337145 32169 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:40.337771 32164 tablet_bootstrap.cc:492] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:40.338001 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 58d45de750cd4fd1a0cdd13ad4f731fb (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:40.341508 31960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.346403 32105 tablet_service.cc:1511] Processing CreateTablet for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:40.342412 32162 tablet_bootstrap.cc:654] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.307021 32163 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.357218 32164 tablet_bootstrap.cc:654] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.357759 32163 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.339253 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d45de750cd4fd1a0cdd13ad4f731fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.362218 31954 tablet_service.cc:1511] Processing CreateTablet for tablet 2c59a3d3077c4b8db6380a834f6ce3c0 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:40.366235 31954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c59a3d3077c4b8db6380a834f6ce3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.367889 32107 tablet_service.cc:1511] Processing CreateTablet for tablet 2c59a3d3077c4b8db6380a834f6ce3c0 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:40.369642 31962 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca59f0a11104ab882fac419c003e8b7 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:40.364094 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 2c59a3d3077c4b8db6380a834f6ce3c0 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:40.370886 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca59f0a11104ab882fac419c003e8b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.370992 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c59a3d3077c4b8db6380a834f6ce3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.372301 32163 ts_tablet_manager.cc:1434] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.000s
I20260709 12:16:40.380201 32036 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca59f0a11104ab882fac419c003e8b7 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:40.396557 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca59f0a11104ab882fac419c003e8b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.397058 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 0e910ef3c0af41bf8d4fba5315d68eef (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:40.381122 31870 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.382862 32103 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca59f0a11104ab882fac419c003e8b7 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:40.390223 32181 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.402921 32181 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.403376 32035 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2cc7077f01430da5a31385d674e266 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:40.404628 32035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2cc7077f01430da5a31385d674e266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.405220 32181 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.406905 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.406828 31959 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2cc7077f01430da5a31385d674e266 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:40.407630 32121 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:40.408078 31959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2cc7077f01430da5a31385d674e266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.385836 31958 tablet_service.cc:1511] Processing CreateTablet for tablet 0e910ef3c0af41bf8d4fba5315d68eef (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:40.410391 31958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e910ef3c0af41bf8d4fba5315d68eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.378605 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.411453 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e910ef3c0af41bf8d4fba5315d68eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.412094 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca59f0a11104ab882fac419c003e8b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.412444 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:40.414142 31938 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:40.414960 32181 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:40.415130 31935 leader_election.cc:343] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.415426 32181 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.415910 32181 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.423086 32181 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.425202 32181 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.426031 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:40.428300 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:40.428841 32121 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.430678 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c59a3d3077c4b8db6380a834f6ce3c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:40.434856 31935 leader_election.cc:343] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.388695 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 0e910ef3c0af41bf8d4fba5315d68eef (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:40.437139 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e910ef3c0af41bf8d4fba5315d68eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.374055 32163 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:40.442898 32121 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:40.444308 31938 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:40.446094 32181 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.448280 32164 tablet_bootstrap.cc:492] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:40.448772 32164 ts_tablet_manager.cc:1403] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.111s	user 0.008s	sys 0.007s
I20260709 12:16:40.450151 32162 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:40.450625 32162 ts_tablet_manager.cc:1403] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.000s
I20260709 12:16:40.453526 32162 raft_consensus.cc:359] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.454206 32162 raft_consensus.cc:385] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.454715 32162 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.455621 32162 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.457860 32162 ts_tablet_manager.cc:1434] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:16:40.459689 32181 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:40.460577 32181 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.463924 32162 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:40.465785 32164 raft_consensus.cc:359] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.466605 32164 raft_consensus.cc:385] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.466917 32164 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.472379 32104 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2cc7077f01430da5a31385d674e266 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:40.475953 32164 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.476460 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 16a6933cca8f4bce92ae16ff487ca7c1 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:40.477742 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16a6933cca8f4bce92ae16ff487ca7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.481222 32103 tablet_service.cc:1511] Processing CreateTablet for tablet 16a6933cca8f4bce92ae16ff487ca7c1 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:40.472379 31954 tablet_service.cc:1511] Processing CreateTablet for tablet 16a6933cca8f4bce92ae16ff487ca7c1 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:40.483718 31954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16a6933cca8f4bce92ae16ff487ca7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.486934 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2cc7077f01430da5a31385d674e266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.488772 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16a6933cca8f4bce92ae16ff487ca7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.493773 32164 ts_tablet_manager.cc:1434] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.005s
I20260709 12:16:40.494203 31958 tablet_service.cc:1511] Processing CreateTablet for tablet 1612f8e575bd46cf8069fa1c21ca192b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:40.500631 31958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1612f8e575bd46cf8069fa1c21ca192b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.500525 32036 tablet_service.cc:1511] Processing CreateTablet for tablet 1612f8e575bd46cf8069fa1c21ca192b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:40.501895 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1612f8e575bd46cf8069fa1c21ca192b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.503222 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 1612f8e575bd46cf8069fa1c21ca192b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:40.504457 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1612f8e575bd46cf8069fa1c21ca192b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.511147 32164 tablet_bootstrap.cc:492] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:40.514022 32181 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.514578 32181 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:40.498064 32163 tablet_bootstrap.cc:654] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.516738 32181 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.518040 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:40.518553 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:40.521580 31935 leader_election.cc:343] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.519634 32121 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:40.523674 31938 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:40.524789 32181 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.525166 32181 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.525339 32164 tablet_bootstrap.cc:654] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.525504 32181 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.470297 32162 tablet_bootstrap.cc:654] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.529654 31962 tablet_service.cc:1511] Processing CreateTablet for tablet d1ae9fcf8eb748c6b0cb9703adc9f338 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:40.530984 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ae9fcf8eb748c6b0cb9703adc9f338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.532987 32181 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:40.535785 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:16:40.537138 31935 leader_election.cc:343] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.550699 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:40.551550 32121 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.553138 32104 tablet_service.cc:1511] Processing CreateTablet for tablet d1ae9fcf8eb748c6b0cb9703adc9f338 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:40.554811 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ae9fcf8eb748c6b0cb9703adc9f338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.556263 32036 tablet_service.cc:1511] Processing CreateTablet for tablet d1ae9fcf8eb748c6b0cb9703adc9f338 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:40.557533 32036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ae9fcf8eb748c6b0cb9703adc9f338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.559722 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 78b16558212549a98cd68f4e744eb410 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:40.562328 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b16558212549a98cd68f4e744eb410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.564260 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 78b16558212549a98cd68f4e744eb410 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:40.564790 31957 tablet_service.cc:1511] Processing CreateTablet for tablet 74fd2ead38d34aac9ef68364e1f4a2f8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:40.565560 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b16558212549a98cd68f4e744eb410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.566226 31957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fd2ead38d34aac9ef68364e1f4a2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.539810 32181 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.587777 32164 tablet_bootstrap.cc:492] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:40.588290 32164 ts_tablet_manager.cc:1403] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.012s
I20260709 12:16:40.589226 32107 tablet_service.cc:1511] Processing CreateTablet for tablet 74fd2ead38d34aac9ef68364e1f4a2f8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:40.590782 32107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fd2ead38d34aac9ef68364e1f4a2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.591305 32164 raft_consensus.cc:359] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.591428 31959 tablet_service.cc:1511] Processing CreateTablet for tablet 78b16558212549a98cd68f4e744eb410 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:40.592782 31959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b16558212549a98cd68f4e744eb410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.593669 32164 raft_consensus.cc:385] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:40.593768 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:40.594007 32164 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.596033 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 74fd2ead38d34aac9ef68364e1f4a2f8 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:40.597358 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fd2ead38d34aac9ef68364e1f4a2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.600351 32162 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:40.601020 32162 ts_tablet_manager.cc:1403] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.137s	user 0.022s	sys 0.009s
I20260709 12:16:40.607815 32164 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.608106 31868 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.612747 31961 tablet_service.cc:1511] Processing CreateTablet for tablet d2f941a3c6cb4a188198a804d603db3b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:40.614022 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2f941a3c6cb4a188198a804d603db3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.614813 32163 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:40.615329 32163 ts_tablet_manager.cc:1403] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.242s	user 0.018s	sys 0.000s
I20260709 12:16:40.616299 32035 tablet_service.cc:1511] Processing CreateTablet for tablet d2f941a3c6cb4a188198a804d603db3b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:40.617542 32035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2f941a3c6cb4a188198a804d603db3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.621403 32103 tablet_service.cc:1511] Processing CreateTablet for tablet d2f941a3c6cb4a188198a804d603db3b (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:40.622690 32103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2f941a3c6cb4a188198a804d603db3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.628116 32121 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:40.629351 31938 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:40.632437 31953 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff18ae9165241a3937358efc94d1309 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:40.633387 32181 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.633726 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff18ae9165241a3937358efc94d1309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.634177 32181 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:40.635069 32181 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:40.639725 32164 ts_tablet_manager.cc:1434] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.005s
I20260709 12:16:40.640720 32164 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:40.641362 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff18ae9165241a3937358efc94d1309 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:40.642601 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff18ae9165241a3937358efc94d1309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.614447 32162 raft_consensus.cc:359] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.649364 32162 raft_consensus.cc:385] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.649739 32162 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.669521 32169 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.674108 32030 tablet_service.cc:1511] Processing CreateTablet for tablet f3f78e3424864d319ee8da87f4848fe9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:40.675593 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f78e3424864d319ee8da87f4848fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.670257 32169 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.677659 32164 tablet_bootstrap.cc:654] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.679878 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:40.680474 32105 tablet_service.cc:1511] Processing CreateTablet for tablet f3f78e3424864d319ee8da87f4848fe9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:40.681053 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:40.681716 32105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f78e3424864d319ee8da87f4848fe9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:40.682188 32084 leader_election.cc:343] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.618220 32163 raft_consensus.cc:359] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.688237 32163 raft_consensus.cc:385] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.688686 32163 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.689210 31958 tablet_service.cc:1511] Processing CreateTablet for tablet 08b3d325c1944e45a59e666e91d0dbb6 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:40.691231 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff18ae9165241a3937358efc94d1309 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:40.692319 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff18ae9165241a3937358efc94d1309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.695190 31953 tablet_service.cc:1511] Processing CreateTablet for tablet f3f78e3424864d319ee8da87f4848fe9 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:40.699324 32032 tablet_service.cc:1511] Processing CreateTablet for tablet 08b3d325c1944e45a59e666e91d0dbb6 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:40.700695 32032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b3d325c1944e45a59e666e91d0dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.678109 32169 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:40.689627 32163 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.704098 32047 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:40.705916 32163 ts_tablet_manager.cc:1434] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.000s
I20260709 12:16:40.707031 32163 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:40.707617 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:40.708922 32169 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.709308 32169 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.709649 32169 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.710842 31958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b3d325c1944e45a59e666e91d0dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.714748 32162 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.716977 31953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f78e3424864d319ee8da87f4848fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.718348 32111 tablet_service.cc:1511] Processing CreateTablet for tablet 08b3d325c1944e45a59e666e91d0dbb6 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:40.719648 32111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b3d325c1944e45a59e666e91d0dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.744926 32169 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.747149 32030 tablet_service.cc:1511] Processing CreateTablet for tablet 6679c928e92e4f799b88357707650b60 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:40.748730 32030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6679c928e92e4f799b88357707650b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.756991 32110 tablet_service.cc:1511] Processing CreateTablet for tablet 6679c928e92e4f799b88357707650b60 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:40.758435 32110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6679c928e92e4f799b88357707650b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.763407 32188 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:40.746668 31961 tablet_service.cc:1511] Processing CreateTablet for tablet 6679c928e92e4f799b88357707650b60 (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:40.767081 31961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6679c928e92e4f799b88357707650b60. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:40.771413 32013 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:40.773376 31962 tablet_service.cc:1511] Processing CreateTablet for tablet 1da57956732d44d1b912fb270de7d63f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:40.757699 32163 tablet_bootstrap.cc:654] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.774402 32028 tablet_service.cc:1511] Processing CreateTablet for tablet 1da57956732d44d1b912fb270de7d63f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:40.774859 31962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1da57956732d44d1b912fb270de7d63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.775648 32028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1da57956732d44d1b912fb270de7d63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.779909 32169 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:40.780989 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:40.781105 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:40.781687 32047 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:40.782599 32084 leader_election.cc:343] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.771522 32162 ts_tablet_manager.cc:1434] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.170s	user 0.016s	sys 0.001s
I20260709 12:16:40.799561 32171 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.802999 32104 tablet_service.cc:1511] Processing CreateTablet for tablet 1da57956732d44d1b912fb270de7d63f (DEFAULT_TABLE table=test-workload [id=39fe72954527408bbd18e5daabbcf01a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:40.806644 32171 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.810190 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:40.810842 31971 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:40.811337 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.812024 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:40.812856 32188 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:40.813037 32013 leader_election.cc:343] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.813236 32188 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.813694 32188 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.814420 32171 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.819743 32188 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.823071 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:40.823794 31971 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.824373 32047 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:40.826025 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:40.827055 32169 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.827520 32169 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:40.828357 32104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1da57956732d44d1b912fb270de7d63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:40.828311 32169 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.843187 32162 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:40.844070 31971 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:40.845963 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:40.853886 32171 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.857640 32188 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.858332 32171 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:40.859200 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:16:40.860914 32013 leader_election.cc:343] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:40.863273 31935 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:40.863380 32162 tablet_bootstrap.cc:654] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.859230 32171 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.879493 32163 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:40.880026 32163 ts_tablet_manager.cc:1403] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.173s	user 0.012s	sys 0.031s
I20260709 12:16:40.883435 32181 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.883931 32181 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.884280 32163 raft_consensus.cc:359] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.885076 32163 raft_consensus.cc:385] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.885462 32163 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.886178 32181 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.878134 31869 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:40.887732 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:40.888370 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.886297 32163 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:40.890378 31938 leader_election.cc:343] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.890884 32163 ts_tablet_manager.cc:1434] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:16:40.888345 32047 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:40.891968 32163 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:40.893168 32164 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:40.893618 32164 ts_tablet_manager.cc:1403] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.253s	user 0.025s	sys 0.027s
I20260709 12:16:40.895761 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:40.896569 32164 raft_consensus.cc:359] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.897342 32164 raft_consensus.cc:385] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.897655 32164 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.898715 32163 tablet_bootstrap.cc:654] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.898443 32164 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.900915 32194 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.901297 32194 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.901664 32194 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.902086 32164 ts_tablet_manager.cc:1434] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:40.904233 32164 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:40.903838 31868 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:40.911375 32164 tablet_bootstrap.cc:654] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.916952 32162 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:40.917222 32163 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:40.914196 32194 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.917452 32162 ts_tablet_manager.cc:1403] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.001s
I20260709 12:16:40.917680 32163 ts_tablet_manager.cc:1403] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.007s
I20260709 12:16:40.919472 32194 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.920043 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:40.920549 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:40.921084 32047 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.920848 32163 raft_consensus.cc:359] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.921701 32163 raft_consensus.cc:385] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:40.921927 31938 leader_election.cc:343] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.922039 32163 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.922883 32163 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.923866 32164 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:40.923745 32162 raft_consensus.cc:359] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.924355 32164 ts_tablet_manager.cc:1403] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 12:16:40.924506 32162 raft_consensus.cc:385] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.924844 32162 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.925132 32163 ts_tablet_manager.cc:1434] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:16:40.926127 32163 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:40.925671 32162 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.927533 32188 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.928088 32188 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.929399 32169 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.930262 32188 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:40.929893 32169 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.932140 32169 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:40.932978 32163 tablet_bootstrap.cc:654] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.933702 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:40.934468 32046 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:40.936316 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:40.937115 31971 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:40.938295 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:40.940146 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:40.940979 32121 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:40.941262 32194 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:40.942724 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:40.944157 32047 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:40.944876 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:40.948076 31971 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:40.945645 32188 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.948809 32162 ts_tablet_manager.cc:1434] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:16:40.948951 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:40.949172 32188 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.949714 32188 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.927228 32164 raft_consensus.cc:359] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.952349 32164 raft_consensus.cc:385] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.952728 32164 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.953657 32164 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.960465 32164 ts_tablet_manager.cc:1434] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:16:40.961609 32164 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:40.965852 31869 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.969257 32164 tablet_bootstrap.cc:654] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.975277 32163 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:40.975688 32163 ts_tablet_manager.cc:1403] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.050s	user 0.027s	sys 0.012s
I20260709 12:16:40.949929 32162 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:40.978183 32163 raft_consensus.cc:359] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.978853 32163 raft_consensus.cc:385] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.979133 32163 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.979780 32163 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.981866 32163 ts_tablet_manager.cc:1434] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:16:40.950506 32194 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.946171 32169 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.982750 32163 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:40.983249 32169 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.983654 32169 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.991837 32194 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:40.993343 32194 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:40.994207 32163 tablet_bootstrap.cc:654] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.004051 32169 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.005867 32169 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.007365 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.008183 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.008695 31972 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.011140 32171 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.011651 32171 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.013835 32171 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.014457 32188 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.015501 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.015944 32047 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:41.016093 32120 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.015309 31972 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:41.019797 32188 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.020749 32164 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.021219 32164 ts_tablet_manager.cc:1403] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.012s
I20260709 12:16:41.021857 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.022498 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.022876 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.025308 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.025650 32162 tablet_bootstrap.cc:654] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.026067 32121 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:41.031606 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:41.030685 32164 raft_consensus.cc:359] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.032292 32164 raft_consensus.cc:385] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.032621 32164 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.032833 32010 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.033671 32188 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.033599 32164 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.034181 32188 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.034942 32188 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.035811 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.036546 32171 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.037317 32164 ts_tablet_manager.cc:1434] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:16:41.034013 32169 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.038532 32169 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:41.039376 32164 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.039376 32169 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.053373 32163 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.053858 32163 ts_tablet_manager.cc:1403] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.071s	user 0.020s	sys 0.037s
I20260709 12:16:41.055625 32162 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.056305 32162 ts_tablet_manager.cc:1403] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.107s	user 0.007s	sys 0.014s
I20260709 12:16:41.056651 32163 raft_consensus.cc:359] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.057318 32163 raft_consensus.cc:385] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.057592 32163 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.058276 32163 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.059800 32162 raft_consensus.cc:359] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.060534 32163 ts_tablet_manager.cc:1434] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:16:41.060523 32162 raft_consensus.cc:385] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.061113 32162 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.061872 32163 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.062219 32162 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.063838 32164 tablet_bootstrap.cc:654] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.065130 32162 ts_tablet_manager.cc:1434] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:16:41.066399 32162 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.068005 32163 tablet_bootstrap.cc:654] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.074199 32162 tablet_bootstrap.cc:654] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.079034 32163 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.079499 32163 ts_tablet_manager.cc:1403] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:16:41.081276 32164 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.081864 32164 ts_tablet_manager.cc:1403] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.002s
I20260709 12:16:41.082050 32163 raft_consensus.cc:359] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.082767 32163 raft_consensus.cc:385] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.083050 32163 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.083878 32163 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.085268 32164 raft_consensus.cc:359] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.086148 32164 raft_consensus.cc:385] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.086508 32164 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.087405 32164 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.089325 32163 ts_tablet_manager.cc:1434] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:41.090055 32164 ts_tablet_manager.cc:1434] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:16:41.090255 32163 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.093343 32162 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.093961 32162 ts_tablet_manager.cc:1403] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.028s	user 0.005s	sys 0.007s
I20260709 12:16:41.097386 32163 tablet_bootstrap.cc:654] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.097457 32162 raft_consensus.cc:359] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.098485 32162 raft_consensus.cc:385] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.098860 32162 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.099915 32162 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.056080 32188 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.101392 32164 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.104177 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.104290 32188 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.104343 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.104810 31972 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.105042 32121 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.107671 32162 ts_tablet_manager.cc:1434] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:16:41.108866 32162 tablet_bootstrap.cc:492] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.112016 31972 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.113258 32010 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.114118 32171 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.114609 32171 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:41.115445 32171 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.116045 32121 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.087345 31870 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.119588 32164 tablet_bootstrap.cc:654] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.122975 32162 tablet_bootstrap.cc:654] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.127935 32162 tablet_bootstrap.cc:492] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.128346 32162 ts_tablet_manager.cc:1403] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 12:16:41.130309 32162 raft_consensus.cc:359] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.131016 32162 raft_consensus.cc:385] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.131268 32162 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.131927 32162 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.132650 32163 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.133100 32163 ts_tablet_manager.cc:1403] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.043s	user 0.006s	sys 0.010s
I20260709 12:16:41.132324 31868 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:41.133906 32162 ts_tablet_manager.cc:1434] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 12:16:41.134928 32162 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.135856 32163 raft_consensus.cc:359] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:41.136544 32163 raft_consensus.cc:385] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.136816 32163 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.137796 32164 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.138260 32164 ts_tablet_manager.cc:1403] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.002s
I20260709 12:16:41.137581 31869 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.137560 32163 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } }
I20260709 12:16:41.140988 32164 raft_consensus.cc:359] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.141703 32164 raft_consensus.cc:385] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.141975 32164 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.142794 32164 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.143592 32163 ts_tablet_manager.cc:1434] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:16:41.144528 32163 tablet_bootstrap.cc:492] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.144748 32164 ts_tablet_manager.cc:1434] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:41.144811 32162 tablet_bootstrap.cc:654] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.145895 32164 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.151011 32163 tablet_bootstrap.cc:654] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.152904 32047 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.155580 32164 tablet_bootstrap.cc:654] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.159282 32169 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.159967 32169 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.162654 32169 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.163448 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.164373 31972 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:41.166801 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.167647 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:41.169755 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:41.171541 32169 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.172017 32169 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.172438 32169 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.182511 32169 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.186299 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.187136 31972 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.197116 31972 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:41.198726 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.199965 32203 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.212608 32203 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:41.213905 32203 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.218155 32194 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.221521 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.222196 32046 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.232990 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:41.233649 32194 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.234684 32169 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.244267 31870 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:41.253675 32164 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.254287 32164 ts_tablet_manager.cc:1403] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.109s	user 0.019s	sys 0.001s
I20260709 12:16:41.257112 32164 raft_consensus.cc:359] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.257786 32164 raft_consensus.cc:385] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.258044 32164 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.260921 32164 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.264015 32164 ts_tablet_manager.cc:1434] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:16:41.265328 32164 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.273460 32164 tablet_bootstrap.cc:654] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.290575 32162 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.291064 32162 ts_tablet_manager.cc:1403] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.156s	user 0.033s	sys 0.095s
I20260709 12:16:41.293187 32164 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.290580 32218 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.293732 32162 raft_consensus.cc:359] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.294548 32162 raft_consensus.cc:385] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.294971 32162 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.294580 32218 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.295971 32163 tablet_bootstrap.cc:492] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.295646 32162 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.296438 32163 ts_tablet_manager.cc:1403] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.152s	user 0.052s	sys 0.078s
I20260709 12:16:41.296844 32218 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.297339 32203 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.297699 32162 ts_tablet_manager.cc:1434] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:41.297717 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.298604 32046 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:41.293722 32164 ts_tablet_manager.cc:1403] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 12:16:41.299170 32163 raft_consensus.cc:359] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.299813 32163 raft_consensus.cc:385] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.299803 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.300006 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.300272 32163 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.298736 32162 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.301056 32163 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.301667 32164 raft_consensus.cc:359] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.302325 32164 raft_consensus.cc:385] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.302599 32164 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.303308 32164 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.303789 32218 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.297796 32203 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.306465 32120 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:41.307286 32163 ts_tablet_manager.cc:1434] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:16:41.308212 32163 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.308435 32203 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.309374 32162 tablet_bootstrap.cc:654] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.310909 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.311802 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.312414 32046 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:41.313726 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:41.314968 32163 tablet_bootstrap.cc:654] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.315255 32169 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.315634 32169 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.305387 32164 ts_tablet_manager.cc:1434] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:41.305708 32218 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.317070 32218 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.317240 32164 tablet_bootstrap.cc:492] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.324115 32164 tablet_bootstrap.cc:654] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.329420 32220 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.330084 32220 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.333469 32220 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.333923 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.335602 32164 tablet_bootstrap.cc:492] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.336058 32164 ts_tablet_manager.cc:1403] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
W20260709 12:16:41.335997 31935 leader_election.cc:343] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.340013 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:41.341763 31938 leader_election.cc:343] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.342190 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:41.347024 32121 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.315972 32169 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.352646 32220 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.356554 32218 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.357789 31972 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:41.339215 32164 raft_consensus.cc:359] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.362898 32164 raft_consensus.cc:385] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.362618 32169 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.363233 32164 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.365608 32164 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.366679 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.367329 32046 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.368650 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.369432 31972 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:41.370710 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.371474 32120 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:41.372949 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.373196 32169 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.375916 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.376806 32169 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.366855 32218 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.377441 32169 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.377754 32046 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:41.379056 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:41.379853 32218 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.380329 32218 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:41.381001 32164 ts_tablet_manager.cc:1434] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:16:41.381122 32218 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.383844 32164 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.384593 32162 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.385038 32162 ts_tablet_manager.cc:1403] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.034s
I20260709 12:16:41.385881 32163 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.388682 32163 ts_tablet_manager.cc:1403] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.081s	user 0.035s	sys 0.025s
I20260709 12:16:41.390446 32164 tablet_bootstrap.cc:654] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.391386 32171 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.391459 32163 raft_consensus.cc:359] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.391903 32171 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.394920 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.395452 32171 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.396157 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:41.396457 32010 leader_election.cc:343] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:41.397321 32013 leader_election.cc:343] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.397742 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.390784 32162 raft_consensus.cc:359] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.399364 32162 raft_consensus.cc:385] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.399456 32171 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.399682 32162 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.400487 32162 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.392267 32163 raft_consensus.cc:385] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.401752 32163 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.399580 32164 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.402626 32164 ts_tablet_manager.cc:1403] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.003s
I20260709 12:16:41.402478 32163 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.404762 32220 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.405032 32171 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.405197 32220 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.405882 32162 ts_tablet_manager.cc:1434] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:16:41.406780 32162 tablet_bootstrap.cc:492] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.406987 31972 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.407284 32220 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.408712 32163 ts_tablet_manager.cc:1434] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:16:41.409648 32163 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.410861 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.411533 32047 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:41.412796 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.416350 32169 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.417793 32171 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:41.414090 31938 leader_election.cc:343] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.418280 32171 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.417335 31867 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.421316 32162 tablet_bootstrap.cc:654] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.421763 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.422381 32120 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.423295 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:41.416379 32163 tablet_bootstrap.cc:654] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.424865 32218 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.424798 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:41.426162 32188 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.426543 32188 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.426879 32188 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.427136 32047 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.429557 32121 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.434685 32164 raft_consensus.cc:359] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.435361 32164 raft_consensus.cc:385] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.435616 32164 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.435740 32162 tablet_bootstrap.cc:492] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.436169 32162 ts_tablet_manager.cc:1403] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.004s
I20260709 12:16:41.436367 32164 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.438712 32164 ts_tablet_manager.cc:1434] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.002s
I20260709 12:16:41.439558 32164 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.439284 32162 raft_consensus.cc:359] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.439917 32162 raft_consensus.cc:385] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.440183 32162 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
W20260709 12:16:41.440364 32013 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.440809 32162 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.441484 32203 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.443665 32188 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.446646 32163 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.455250 32163 ts_tablet_manager.cc:1403] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.002s
I20260709 12:16:41.455288 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.455782 32171 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.425355 32218 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.456367 32218 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:41.456914 32010 leader_election.cc:343] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.446892 32162 ts_tablet_manager.cc:1434] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:16:41.459064 32162 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.457943 32163 raft_consensus.cc:359] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.461967 32163 raft_consensus.cc:385] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.462386 32163 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.447141 32164 tablet_bootstrap.cc:654] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.463807 32163 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.470110 32162 tablet_bootstrap.cc:654] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.473978 32163 ts_tablet_manager.cc:1434] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 12:16:41.475528 32163 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.479900 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.480504 32120 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.483587 32171 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.471493 32188 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.484411 32171 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.488472 32171 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.493153 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.493798 31968 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.495157 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.485087 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.495302 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.497603 32188 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.498919 32188 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.499043 32119 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.499300 32188 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.498060 32169 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:16:41.508324 32010 leader_election.cc:343] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.509137 32203 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.509639 32203 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.511271 32188 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.513695 32163 tablet_bootstrap.cc:654] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.514163 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.515055 31972 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.516005 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.516809 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.517465 32046 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:41.518884 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:41.519114 32171 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.519691 32169 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:41.519743 32203 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.520195 32169 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.520606 32169 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:41.520751 32218 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.525730 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.527323 32220 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.527827 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.526331 32046 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.530714 32188 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.531234 32218 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.531898 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.533576 32118 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:41.532275 31938 leader_election.cc:343] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.544866 32120 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.546576 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.546979 32118 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.548056 32188 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.548676 32188 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:41.551215 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:41.552341 32171 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.552977 32171 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:41.554025 32171 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.549712 32188 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.582691 32216 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:16:41.583284 32216 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.589409 32218 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.595067 32220 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.595575 32220 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.597667 32220 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.598536 32236 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.599107 32236 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.601209 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:41.602780 31935 leader_election.cc:343] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.603660 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.603665 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.604272 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:41.605446 31938 leader_election.cc:343] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.605871 31938 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:41.597524 32046 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:41.607522 32220 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.607481 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:41.609014 32218 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.609103 32169 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.609560 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.609905 32218 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:41.604239 32236 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.610502 32119 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:16:41.610534 32253 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.610950 32218 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.612283 32162 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.612742 32162 ts_tablet_manager.cc:1403] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.154s	user 0.027s	sys 0.013s
I20260709 12:16:41.623572 32216 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.626873 32164 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.627530 32164 ts_tablet_manager.cc:1403] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.188s	user 0.010s	sys 0.013s
I20260709 12:16:41.602952 31972 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.641494 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.642849 32169 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.644629 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.647354 32163 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.647828 32163 ts_tablet_manager.cc:1403] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.173s	user 0.013s	sys 0.008s
I20260709 12:16:41.648876 32164 raft_consensus.cc:359] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.649555 32164 raft_consensus.cc:385] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.649827 32164 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.650820 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.650609 32164 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.652609 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.652887 32164 ts_tablet_manager.cc:1434] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 12:16:41.653416 32169 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:41.653885 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.655639 32220 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.656435 32220 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:41.656250 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.656790 32220 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:41.657595 32220 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.658489 32216 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:41.659050 32220 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:41.660683 32163 raft_consensus.cc:359] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.661569 32163 raft_consensus.cc:385] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.661909 32163 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.657696 32188 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:41.665788 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:41.667176 32163 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.672174 32162 raft_consensus.cc:359] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.672920 32162 raft_consensus.cc:385] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.673244 32162 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.673895 32162 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.675446 32162 ts_tablet_manager.cc:1434] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:16:41.676280 32164 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.676280 32162 tablet_bootstrap.cc:492] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.684532 32163 ts_tablet_manager.cc:1434] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:16:41.685456 32163 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.689968 31867 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.689693 31870 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.701015 32162 tablet_bootstrap.cc:654] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.704618 32163 tablet_bootstrap.cc:654] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.712435 32164 tablet_bootstrap.cc:654] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.726691 32163 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.726857 32164 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.727172 32163 ts_tablet_manager.cc:1403] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.002s
I20260709 12:16:41.727322 32164 ts_tablet_manager.cc:1403] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.005s
I20260709 12:16:41.726807 31869 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.730297 32164 raft_consensus.cc:359] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.730360 32163 raft_consensus.cc:359] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.731190 32164 raft_consensus.cc:385] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.731277 32163 raft_consensus.cc:385] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.731483 32164 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.731724 32163 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.732192 32164 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.732445 32163 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.730863 32162 tablet_bootstrap.cc:492] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.733902 32164 ts_tablet_manager.cc:1434] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:41.734640 32162 ts_tablet_manager.cc:1403] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.006s
I20260709 12:16:41.734901 32164 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.737249 32162 raft_consensus.cc:359] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.737960 32162 raft_consensus.cc:385] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.738245 32162 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.738961 32162 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.740176 32164 tablet_bootstrap.cc:654] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.740586 32163 ts_tablet_manager.cc:1434] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:16:41.741581 32163 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.747428 32163 tablet_bootstrap.cc:654] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.752981 32162 ts_tablet_manager.cc:1434] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:16:41.753933 32162 tablet_bootstrap.cc:492] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.756369 32164 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.756036 32169 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.757884 32164 ts_tablet_manager.cc:1403] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.000s
I20260709 12:16:41.759871 32162 tablet_bootstrap.cc:654] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.760936 32164 raft_consensus.cc:359] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.761648 32164 raft_consensus.cc:385] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.761957 32164 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.764004 32164 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.766496 32164 ts_tablet_manager.cc:1434] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:41.766597 32169 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.766891 32163 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.767350 32164 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.767028 32169 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.767558 32163 ts_tablet_manager.cc:1403] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 12:16:41.769238 32169 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.771535 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.771435 32163 raft_consensus.cc:359] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.772099 32163 raft_consensus.cc:385] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.772372 32163 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.772621 32162 tablet_bootstrap.cc:492] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.773021 32162 ts_tablet_manager.cc:1403] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:16:41.774518 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.775265 32047 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:41.775774 32162 raft_consensus.cc:359] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.776479 32162 raft_consensus.cc:385] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.776746 32162 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.776930 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:41.772994 32163 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.777994 32236 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.779008 32163 ts_tablet_manager.cc:1434] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:16:41.780335 32163 tablet_bootstrap.cc:492] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.777431 32162 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.783746 32203 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.784327 32162 ts_tablet_manager.cc:1434] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:16:41.786032 32162 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.793816 32236 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.796202 32164 tablet_bootstrap.cc:654] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.799052 32163 tablet_bootstrap.cc:654] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.799145 32218 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.799540 32162 tablet_bootstrap.cc:654] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.799775 32218 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.804072 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.804849 32046 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:41.808153 31935 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.810032 32220 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.810529 32220 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.810946 32220 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.811311 32162 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.811932 32162 ts_tablet_manager.cc:1403] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.008s
I20260709 12:16:41.815670 32162 raft_consensus.cc:359] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.816572 32162 raft_consensus.cc:385] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.816960 32162 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.818240 32162 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.821403 32162 ts_tablet_manager.cc:1434] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:16:41.822718 32162 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.823112 32218 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.824369 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:41.826004 31938 leader_election.cc:343] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.831403 32162 tablet_bootstrap.cc:654] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.836848 32163 tablet_bootstrap.cc:492] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.837469 32163 ts_tablet_manager.cc:1403] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.057s	user 0.016s	sys 0.018s
I20260709 12:16:41.841063 32163 raft_consensus.cc:359] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.841996 32163 raft_consensus.cc:385] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.842274 32163 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.842978 32163 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.845301 32163 ts_tablet_manager.cc:1434] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.001s
I20260709 12:16:41.845973 32162 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.846246 32163 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.846544 32162 ts_tablet_manager.cc:1403] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.005s
I20260709 12:16:41.851027 32220 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.849574 32162 raft_consensus.cc:359] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.852759 32120 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.852922 32162 raft_consensus.cc:385] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.853200 32162 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.853601 32220 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.855697 32188 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.853853 32162 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.856096 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.856217 32188 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.855470 32163 tablet_bootstrap.cc:654] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.857801 32162 ts_tablet_manager.cc:1434] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
W20260709 12:16:41.857719 31938 leader_election.cc:343] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.858741 32162 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.860140 32188 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.861649 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.862193 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.862248 32119 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.862936 31967 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.863677 32013 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:41.864516 32188 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.864916 32188 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.865258 32188 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.864423 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:41.866012 32046 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.877322 32163 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.879534 32163 ts_tablet_manager.cc:1403] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.012s
I20260709 12:16:41.881902 32046 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:41.882439 32162 tablet_bootstrap.cc:654] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.883401 32164 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.883878 32164 ts_tablet_manager.cc:1403] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.117s	user 0.023s	sys 0.030s
I20260709 12:16:41.883630 31935 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:41.885787 32220 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.886407 32220 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:41.887548 32220 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.888443 32164 raft_consensus.cc:359] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.889179 32164 raft_consensus.cc:385] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.889442 32164 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.890475 32164 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.892657 32164 ts_tablet_manager.cc:1434] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:16:41.885051 32188 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.896160 32163 raft_consensus.cc:359] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.896848 32163 raft_consensus.cc:385] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.897137 32163 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.899072 32188 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.893567 32164 tablet_bootstrap.cc:492] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.901005 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.901702 31967 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.900970 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.906941 32119 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.906641 32163 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.908597 32163 ts_tablet_manager.cc:1434] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 12:16:41.909554 32163 tablet_bootstrap.cc:492] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.913800 31869 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.915958 32163 tablet_bootstrap.cc:654] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.917294 32164 tablet_bootstrap.cc:654] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.918702 31967 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.920094 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.922367 32119 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:41.922796 32171 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.923544 32171 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:41.924793 32171 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.926482 32217 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.927210 32217 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.939720 32162 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.940232 32162 ts_tablet_manager.cc:1403] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.001s
I20260709 12:16:41.942554 32162 raft_consensus.cc:359] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.943235 32162 raft_consensus.cc:385] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.943068 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.943552 32162 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.943811 31967 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:41.945089 32220 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.945087 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.946251 32188 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.946489 32163 tablet_bootstrap.cc:492] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.946798 32188 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.947041 32163 ts_tablet_manager.cc:1403] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.015s
I20260709 12:16:41.947256 32188 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.947853 32162 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.949941 32163 raft_consensus.cc:359] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.950627 32163 raft_consensus.cc:385] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.949328 32164 tablet_bootstrap.cc:492] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:41.950901 32163 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.951437 32164 ts_tablet_manager.cc:1403] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.009s
I20260709 12:16:41.951829 32163 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.953986 32162 ts_tablet_manager.cc:1434] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:16:41.954252 32163 ts_tablet_manager.cc:1434] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:41.955264 32163 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.955876 32164 raft_consensus.cc:359] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.956606 32164 raft_consensus.cc:385] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.956926 32164 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.957010 32217 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:41.955268 32162 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:41.957719 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:41.963987 32162 tablet_bootstrap.cc:654] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:41.965889 32013 leader_election.cc:343] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.965935 32164 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.969689 32164 ts_tablet_manager.cc:1434] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 12:16:41.972131 32163 tablet_bootstrap.cc:654] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.972450 32220 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.976089 32164 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:41.978247 31868 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.980300 32236 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.981056 32236 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.983421 32236 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:41.984308 32163 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:41.985225 32163 ts_tablet_manager.cc:1403] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.000s
I20260709 12:16:41.985549 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:41.986234 31967 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:41.987519 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:41.987675 32188 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.987962 32163 raft_consensus.cc:359] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.988648 32163 raft_consensus.cc:385] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.988932 32163 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.989097 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:41.989737 32047 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:41.990705 32163 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:41.991855 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:41.992786 32163 ts_tablet_manager.cc:1434] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 12:16:41.993274 32013 leader_election.cc:343] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:41.993713 32163 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:41.994088 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:41.994731 31967 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.996207 32164 tablet_bootstrap.cc:654] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.997326 32236 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.997718 32236 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.998075 32236 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.998209 32162 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:41.998703 32162 ts_tablet_manager.cc:1403] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.006s
I20260709 12:16:42.002175 32162 raft_consensus.cc:359] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.002863 32162 raft_consensus.cc:385] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.003123 32162 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.003747 32162 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.006078 32162 ts_tablet_manager.cc:1434] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 12:16:42.007033 32162 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.013309 32162 tablet_bootstrap.cc:654] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.990134 32188 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.024190 31967 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:42.030969 32163 tablet_bootstrap.cc:654] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.035471 32181 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:42.032415 32188 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.042599 32188 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.047510 32188 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.047782 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:42.048038 32236 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:42.049238 32013 leader_election.cc:343] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.050837 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:42.051403 31967 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.052570 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:42.053184 32045 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.071369 31967 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:42.072854 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:42.073726 32203 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:42.074232 32203 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:42.075350 32203 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.076653 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:42.077826 32217 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:42.078357 32217 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:42.079181 32217 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.081041 32181 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:42.057775 32236 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.082587 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:42.092505 32010 leader_election.cc:343] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.093170 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.094815 32162 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.095448 32162 ts_tablet_manager.cc:1403] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.089s	user 0.013s	sys 0.032s
I20260709 12:16:42.099272 32263 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.099434 32169 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.099546 32162 raft_consensus.cc:359] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.099906 32169 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.100725 32171 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:42.102918 32162 raft_consensus.cc:385] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.103700 32162 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.106577 32169 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.107447 32181 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.108012 32181 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.108886 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.109726 32046 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:42.111498 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:42.112504 32169 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:42.112880 32169 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:42.113242 32169 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.105937 32162 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.115895 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:42.117293 31938 leader_election.cc:343] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.118141 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.118901 32046 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:42.121490 32188 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:42.112519 32181 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.124902 32045 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:42.127265 32181 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.127944 32181 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.130782 32181 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.131922 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:42.133987 32084 leader_election.cc:343] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.132323 32162 ts_tablet_manager.cc:1434] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:16:42.132042 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:42.135967 32162 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.130116 31972 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.145092 32118 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:42.164551 32164 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.164837 32188 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:42.143848 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:42.197705 32164 ts_tablet_manager.cc:1403] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.227s	user 0.036s	sys 0.070s
I20260709 12:16:42.190138 32203 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.202551 32181 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:42.215152 32163 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:42.215574 32163 ts_tablet_manager.cc:1403] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.222s	user 0.018s	sys 0.005s
I20260709 12:16:42.217381 32290 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.218650 32290 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.213716 32181 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:42.232782 32164 raft_consensus.cc:359] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.233773 32164 raft_consensus.cc:385] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.234126 32164 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.235047 32164 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.207486 32203 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:42.207683 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 69783.7us
I20260709 12:16:42.249538 32203 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.231518 32181 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.250748 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.252405 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.252888 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.253319 32044 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:42.253523 32046 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:42.251231 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.255406 32216 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.255581 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.255968 32216 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.256729 32203 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.226918 32290 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.261989 32163 raft_consensus.cc:359] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.264091 32163 raft_consensus.cc:385] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.264437 32163 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.258483 32164 ts_tablet_manager.cc:1434] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 12:16:42.268545 32164 tablet_bootstrap.cc:492] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:42.292788 32162 tablet_bootstrap.cc:654] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.266023 32163 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.258507 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:42.257947 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:42.307122 32216 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.316646 32163 ts_tablet_manager.cc:1434] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.101s	user 0.012s	sys 0.000s
I20260709 12:16:42.319308 32163 tablet_bootstrap.cc:492] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:42.307634 32253 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.309562 32120 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:42.325217 32220 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.328677 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:42.336365 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78529.3us
I20260709 12:16:42.348729 32216 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.349252 32216 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.334182 32253 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.334810 32169 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.352449 32121 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:42.372006 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.372925 32253 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.373250 32013 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:42.373051 31967 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:42.347914 31935 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:42.388329 32290 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:42.388718 32290 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:42.389061 32290 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.410913 32171 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:42.409864 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84086.3us
I20260709 12:16:42.424225 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.425040 32220 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:42.426837 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:42.427429 31967 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.432339 32203 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.434408 32290 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.435804 31968 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:42.435811 31967 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:42.437175 32290 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.437829 32169 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.440034 32163 tablet_bootstrap.cc:654] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.421854 32171 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:42.441912 32216 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.445909 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.446957 32171 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.458438 32162 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.457477 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:42.458902 32162 ts_tablet_manager.cc:1403] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.323s	user 0.019s	sys 0.019s
I20260709 12:16:42.459661 32120 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.459728 32304 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:42.462368 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:42.464846 32303 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:42.465065 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:42.471414 32164 tablet_bootstrap.cc:654] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.473006 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.446095 31867 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:42.478206 31869 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:42.479015 32181 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.480103 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:42.481210 32181 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.482425 32306 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.482856 32306 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:42.483135 32013 leader_election.cc:343] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.490120 32169 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:42.493140 32169 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:42.493969 32169 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.494663 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:42.494451 32162 raft_consensus.cc:359] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.495357 32162 raft_consensus.cc:385] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.495707 32162 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.497817 32162 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.506963 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:42.507586 32045 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:42.512868 31938 leader_election.cc:343] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.515139 32120 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:42.516680 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:42.516671 32306 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.517423 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:42.517520 32220 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:42.518383 32047 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.518425 32220 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:42.519416 32220 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.526957 32162 ts_tablet_manager.cc:1434] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 12:16:42.532207 32163 tablet_bootstrap.cc:492] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:42.532752 32163 ts_tablet_manager.cc:1403] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.214s	user 0.015s	sys 0.007s
I20260709 12:16:42.540426 32163 raft_consensus.cc:359] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:42.537214 32010 leader_election.cc:343] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.541929 32010 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.515931 32171 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.544092 32171 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.544473 32263 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.534579 32303 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:16:42.541469 32163 raft_consensus.cc:385] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.550272 32163 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.551611 32163 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.536962 32045 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:42.559147 32163 ts_tablet_manager.cc:1434] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.005s
I20260709 12:16:42.561435 32164 tablet_bootstrap.cc:492] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.561877 32164 ts_tablet_manager.cc:1403] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.294s	user 0.013s	sys 0.005s
I20260709 12:16:42.550613 32084 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.563297 32169 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.563169 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:42.564008 32120 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:42.559113 32046 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:42.557125 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:42.570263 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:42.570758 32236 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.570645 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.571401 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.571631 32047 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:42.574029 32164 raft_consensus.cc:359] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.577129 32164 raft_consensus.cc:385] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.577448 32164 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.578045 32164 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.578995 32162 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.578577 32236 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.582845 32236 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:42.581264 32263 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:42.587299 32164 ts_tablet_manager.cc:1434] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:16:42.587960 32164 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:42.581765 32181 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:42.589093 32181 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:42.593268 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.594527 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.598955 32163 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:42.603360 32169 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.593734 32181 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.605078 32010 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:16:42.605091 32084 leader_election.cc:343] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.614598 32162 tablet_bootstrap.cc:654] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.617663 32263 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.618680 32303 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:42.615227 32084 leader_election.cc:343] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.624397 32164 tablet_bootstrap.cc:654] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.642071 32171 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:42.670588 32304 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.682976 32304 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.741115 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:42.741776 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:42.744184 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:42.745090 32181 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:42.745472 32181 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:42.745814 32181 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.747449 32163 tablet_bootstrap.cc:654] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.748977 31867 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:42.750795 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:42.752409 32162 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.734534 32304 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:16:42.735275 32084 leader_election.cc:343] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.753206 32162 ts_tablet_manager.cc:1403] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.174s	user 0.014s	sys 0.009s
I20260709 12:16:42.752992 31867 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:42.756619 32164 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.757043 32164 ts_tablet_manager.cc:1403] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.169s	user 0.039s	sys 0.038s
I20260709 12:16:42.758518 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:42.760041 32162 raft_consensus.cc:359] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.760712 32162 raft_consensus.cc:385] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.760978 32162 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.763383 32181 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.764766 32164 raft_consensus.cc:359] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.765390 32164 raft_consensus.cc:385] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.765658 32164 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
W20260709 12:16:42.766001 31938 leader_election.cc:343] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.766435 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:42.766992 32045 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.766706 32162 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.768915 32162 ts_tablet_manager.cc:1434] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 12:16:42.769807 32162 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.771711 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:42.772533 32164 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:42.773615 31938 leader_election.cc:343] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.773397 32181 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.776356 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:42.776970 32164 ts_tablet_manager.cc:1434] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.003s
I20260709 12:16:42.780260 32164 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:42.781658 32162 tablet_bootstrap.cc:654] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.778103 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:42.757516 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.783886 32169 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.784248 32304 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:42.785502 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58780.6us
I20260709 12:16:42.785784 32304 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:16:42.786243 32084 leader_election.cc:343] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.786677 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.786651 32304 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.787736 32169 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.790041 32169 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:42.791424 31868 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:42.794351 32164 tablet_bootstrap.cc:654] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.794950 32163 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:42.795423 32163 ts_tablet_manager.cc:1403] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.197s	user 0.010s	sys 0.027s
I20260709 12:16:42.798446 32163 raft_consensus.cc:359] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.799299 32163 raft_consensus.cc:385] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.799628 32163 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.801502 32163 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.803045 32169 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:42.804111 32163 ts_tablet_manager.cc:1434] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:42.805418 32163 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:42.812786 32163 tablet_bootstrap.cc:654] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.822613 32162 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.823203 32162 ts_tablet_manager.cc:1403] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.003s
I20260709 12:16:42.826825 32162 raft_consensus.cc:359] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.827740 32162 raft_consensus.cc:385] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.828085 32162 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.828951 32162 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.838371 32162 ts_tablet_manager.cc:1434] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:16:42.839638 32162 tablet_bootstrap.cc:492] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.840008 31867 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:42.856050 32162 tablet_bootstrap.cc:654] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.864008 32162 tablet_bootstrap.cc:492] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.864598 32162 ts_tablet_manager.cc:1403] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.002s
I20260709 12:16:42.867177 32162 raft_consensus.cc:359] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.867833 32162 raft_consensus.cc:385] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.868089 32162 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.868726 32162 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.870958 32162 ts_tablet_manager.cc:1434] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:16:42.889786 32164 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.890246 32164 ts_tablet_manager.cc:1403] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.110s	user 0.053s	sys 0.041s
I20260709 12:16:42.890681 32163 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:42.891624 32163 ts_tablet_manager.cc:1403] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.041s
I20260709 12:16:42.893266 32162 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.894093 32163 raft_consensus.cc:359] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.894750 32163 raft_consensus.cc:385] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.895025 32163 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.895773 32163 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.897974 32163 ts_tablet_manager.cc:1434] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:42.893427 32164 raft_consensus.cc:359] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.898815 32163 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:42.898900 32164 raft_consensus.cc:385] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.899240 32164 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.899860 32164 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.901569 32164 ts_tablet_manager.cc:1434] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.001s
I20260709 12:16:42.902251 32164 tablet_bootstrap.cc:492] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:42.904623 32163 tablet_bootstrap.cc:654] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.906289 32162 tablet_bootstrap.cc:654] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.908011 32164 tablet_bootstrap.cc:654] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.925601 32164 tablet_bootstrap.cc:492] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.926084 32164 ts_tablet_manager.cc:1403] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.004s
I20260709 12:16:42.926360 32162 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.926805 32162 ts_tablet_manager.cc:1403] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.008s
I20260709 12:16:42.929541 32162 raft_consensus.cc:359] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.929481 32164 raft_consensus.cc:359] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.930421 32162 raft_consensus.cc:385] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.930536 32164 raft_consensus.cc:385] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.930730 32162 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.930887 32164 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.931488 32162 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.931929 32164 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.933480 32162 ts_tablet_manager.cc:1434] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:42.934331 32164 ts_tablet_manager.cc:1434] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:42.934674 32162 tablet_bootstrap.cc:492] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.935212 32164 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:42.937191 32163 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:42.937634 32163 ts_tablet_manager.cc:1403] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.012s
I20260709 12:16:42.940482 32163 raft_consensus.cc:359] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.941098 32163 raft_consensus.cc:385] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.941366 32164 tablet_bootstrap.cc:654] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.941385 32163 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.945425 32163 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.948175 32163 ts_tablet_manager.cc:1434] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:16:42.949334 32163 tablet_bootstrap.cc:492] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:42.952919 32162 tablet_bootstrap.cc:654] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.957046 32163 tablet_bootstrap.cc:654] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.962031 32162 tablet_bootstrap.cc:492] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:42.962520 32162 ts_tablet_manager.cc:1403] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.003s
I20260709 12:16:42.964993 32171 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.965759 32164 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.965499 32171 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.966359 32164 ts_tablet_manager.cc:1403] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.011s
I20260709 12:16:42.967562 32171 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:42.969847 32164 raft_consensus.cc:359] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.970808 32164 raft_consensus.cc:385] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.970928 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:42.972388 32164 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.973281 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:42.973083 32164 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.975626 32163 tablet_bootstrap.cc:492] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:42.975989 32164 ts_tablet_manager.cc:1434] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:42.976079 32163 ts_tablet_manager.cc:1403] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.006s
I20260709 12:16:42.977586 32164 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:42.978444 32163 raft_consensus.cc:359] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.979091 32163 raft_consensus.cc:385] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:42.979064 32010 leader_election.cc:343] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.979751 32163 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.980433 32163 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:42.980830 32013 leader_election.cc:343] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:42.981310 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:42.982504 32163 ts_tablet_manager.cc:1434] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:16:42.983433 32163 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:42.983774 32263 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:42.986177 32164 tablet_bootstrap.cc:654] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.989588 32163 tablet_bootstrap.cc:654] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.990533 32162 raft_consensus.cc:359] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.991223 32162 raft_consensus.cc:385] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.991863 32162 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.992540 32162 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:42.994493 32162 ts_tablet_manager.cc:1434] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:16:42.997056 32162 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:42.998180 32164 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:42.998620 32164 ts_tablet_manager.cc:1403] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.002s
I20260709 12:16:43.001159 32164 raft_consensus.cc:359] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.001778 32164 raft_consensus.cc:385] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.002022 32164 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.002626 32162 tablet_bootstrap.cc:654] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.005120 32163 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:43.005681 32163 ts_tablet_manager.cc:1403] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.007s
I20260709 12:16:43.008277 32163 raft_consensus.cc:359] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.008874 32163 raft_consensus.cc:385] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.009137 32163 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.009748 32163 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.012872 32163 ts_tablet_manager.cc:1434] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:16:43.013128 32162 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:43.013509 32162 ts_tablet_manager.cc:1403] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.002s
I20260709 12:16:43.013710 32163 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:43.015897 32162 raft_consensus.cc:359] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.016574 32162 raft_consensus.cc:385] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.016855 32162 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.017611 32162 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.019086 32163 tablet_bootstrap.cc:654] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.019488 32162 ts_tablet_manager.cc:1434] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:43.020428 32162 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:43.010733 32164 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.022902 32164 ts_tablet_manager.cc:1434] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:16:43.023093 32263 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.023691 32164 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:43.023532 32263 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.026041 32263 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.026484 32263 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.026849 32263 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.028771 32263 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.029939 32164 tablet_bootstrap.cc:654] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.029939 32162 tablet_bootstrap.cc:654] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.030925 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.032076 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.032704 31972 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
W20260709 12:16:43.034078 32010 leader_election.cc:343] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.034790 32010 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.034901 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.035477 32120 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:43.035763 32188 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.036121 32188 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.036441 32188 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.036654 32169 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.037091 32169 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.039069 32169 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.040894 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.041177 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.041694 32046 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
W20260709 12:16:43.042604 32084 leader_election.cc:343] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.043360 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:43.044610 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.045404 32188 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.046416 32169 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:43.055191 32013 leader_election.cc:343] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.060963 32169 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.061139 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:43.062063 32263 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.061406 32169 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.069777 32169 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.072636 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:43.072594 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:43.073553 32188 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.074585 32119 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.078233 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:43.081485 32304 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:43.081795 32084 leader_election.cc:343] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.082105 32304 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.083154 32181 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.074118 31972 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.084673 32304 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.088351 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.089030 32045 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:43.091703 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.093124 32169 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.093684 32169 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.096127 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.097172 32290 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.097383 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.097929 32045 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.098747 32290 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.099123 32290 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.099426 32290 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:43.102779 31938 leader_election.cc:343] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.111660 32307 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.112198 32307 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.115680 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.117259 32220 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.126626 32220 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.127218 32220 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.129509 32307 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.130859 32304 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.131366 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.132732 32220 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.131461 32304 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.134290 32188 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:16:43.143330 32188 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.144357 31972 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:43.144565 32304 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.145803 32188 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.146579 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.146919 32010 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.134459 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.147351 32043 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:43.148308 32188 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.148813 32188 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:43.149015 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.134232 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:43.141603 32121 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.149660 32290 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.142035 32307 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:43.152287 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.151371 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.141448 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:43.154584 31938 leader_election.cc:343] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.149660 32188 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:43.156294 31935 leader_election.cc:343] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.156785 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:43.157896 32304 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:16:43.158337 32304 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.158811 32304 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:43.149809 32118 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:16:43.160974 32220 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.166705 32119 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:43.169668 32164 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:43.170322 32164 ts_tablet_manager.cc:1403] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.147s	user 0.035s	sys 0.023s
I20260709 12:16:43.173524 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.174400 32164 raft_consensus.cc:359] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.175351 32164 raft_consensus.cc:385] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.175694 32164 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.176565 32164 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.174396 32043 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.175879 32290 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.181985 32236 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.184314 32162 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:43.184827 32162 ts_tablet_manager.cc:1403] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.165s	user 0.056s	sys 0.062s
I20260709 12:16:43.185459 32220 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.187096 32043 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:43.187688 32162 raft_consensus.cc:359] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.188150 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.188409 32162 raft_consensus.cc:385] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.188807 32162 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.188992 32164 ts_tablet_manager.cc:1434] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:16:43.189919 32164 tablet_bootstrap.cc:492] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:43.189004 32290 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.189702 32162 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.190872 32290 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:43.192051 32162 ts_tablet_manager.cc:1434] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:16:43.192034 32290 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.193249 32162 tablet_bootstrap.cc:492] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:43.193233 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:43.193956 32119 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.196372 32163 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:43.196863 32163 ts_tablet_manager.cc:1403] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.183s	user 0.022s	sys 0.051s
I20260709 12:16:43.199107 32045 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:43.199790 32162 tablet_bootstrap.cc:654] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.199455 32163 raft_consensus.cc:359] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.200196 32163 raft_consensus.cc:385] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.200309 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:43.200523 32163 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.201128 32236 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.201579 32164 tablet_bootstrap.cc:654] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.201465 32163 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.201596 32236 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:43.203289 32236 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.203975 32163 ts_tablet_manager.cc:1434] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:43.209319 32163 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:43.210754 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.216120 32163 tablet_bootstrap.cc:654] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.218717 32162 tablet_bootstrap.cc:492] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:43.219405 32162 ts_tablet_manager.cc:1403] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.003s
I20260709 12:16:43.224247 32162 raft_consensus.cc:359] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.226027 32162 raft_consensus.cc:385] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.226480 32162 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.189772 32304 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.233788 32162 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.260078 31972 raft_consensus.cc:2368] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:16:43.283769 32162 ts_tablet_manager.cc:1434] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.001s
I20260709 12:16:43.285925 32162 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:43.286373 32169 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.286849 32169 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.288717 32169 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.282358 32307 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:43.296725 32348 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:16:43.296999 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.299257 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.300211 32188 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.316936 32304 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.317018 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.317255 32188 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.310240 32045 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.320959 32188 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.321241 32163 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:43.321157 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.317039 31968 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.321768 32163 ts_tablet_manager.cc:1403] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.113s	user 0.019s	sys 0.014s
I20260709 12:16:43.322062 32169 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.309528 32307 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.317772 32044 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:16:43.322430 32169 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.323132 32169 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.305845 32119 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:43.336328 32164 tablet_bootstrap.cc:492] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:43.336824 32164 ts_tablet_manager.cc:1403] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.147s	user 0.025s	sys 0.047s
I20260709 12:16:43.337448 32169 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.343529 32169 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.332046 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.344602 32304 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.348145 32304 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.349138 32307 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.347352 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.347438 32120 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:43.352334 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.354130 32010 leader_election.cc:400] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:16:43.354959 32164 raft_consensus.cc:359] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.354543 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.358453 32164 raft_consensus.cc:385] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.342494 32290 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:43.423920 32164 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.357347 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:43.366606 32163 raft_consensus.cc:359] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.426599 32163 raft_consensus.cc:385] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.426955 32163 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.424785 32164 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.429229 31868 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:43.428995 32163 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.431818 32307 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.433774 32045 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:43.433517 32307 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.434966 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.435540 32307 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.358268 32010 leader_election.cc:403] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:43.436692 32263 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.437070 32263 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.437408 32263 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:43.437304 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83402us
I20260709 12:16:43.438838 32163 ts_tablet_manager.cc:1434] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.117s	user 0.025s	sys 0.015s
I20260709 12:16:43.414361 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:43.440302 32120 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:43.444711 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.445477 32047 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.446758 32120 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:16:43.448423 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.450546 32356 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:43.451153 32356 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:43.452229 32356 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.461167 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.461822 32043 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.463204 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.464023 32307 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.464395 32307 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.464744 32307 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.464818 32263 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.387725 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:43.469321 32163 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:43.472214 32164 ts_tablet_manager.cc:1434] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.135s	user 0.015s	sys 0.021s
I20260709 12:16:43.473934 32164 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:43.474068 31967 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:43.505551 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44985us
I20260709 12:16:43.511817 32263 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.517004 32355 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.522601 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:43.523413 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:43.517215 32304 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:16:43.511462 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54367.5us
I20260709 12:16:43.531558 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:43.538247 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.538578 32120 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.538888 32043 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:43.539283 32044 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:16:43.539808 32171 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:16:43.535848 32236 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.527721 31968 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.582015 32169 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.582675 32169 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.526337 32355 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.613059 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.613786 31970 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.644093 32169 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:16:43.592605 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58645.6us
I20260709 12:16:43.661167 32162 tablet_bootstrap.cc:654] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.610185 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.672040 32236 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.673877 32355 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:16:43.645396 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 125010us
I20260709 12:16:43.684504 32290 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:43.684123 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77967.6us
I20260709 12:16:43.688055 32290 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.692196 32290 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.693188 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.685693 32307 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.694846 32121 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:43.699095 32181 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.701057 32120 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:43.701311 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.702414 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.705060 32164 tablet_bootstrap.cc:654] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.708467 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:43.709143 32046 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.710048 31967 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:43.712464 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.713148 31972 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:43.714128 32181 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.715868 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.719127 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.719975 32171 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.720428 32171 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:43.721258 32171 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.726876 31869 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:43.702967 32307 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.740207 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.741418 32236 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.741461 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.741865 32236 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.742363 32236 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.744050 32047 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:43.756778 32307 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.770610 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:16:43.777287 32365 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:16:43.743121 32290 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.762243 32307 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.779700 32162 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:43.780179 32162 ts_tablet_manager.cc:1403] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.495s	user 0.029s	sys 0.031s
I20260709 12:16:43.779908 32307 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.783257 32163 tablet_bootstrap.cc:654] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.786616 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.789326 31972 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:43.792894 32047 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.800180 32047 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:43.804004 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:43.807076 31867 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:43.808741 31867 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:43.812572 32263 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.815968 32236 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.823849 32363 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:43.827924 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:43.829372 32355 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.829721 32290 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.829869 32355 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:16:43.830044 32084 leader_election.cc:343] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:43.831334 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:43.832302 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:43.833051 32118 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:43.833119 32164 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:43.833678 32164 ts_tablet_manager.cc:1403] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.360s	user 0.016s	sys 0.019s
I20260709 12:16:43.836234 32236 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.836632 32164 raft_consensus.cc:359] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.837256 32164 raft_consensus.cc:385] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.837512 32164 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.839619 32163 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:43.838132 32164 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.840062 32163 ts_tablet_manager.cc:1403] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.371s	user 0.002s	sys 0.018s
I20260709 12:16:43.840878 32355 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.832916 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:43.843492 32181 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:43.847123 32163 raft_consensus.cc:359] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.847635 32163 raft_consensus.cc:385] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.847805 32163 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.848222 32366 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.848779 32366 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:43.849562 32366 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.852591 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.853375 32045 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:43.854714 32164 ts_tablet_manager.cc:1434] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:16:43.855604 32164 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:43.856974 32163 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.851840 31968 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.860288 32162 raft_consensus.cc:359] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.860935 32162 raft_consensus.cc:385] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.861267 32162 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.860935 32163 ts_tablet_manager.cc:1434] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 12:16:43.880086 32163 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:43.881047 32164 tablet_bootstrap.cc:654] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.881816 32162 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.884418 32162 ts_tablet_manager.cc:1434] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.104s	user 0.004s	sys 0.004s
I20260709 12:16:43.886276 32162 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:43.884346 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:43.888597 31970 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.889565 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:43.898933 32290 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.889412 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.902837 32044 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.907011 32263 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.908202 32263 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.918977 31869 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:43.892267 32307 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.929916 32044 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:43.934195 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:43.898622 32355 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:16:43.931977 32263 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:43.950500 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:43.951005 32120 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:43.952283 32169 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.953171 32376 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:43.942353 32013 leader_election.cc:343] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.959261 32357 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:43.964205 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:43.964859 31972 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:43.966589 32307 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:43.964792 32047 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:43.968804 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:43.968818 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:43.954640 32376 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:43.977232 32047 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.979434 32162 tablet_bootstrap.cc:654] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.977475 32376 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:43.982700 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:43.983990 32171 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.986538 32171 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.986877 32171 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:43.990270 32084 leader_election.cc:343] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.991662 32220 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:43.993713 32164 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:43.994127 32164 ts_tablet_manager.cc:1403] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.139s	user 0.005s	sys 0.020s
I20260709 12:16:44.003899 32163 tablet_bootstrap.cc:654] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.009900 31970 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:44.021168 31868 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:44.024132 31868 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:44.036947 32171 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.040132 32164 raft_consensus.cc:359] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.040709 32164 raft_consensus.cc:385] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.041042 32164 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.041714 32164 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.044322 32164 ts_tablet_manager.cc:1434] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 12:16:44.045241 32164 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.052078 32171 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.052233 32047 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:44.052994 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:44.053913 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:16:44.054420 32013 leader_election.cc:343] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.054602 31968 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.064456 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.065735 32162 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.066244 32162 ts_tablet_manager.cc:1403] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.180s	user 0.004s	sys 0.010s
I20260709 12:16:44.072860 32164 tablet_bootstrap.cc:654] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.073416 32162 raft_consensus.cc:359] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.074090 32162 raft_consensus.cc:385] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.074368 32162 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.067539 32307 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.075282 32162 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.093628 32163 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.094132 32163 ts_tablet_manager.cc:1403] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.214s	user 0.019s	sys 0.016s
I20260709 12:16:44.094081 31968 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:44.095440 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:44.095819 32162 ts_tablet_manager.cc:1434] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 12:16:44.078259 32307 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:44.096557 32171 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.097193 32163 raft_consensus.cc:359] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.097875 32171 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:44.097358 32307 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.097877 32163 raft_consensus.cc:385] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.099280 32163 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.099903 32163 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.098939 32171 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.106858 32363 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.108659 32162 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.109783 32164 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.110242 32164 ts_tablet_manager.cc:1403] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.017s
I20260709 12:16:44.111621 32163 ts_tablet_manager.cc:1434] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 12:16:44.112687 32163 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.115051 32162 tablet_bootstrap.cc:654] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.124914 32162 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.125365 32162 ts_tablet_manager.cc:1403] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:16:44.125931 32164 raft_consensus.cc:359] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.126734 32164 raft_consensus.cc:385] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.127077 32164 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.127928 32164 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.128113 32162 raft_consensus.cc:359] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.128937 32162 raft_consensus.cc:385] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.129241 32162 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.130005 32162 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.131101 32164 ts_tablet_manager.cc:1434] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:16:44.132287 32162 ts_tablet_manager.cc:1434] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:44.132535 32164 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.133178 32162 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.133630 32163 tablet_bootstrap.cc:654] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.138934 32164 tablet_bootstrap.cc:654] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.142033 32290 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.142529 32290 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.143998 32362 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.144631 32290 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.147269 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.151279 32047 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:44.152315 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.152652 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.153149 32120 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:44.153746 32357 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.154136 32357 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.154507 32357 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.156258 32169 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.156826 32169 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.161033 31867 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:44.162456 32169 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.164096 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.166615 31867 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:44.170091 32084 leader_election.cc:343] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.171821 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.172477 32047 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:44.177594 32164 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.178073 32164 ts_tablet_manager.cc:1403] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.001s
I20260709 12:16:44.181818 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.184233 32169 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.184600 32169 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.184904 32169 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.186012 32357 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.194347 32162 tablet_bootstrap.cc:654] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.191753 31869 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.197327 32169 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.196633 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:44.199622 32357 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.199749 32047 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.199985 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:44.200220 32164 raft_consensus.cc:359] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.200842 32164 raft_consensus.cc:385] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.201107 32164 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.201743 32164 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.202261 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:44.202888 32120 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:44.203866 32084 leader_election.cc:343] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.204803 32164 ts_tablet_manager.cc:1434] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:16:44.205613 32163 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.205675 32164 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.206104 32163 ts_tablet_manager.cc:1403] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.094s	user 0.024s	sys 0.032s
I20260709 12:16:44.209309 32169 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.209123 32163 raft_consensus.cc:359] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.210518 32163 raft_consensus.cc:385] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.210764 32163 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.213163 32163 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.215763 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:44.216391 32045 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.216988 32163 ts_tablet_manager.cc:1434] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:44.217911 32163 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.230548 32164 tablet_bootstrap.cc:654] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.231400 32045 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:44.234566 32120 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:44.234881 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.236410 32307 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.236881 32307 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:44.237802 32307 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.236826 32047 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:44.242895 32163 tablet_bootstrap.cc:654] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.243373 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.247242 32357 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.247702 32357 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:44.248466 32357 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.252817 32169 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.260249 32164 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.260735 32164 ts_tablet_manager.cc:1403] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.008s
I20260709 12:16:44.260680 32169 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.262305 32163 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.263640 32164 raft_consensus.cc:359] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.264395 32164 raft_consensus.cc:385] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.264500 32163 ts_tablet_manager.cc:1403] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.000s
I20260709 12:16:44.264720 32164 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.265457 32164 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.267385 32164 ts_tablet_manager.cc:1434] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:16:44.267896 32163 raft_consensus.cc:359] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.269340 32163 raft_consensus.cc:385] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.269730 32164 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.269698 32163 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.270726 32163 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.273670 32163 ts_tablet_manager.cc:1434] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:44.274600 32163 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.281267 32164 tablet_bootstrap.cc:654] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.281401 32163 tablet_bootstrap.cc:654] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.304546 32164 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.305002 32164 ts_tablet_manager.cc:1403] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.000s
I20260709 12:16:44.310328 32164 raft_consensus.cc:359] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.311017 32164 raft_consensus.cc:385] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.311311 32164 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.311975 32164 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.313592 32164 ts_tablet_manager.cc:1434] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:16:44.314702 32164 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.322155 32376 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.322873 32376 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.326268 32376 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.329188 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.329802 32164 tablet_bootstrap.cc:654] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.331845 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:44.333773 32084 leader_election.cc:343] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:44.334551 32084 leader_election.cc:343] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.335031 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.335129 31869 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.336050 32169 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.343254 32357 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.344111 32357 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.347903 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.348132 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.349023 32047 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:44.350688 31935 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:44.350548 32357 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.352967 32290 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.361943 32163 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.362453 32163 ts_tablet_manager.cc:1403] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.088s	user 0.026s	sys 0.033s
I20260709 12:16:44.365506 31869 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:44.368510 32163 raft_consensus.cc:359] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.369189 32163 raft_consensus.cc:385] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.369436 32163 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.370054 32163 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.372063 32163 ts_tablet_manager.cc:1434] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:16:44.372911 32163 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.376286 32162 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.376729 32162 ts_tablet_manager.cc:1403] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.244s	user 0.033s	sys 0.025s
I20260709 12:16:44.377467 32164 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.377907 32164 ts_tablet_manager.cc:1403] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.000s
I20260709 12:16:44.381045 32162 raft_consensus.cc:359] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.381757 32162 raft_consensus.cc:385] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.382030 32162 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.382710 32162 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.384761 32162 ts_tablet_manager.cc:1434] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:44.385628 32162 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.386363 32164 raft_consensus.cc:359] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.389501 32164 raft_consensus.cc:385] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.389842 32164 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.390576 32164 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.391171 32162 tablet_bootstrap.cc:654] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.392649 32164 ts_tablet_manager.cc:1434] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:44.393568 32164 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.389195 32163 tablet_bootstrap.cc:654] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.398346 32356 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.398833 32356 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.400918 32356 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.401809 32366 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.402786 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.403501 32120 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:44.404009 32366 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.406442 32163 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.406268 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.406980 32163 ts_tablet_manager.cc:1403] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.003s
I20260709 12:16:44.407162 32356 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.407526 32356 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.407900 32356 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.410015 32163 raft_consensus.cc:359] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.410843 32163 raft_consensus.cc:385] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.411228 32163 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.411005 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.412382 32163 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:44.414640 31935 leader_election.cc:343] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.415712 32162 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.416149 32162 ts_tablet_manager.cc:1403] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.016s
I20260709 12:16:44.416663 32163 ts_tablet_manager.cc:1434] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:44.418677 32162 raft_consensus.cc:359] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.419325 32162 raft_consensus.cc:385] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.419572 32162 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.420243 32163 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.420351 32162 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.422425 32162 ts_tablet_manager.cc:1434] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:16:44.423229 32162 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.425450 32290 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.428812 32162 tablet_bootstrap.cc:654] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.429224 32163 tablet_bootstrap.cc:654] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.414670 32164 tablet_bootstrap.cc:654] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.440740 32162 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.441313 32162 ts_tablet_manager.cc:1403] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 12:16:44.444155 32162 raft_consensus.cc:359] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.444928 32162 raft_consensus.cc:385] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.445219 32162 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.445852 32162 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.451251 32163 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.451769 32163 ts_tablet_manager.cc:1403] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.004s
I20260709 12:16:44.455449 32162 ts_tablet_manager.cc:1434] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:16:44.458673 32164 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.459229 32164 ts_tablet_manager.cc:1403] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.000s
I20260709 12:16:44.458873 32163 raft_consensus.cc:359] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.459798 32163 raft_consensus.cc:385] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.460160 32163 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.460950 32163 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.456791 32162 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.462241 32164 raft_consensus.cc:359] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.448025 32356 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.463012 32164 raft_consensus.cc:385] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.464674 32164 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.465549 32164 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.464210 32163 ts_tablet_manager.cc:1434] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:16:44.467070 32163 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.467991 32162 tablet_bootstrap.cc:654] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.469041 32357 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.469513 32357 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.471643 32357 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.472627 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.473089 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.473821 32047 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:44.474175 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:44.474835 32045 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.475440 32164 ts_tablet_manager.cc:1434] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:44.477363 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:44.479408 32290 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.486338 32164 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.488309 32045 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:44.489461 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.490221 32357 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.490736 32357 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:44.491533 32357 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.494665 32164 tablet_bootstrap.cc:654] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.496012 32307 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.497402 32362 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.472597 32163 tablet_bootstrap.cc:654] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.497857 32362 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.500870 32362 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.502228 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.503160 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.503767 31972 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
W20260709 12:16:44.506251 32013 leader_election.cc:343] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.508584 32010 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:44.509447 32163 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.509908 32171 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.510069 32163 ts_tablet_manager.cc:1403] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.043s	user 0.007s	sys 0.006s
I20260709 12:16:44.510356 32171 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.510740 32171 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.513168 32163 raft_consensus.cc:359] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.513856 32163 raft_consensus.cc:385] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.514130 32163 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.514920 32163 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.515686 32169 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.516155 32169 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.517690 32163 ts_tablet_manager.cc:1434] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:44.517879 32169 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.520345 32171 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.522336 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.523077 31972 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:44.522441 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.523785 32163 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.524125 32047 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:44.526285 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:44.528079 32169 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.528599 32169 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.529340 32169 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.540087 32356 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.540799 32163 tablet_bootstrap.cc:654] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.540889 32363 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.541395 32363 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.541899 32171 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.542788 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:44.543429 32120 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.543742 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:16:44.545456 32013 leader_election.cc:343] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.549659 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:44.550144 32290 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.550297 31968 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.553174 32163 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.553311 32120 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:44.553701 32163 ts_tablet_manager.cc:1403] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.002s
I20260709 12:16:44.557329 32163 raft_consensus.cc:359] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.557502 32363 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.558681 32163 raft_consensus.cc:385] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.559183 32163 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.558290 32162 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.560321 32162 ts_tablet_manager.cc:1403] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.104s	user 0.018s	sys 0.044s
I20260709 12:16:44.562937 32162 raft_consensus.cc:359] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.563650 32162 raft_consensus.cc:385] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.565292 32162 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.565140 32363 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.566573 32162 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.570046 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.585741 32162 ts_tablet_manager.cc:1434] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:16:44.588145 32162 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.591223 32164 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.591712 32164 ts_tablet_manager.cc:1403] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.115s	user 0.022s	sys 0.035s
I20260709 12:16:44.592466 32121 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.594296 32164 raft_consensus.cc:359] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.600709 32164 raft_consensus.cc:385] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.601028 32164 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.601830 32164 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.603791 32164 ts_tablet_manager.cc:1434] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:44.604784 32164 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.577029 32169 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.577863 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.583243 31968 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:44.607702 32169 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.607885 32362 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.564451 32163 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.571835 31870 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.610250 32163 ts_tablet_manager.cc:1434] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.001s
I20260709 12:16:44.610735 32164 tablet_bootstrap.cc:654] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:44.614442 32010 leader_election.cc:343] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.614841 32162 tablet_bootstrap.cc:654] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.616904 32163 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
W20260709 12:16:44.622572 32013 leader_election.cc:343] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.641109 32162 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.629972 32010 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:44.643680 32162 ts_tablet_manager.cc:1403] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.010s
I20260709 12:16:44.644851 32363 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.645363 32363 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:44.645392 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:44.646073 32045 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.646345 32363 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.646816 32162 raft_consensus.cc:359] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.647432 32162 raft_consensus.cc:385] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.647704 32162 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.648298 32162 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.649742 32162 ts_tablet_manager.cc:1434] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 12:16:44.650673 32162 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.651280 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:44.651878 31970 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.652062 32013 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:16:44.652951 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45347.2us
I20260709 12:16:44.653937 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:44.654217 32263 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.665184 32169 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.667474 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:44.669008 31970 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:44.671439 32169 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.671926 32169 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:44.672857 32169 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.658612 32162 tablet_bootstrap.cc:654] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.678048 32366 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.676620 32355 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:44.686082 32163 tablet_bootstrap.cc:654] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.694600 32357 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.695638 32355 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.696072 32357 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.708277 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.708937 32045 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:44.715548 32357 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.723150 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.730562 32290 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.733903 32290 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.734407 32290 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.735693 32171 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.736382 32171 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.745787 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.745918 32171 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.747520 32220 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.748590 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.747934 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.754181 31972 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:44.754206 32121 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:44.759544 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:44.760286 32363 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.760635 32363 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.761462 32164 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.762042 32164 ts_tablet_manager.cc:1403] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.157s	user 0.011s	sys 0.027s
I20260709 12:16:44.760939 32363 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.772270 31867 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:44.756829 32119 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:44.787125 32290 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.790580 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:44.791788 32046 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.793344 32164 raft_consensus.cc:359] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.794195 32164 raft_consensus.cc:385] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.797171 32164 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.797808 32164 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.785984 32363 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.799993 32046 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:44.801077 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.801728 32357 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.802174 32357 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:44.802205 32363 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.802906 32164 ts_tablet_manager.cc:1434] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 12:16:44.803113 32357 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.803992 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:44.804265 32164 tablet_bootstrap.cc:492] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:44.804594 32119 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.768532 31868 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.808808 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:44.809719 31967 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.810885 32290 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.817843 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:44.818054 32220 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.802862 32362 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.818986 32362 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.818125 31967 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:44.821087 32362 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:44.823572 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:44.825321 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:44.825601 32362 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.826189 32362 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:44.818521 32120 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.826818 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:44.828388 32010 leader_election.cc:343] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:44.832124 32013 leader_election.cc:343] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.832558 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.833879 32363 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.846215 32164 tablet_bootstrap.cc:654] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.851719 32119 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:44.852986 32362 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.854817 32355 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.856382 32162 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:44.856885 32162 ts_tablet_manager.cc:1403] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.206s	user 0.023s	sys 0.040s
I20260709 12:16:44.865260 32162 raft_consensus.cc:359] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.865998 32162 raft_consensus.cc:385] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.866353 32162 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.867146 32162 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.870879 32162 ts_tablet_manager.cc:1434] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:16:44.872262 32162 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:44.877812 32163 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:44.878470 32163 ts_tablet_manager.cc:1403] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.267s	user 0.010s	sys 0.006s
I20260709 12:16:44.884707 32120 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:44.900270 32163 raft_consensus.cc:359] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.900915 32163 raft_consensus.cc:385] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.901151 32163 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.895948 32366 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.904551 32366 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.904645 32163 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.911064 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.909485 32366 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.911954 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:44.915594 32164 tablet_bootstrap.cc:492] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:44.915956 31972 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:44.916021 32164 ts_tablet_manager.cc:1403] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.007s
I20260709 12:16:44.919049 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.917672 31867 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.922422 32163 ts_tablet_manager.cc:1434] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:16:44.926064 32366 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.937523 32355 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.938077 32355 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.940958 32163 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:44.948391 32355 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:44.949481 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:44.950675 32084 leader_election.cc:343] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.950599 32443 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:16:44.961397 32164 raft_consensus.cc:359] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.962081 32164 raft_consensus.cc:385] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.962318 32164 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.960776 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:44.963052 32164 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:44.964807 31870 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.964835 32164 ts_tablet_manager.cc:1434] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:16:44.966594 32164 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
W20260709 12:16:44.978088 32084 leader_election.cc:343] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.979281 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:44.980189 32355 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.995347 32443 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:16:45.007220 32366 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.040520 32443 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.041072 32443 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:45.150949 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 172892us
I20260709 12:16:45.151654 32162 tablet_bootstrap.cc:700] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Time spent opening tablet: real 0.174s	user 0.000s	sys 0.003s
I20260709 12:16:45.152475 32162 tablet_bootstrap.cc:654] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.157436 32442 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.158003 32442 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.162618 32163 tablet_bootstrap.cc:654] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.165182 32442 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.179054 32164 tablet_bootstrap.cc:654] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.181552 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.176010 32443 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.187848 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:45.196090 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:16:45.185268 31938 leader_election.cc:343] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:45.197310 31938 leader_election.cc:343] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.231630 32445 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.239904 32164 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:45.242353 32164 ts_tablet_manager.cc:1403] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.276s	user 0.016s	sys 0.013s
I20260709 12:16:45.246681 32164 raft_consensus.cc:359] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.248131 32445 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.250830 32163 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:45.251297 32163 ts_tablet_manager.cc:1403] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.311s	user 0.010s	sys 0.007s
I20260709 12:16:45.252311 32164 raft_consensus.cc:385] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.252998 32445 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.248981 32162 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.254153 32162 ts_tablet_manager.cc:1403] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.382s	user 0.013s	sys 0.006s
I20260709 12:16:45.253651 32445 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.256599 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:45.257180 31967 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:45.257241 32162 raft_consensus.cc:359] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.257965 32162 raft_consensus.cc:385] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.258409 32163 raft_consensus.cc:359] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.259034 32163 raft_consensus.cc:385] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.259310 32163 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.259954 32163 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.260152 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:45.262070 32171 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.262451 32171 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.262773 32171 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.263067 32163 ts_tablet_manager.cc:1434] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:16:45.263504 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.264004 32163 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:45.264305 32445 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.258229 32162 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.265403 32162 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.266973 32119 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:45.266139 32164 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.271165 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.275887 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.276791 32164 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.280272 32164 ts_tablet_manager.cc:1434] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.000s
I20260709 12:16:45.273000 32162 ts_tablet_manager.cc:1434] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:45.283533 32355 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.283548 32162 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.284408 32164 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
W20260709 12:16:45.284847 31935 leader_election.cc:343] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.285666 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:45.286636 32357 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.292498 32164 tablet_bootstrap.cc:654] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.283108 32171 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.305742 32355 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.305760 32046 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:45.328370 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:45.329401 32304 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.331059 32162 tablet_bootstrap.cc:654] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.333869 32171 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.339637 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:45.340302 32120 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.343343 32164 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:45.343837 32164 ts_tablet_manager.cc:1403] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.003s
I20260709 12:16:45.356664 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:45.370939 32164 raft_consensus.cc:359] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.371690 32164 raft_consensus.cc:385] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.371954 32164 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.373637 32120 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:45.372995 32164 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.377549 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:45.385120 32363 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.385665 32363 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:45.386538 32363 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.399179 32163 tablet_bootstrap.cc:654] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.399169 32304 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.400995 32454 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.401376 32164 ts_tablet_manager.cc:1434] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.057s	user 0.002s	sys 0.006s
I20260709 12:16:45.402344 32164 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:45.413962 31968 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:45.419555 32162 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.421406 32162 ts_tablet_manager.cc:1403] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.138s	user 0.022s	sys 0.009s
I20260709 12:16:45.424381 32442 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.423097 32304 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:45.433282 32357 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.425246 32442 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.439601 31870 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.441502 32442 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.444190 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.447021 32162 raft_consensus.cc:359] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.449018 32454 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:45.449939 31938 leader_election.cc:343] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.454231 32164 tablet_bootstrap.cc:654] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.455446 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.465745 32162 raft_consensus.cc:385] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.466152 32304 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.469429 32162 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.470134 32162 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.472249 32162 ts_tablet_manager.cc:1434] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.000s
I20260709 12:16:45.473042 32162 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.469607 32304 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:45.476069 31935 leader_election.cc:343] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.476434 32304 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.476485 31935 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:45.476655 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.477324 32118 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:45.478675 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.478914 32363 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.479468 32044 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:45.479506 32363 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.481467 32164 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:45.481904 32164 ts_tablet_manager.cc:1403] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 12:16:45.482275 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:45.483422 32442 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.483824 32442 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.484182 32442 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.492908 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:45.496695 32010 leader_election.cc:343] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.504458 32162 tablet_bootstrap.cc:654] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.506669 32163 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:45.507782 32163 ts_tablet_manager.cc:1403] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.244s	user 0.016s	sys 0.000s
I20260709 12:16:45.506728 32164 raft_consensus.cc:359] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.510917 32164 raft_consensus.cc:385] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.511181 32164 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.511495 32363 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.511840 32164 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.514281 32164 ts_tablet_manager.cc:1434] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 12:16:45.515429 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.515862 32162 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.516062 32119 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:45.516322 32162 ts_tablet_manager.cc:1403] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.002s
I20260709 12:16:45.517055 32164 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:45.517179 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:45.518143 32363 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.518512 32363 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.518841 32363 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.518951 32162 raft_consensus.cc:359] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.519764 32162 raft_consensus.cc:385] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.520051 32162 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.520799 32162 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.522773 32162 ts_tablet_manager.cc:1434] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:16:45.523612 32162 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.510746 32163 raft_consensus.cc:359] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.526973 32163 raft_consensus.cc:385] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.527220 32164 tablet_bootstrap.cc:654] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.527281 32163 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.528456 32163 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.529192 32162 tablet_bootstrap.cc:654] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.532868 32163 ts_tablet_manager.cc:1434] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:16:45.539785 32363 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.541725 32163 tablet_bootstrap.cc:492] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:45.544934 32442 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.546877 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:45.547889 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:16:45.548235 32010 leader_election.cc:343] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.548477 32119 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.477320 32304 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.545895 32162 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.549393 32162 ts_tablet_manager.cc:1403] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.006s
I20260709 12:16:45.559608 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:45.560232 32046 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.561088 32163 tablet_bootstrap.cc:654] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.546903 32363 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.559914 32162 raft_consensus.cc:359] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.563426 32162 raft_consensus.cc:385] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.564044 32162 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.564358 32164 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:45.564880 32164 ts_tablet_manager.cc:1403] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.010s
I20260709 12:16:45.564747 32162 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.567896 32046 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:45.567862 32164 raft_consensus.cc:359] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.568689 32164 raft_consensus.cc:385] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.568969 32164 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.569255 32162 ts_tablet_manager.cc:1434] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:16:45.569340 31935 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:45.569752 32164 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.570154 32162 tablet_bootstrap.cc:492] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.570202 32304 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.571082 32304 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:45.571421 32119 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:45.572027 32304 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.573889 32442 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.575388 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:45.576911 32363 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.577536 32363 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:45.578958 32363 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.579635 32162 tablet_bootstrap.cc:654] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.602556 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:45.603241 32119 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.607664 32164 ts_tablet_manager.cc:1434] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.004s
I20260709 12:16:45.608939 32164 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:45.620101 32164 tablet_bootstrap.cc:654] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.621479 32162 tablet_bootstrap.cc:492] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.621106 31868 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.622052 32162 ts_tablet_manager.cc:1403] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.033s
I20260709 12:16:45.625391 32163 tablet_bootstrap.cc:492] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:45.625160 32162 raft_consensus.cc:359] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.625849 32163 ts_tablet_manager.cc:1403] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.017s
I20260709 12:16:45.625912 32162 raft_consensus.cc:385] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.626361 32162 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.627197 32162 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.629267 32162 ts_tablet_manager.cc:1434] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:45.631053 32162 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.630892 32163 raft_consensus.cc:359] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.631572 32163 raft_consensus.cc:385] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.631752 32119 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:45.631856 32163 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.632754 32163 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.647830 32162 tablet_bootstrap.cc:654] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.654999 32162 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.655709 32162 ts_tablet_manager.cc:1403] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 12:16:45.657038 32163 ts_tablet_manager.cc:1434] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.031s	user 0.002s	sys 0.005s
I20260709 12:16:45.660318 32162 raft_consensus.cc:359] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.661065 32162 raft_consensus.cc:385] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.661371 32162 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.662153 32162 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.664448 32162 ts_tablet_manager.cc:1434] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:45.662694 32163 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:45.666276 32162 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.672127 32162 tablet_bootstrap.cc:654] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.673004 32163 tablet_bootstrap.cc:654] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.673667 31868 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.683557 32162 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.687743 32163 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:45.686902 32162 ts_tablet_manager.cc:1403] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
I20260709 12:16:45.685690 32164 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:45.688681 32164 ts_tablet_manager.cc:1403] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.014s
I20260709 12:16:45.691217 32164 raft_consensus.cc:359] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.691792 32164 raft_consensus.cc:385] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.692039 32164 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.692646 32164 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.693125 32162 raft_consensus.cc:359] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.693761 32162 raft_consensus.cc:385] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.693994 32162 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.694919 32164 ts_tablet_manager.cc:1434] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:45.694644 32162 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.688182 32163 ts_tablet_manager.cc:1403] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.006s
I20260709 12:16:45.696786 32164 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:45.697774 32163 raft_consensus.cc:359] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.698439 32163 raft_consensus.cc:385] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.698714 32163 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.699322 32163 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.701246 32163 ts_tablet_manager.cc:1434] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:16:45.701784 32446 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.702261 32446 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.704272 32446 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:45.704932 32162 ts_tablet_manager.cc:1434] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:16:45.705861 32162 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.707326 32164 tablet_bootstrap.cc:654] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.710686 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.711334 32044 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:45.712230 32162 tablet_bootstrap.cc:654] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.712368 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:45.713742 32169 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.713691 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:45.714262 32169 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.714627 32169 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.714515 31967 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:45.716914 32163 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:45.723229 32163 tablet_bootstrap.cc:654] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.730835 32162 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:45.731299 32162 ts_tablet_manager.cc:1403] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.001s
I20260709 12:16:45.732865 32164 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:45.733366 32164 ts_tablet_manager.cc:1403] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.004s
I20260709 12:16:45.733727 32162 raft_consensus.cc:359] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.734598 32162 raft_consensus.cc:385] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.734877 32162 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.735522 32162 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.736402 32164 raft_consensus.cc:359] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.737107 32164 raft_consensus.cc:385] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.737403 32164 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.738145 32164 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.739691 32164 ts_tablet_manager.cc:1434] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:45.740648 32164 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:45.741242 32363 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.741712 32363 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.742758 32454 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.743232 32454 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.744688 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.755010 32120 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:45.755463 32164 tablet_bootstrap.cc:654] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.756320 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:45.757970 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:45.758822 31967 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:45.761030 32454 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.762578 32447 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.762945 32447 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.763310 32447 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.737448 32162 ts_tablet_manager.cc:1434] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:45.767027 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:45.768373 32013 leader_election.cc:343] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.768240 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:45.768397 32162 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:45.769181 31967 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:45.721846 32169 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.769233 32363 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.775990 32169 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:45.779148 32355 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.779764 32355 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.795290 32162 tablet_bootstrap.cc:654] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.819087 32363 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.819644 32363 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.808684 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:45.821722 32363 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.821866 31971 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:45.809377 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.811715 32220 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.795535 32304 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.807104 32163 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:45.824676 32163 ts_tablet_manager.cc:1403] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.005s
I20260709 12:16:45.823926 32220 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.807519 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:45.825501 31968 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.827584 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.828137 32046 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:45.828516 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.829248 31935 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:45.830041 32442 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.830422 32442 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.806115 32010 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:16:45.830430 31938 leader_election.cc:343] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.830869 32442 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.831749 32163 raft_consensus.cc:359] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.832517 32163 raft_consensus.cc:385] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.832830 32163 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.824209 32304 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.833542 32163 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.835052 32304 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.823210 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.837687 32445 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.837939 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:45.838240 32445 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.838670 32445 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.839211 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:45.837741 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:45.835649 32163 ts_tablet_manager.cc:1434] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:45.841858 32163 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
W20260709 12:16:45.843093 31935 leader_election.cc:343] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.838694 32119 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:45.808269 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:45.846670 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:45.847126 32044 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.782279 32355 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:45.853256 32304 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.853817 32304 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.854274 32304 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.854725 32220 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.791805 32447 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.856344 32445 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.857247 32447 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.858909 32445 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.860296 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:45.860599 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:45.860934 31967 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.861362 31972 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:45.848726 32013 leader_election.cc:343] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.882640 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:45.883503 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:45.884128 32120 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.894948 32120 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:45.906956 31968 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:45.907769 31972 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:45.910055 32163 tablet_bootstrap.cc:654] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:45.914392 32013 leader_election.cc:343] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.915704 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:45.922921 32044 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:45.923321 32362 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.922595 32304 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.926560 32362 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:45.929057 32362 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.930977 31967 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:45.934335 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:45.934561 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:45.934999 32118 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:45.936501 31935 leader_election.cc:343] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.933763 32304 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:45.939334 32447 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.939518 32010 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:45.940321 32363 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.941030 32363 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:45.942276 32363 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.947211 32442 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:45.948745 31971 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:16:45.953084 32084 leader_election.cc:343] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.958755 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:45.959546 32044 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.977012 32447 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:45.978164 32010 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:45.989665 32169 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.998994 32169 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.001075 32169 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:46.001376 32118 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:45.983713 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:46.003945 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:46.008490 32162 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:46.008769 32044 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:46.008965 32162 ts_tablet_manager.cc:1403] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.242s	user 0.036s	sys 0.035s
I20260709 12:16:46.009241 32447 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.010514 32220 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.011044 32220 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:46.013911 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:45.954929 32442 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:16:46.019481 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63322.1us
I20260709 12:16:46.020175 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:16:46.018854 31938 leader_election.cc:343] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.023633 32169 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.024112 32169 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.024607 32169 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.012015 32220 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.029192 32163 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:46.029850 32163 ts_tablet_manager.cc:1403] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.188s	user 0.018s	sys 0.010s
I20260709 12:16:46.033350 31935 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:46.036055 32162 raft_consensus.cc:359] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.036751 32162 raft_consensus.cc:385] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.037060 32162 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.037949 32162 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.024065 32482 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.034399 32363 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.046779 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:46.050215 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:46.051437 32363 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.053407 32363 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:46.052740 31869 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.054970 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:46.055644 31969 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:46.037042 32446 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.056622 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:46.056968 32446 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.057488 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:46.058301 32363 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.058706 32363 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.059013 32446 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:46.059034 32363 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:46.059285 32084 leader_election.cc:343] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.059962 32482 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:46.060856 32482 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.035127 32442 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.067503 32169 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.068446 32120 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:46.069010 32162 ts_tablet_manager.cc:1434] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 12:16:46.070834 32162 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:46.074545 32169 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:46.077703 32164 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:46.077790 32446 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.078271 32164 ts_tablet_manager.cc:1403] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.338s	user 0.070s	sys 0.120s
I20260709 12:16:46.079237 32446 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:46.084883 32485 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:16:46.086531 32446 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:46.081964 32164 raft_consensus.cc:359] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.091773 32164 raft_consensus.cc:385] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.092082 32164 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.092724 32164 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.110695 32164 ts_tablet_manager.cc:1434] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:16:46.072468 32442 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:46.112102 32164 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:46.111914 32163 raft_consensus.cc:359] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.112673 32163 raft_consensus.cc:385] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.113008 32163 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.113795 32163 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:46.125458 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62349us
W20260709 12:16:46.126753 32084 leader_election.cc:343] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.127163 32084 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:46.130424 32169 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.127816 31869 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.136564 32442 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.137988 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:46.138725 31972 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.143667 32163 ts_tablet_manager.cc:1434] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.113s	user 0.011s	sys 0.000s
I20260709 12:16:46.145088 32163 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:46.150134 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:46.163054 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:46.163923 31971 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:46.150842 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:46.165011 32162 tablet_bootstrap.cc:654] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.161823 31972 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:46.169449 31868 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.172873 32447 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:46.165025 32045 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:46.149255 32363 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.179303 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:46.185314 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:46.193114 32084 leader_election.cc:343] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.194489 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:46.198704 32363 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:46.202469 32482 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.202886 32482 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.203253 32482 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.204962 32164 tablet_bootstrap.cc:654] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.208680 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:46.209425 32446 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.209954 32446 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:46.210768 32446 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.212458 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:46.214764 32355 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.217175 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:46.209357 31971 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.220021 32043 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:46.224910 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:46.225605 31967 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:46.225761 32163 tablet_bootstrap.cc:654] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.229122 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:46.239481 32443 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.243213 32355 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.247463 32482 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.263725 31869 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.237442 32362 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.275518 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:46.256461 32482 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:16:46.342768 32084 leader_election.cc:343] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.294458 32497 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:46.375283 32196 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:16:46.306880 32362 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260709 12:16:46.383993 32497 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.427631 32494 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:46.424474 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:46.432487 32045 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.441479 32163 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:46.442036 32163 ts_tablet_manager.cc:1403] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.297s	user 0.020s	sys 0.004s
I20260709 12:16:46.441493 32164 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
W20260709 12:16:46.442468 32500 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.104s	user 0.003s	sys 0.000s
I20260709 12:16:46.442780 32164 ts_tablet_manager.cc:1403] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.331s	user 0.017s	sys 0.019s
I20260709 12:16:46.469759 32263 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.471877 32045 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:46.476073 32263 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.478652 32162 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:46.478500 32164 raft_consensus.cc:359] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.479135 32162 ts_tablet_manager.cc:1403] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.409s	user 0.007s	sys 0.022s
I20260709 12:16:46.479228 32164 raft_consensus.cc:385] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.479565 32164 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.480151 32164 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.481846 32162 raft_consensus.cc:359] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.482254 32164 ts_tablet_manager.cc:1434] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.000s
I20260709 12:16:46.482859 32162 raft_consensus.cc:385] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.483206 32162 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.483592 32164 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:46.483819 32162 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:46.519233 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:46.534655 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:46.546504 32490 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.547045 32490 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:46.545914 31868 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.551036 32162 ts_tablet_manager.cc:1434] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.072s	user 0.018s	sys 0.006s
I20260709 12:16:46.533288 32362 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:16:46.560673 32497 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:46.547816 32490 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.569661 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:46.565815 32163 raft_consensus.cc:359] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.570566 32163 raft_consensus.cc:385] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.570833 32163 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.571434 32163 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.573431 32163 ts_tablet_manager.cc:1434] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.131s	user 0.006s	sys 0.000s
I20260709 12:16:46.574242 32163 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:46.584290 32497 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:16:46.605933 32162 tablet_bootstrap.cc:492] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:46.614545 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:46.624114 31935 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:46.627206 32446 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.630025 32362 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.634205 32164 tablet_bootstrap.cc:654] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.638240 32443 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:16:46.658372 31935 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:46.670687 32163 tablet_bootstrap.cc:654] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.674849 32355 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.675385 32355 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.677315 32355 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:46.685406 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:46.690423 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:46.691573 32040 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:46.682260 31870 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.699914 32121 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.712035 32497 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.717159 32497 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.718640 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:46.722680 32355 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:16:46.757269 32162 tablet_bootstrap.cc:654] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.766714 32482 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.777575 32164 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:46.780290 32164 ts_tablet_manager.cc:1403] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.297s	user 0.025s	sys 0.042s
I20260709 12:16:46.784216 32490 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.796015 32482 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.800232 32164 raft_consensus.cc:359] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.800900 32164 raft_consensus.cc:385] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.801142 32164 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.801731 32164 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.803761 32164 ts_tablet_manager.cc:1434] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:16:46.816900 32164 tablet_bootstrap.cc:492] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:46.830034 31868 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.840866 32163 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:46.841948 32163 ts_tablet_manager.cc:1403] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.268s	user 0.027s	sys 0.000s
I20260709 12:16:46.845949 32162 tablet_bootstrap.cc:492] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:46.846422 32162 ts_tablet_manager.cc:1403] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.241s	user 0.018s	sys 0.053s
I20260709 12:16:46.853677 32163 raft_consensus.cc:359] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.860126 32163 raft_consensus.cc:385] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.859246 32164 tablet_bootstrap.cc:654] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.862057 32162 raft_consensus.cc:359] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.862766 32162 raft_consensus.cc:385] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.863049 32162 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.863734 32162 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.860419 32163 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.868085 32162 ts_tablet_manager.cc:1434] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:16:46.868237 32163 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.869084 32162 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:46.870805 32163 ts_tablet_manager.cc:1434] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.003s
I20260709 12:16:46.874867 32162 tablet_bootstrap.cc:654] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.875535 32163 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:46.885231 32163 tablet_bootstrap.cc:654] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.885545 32164 tablet_bootstrap.cc:492] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:46.886579 32162 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:46.887118 32162 ts_tablet_manager.cc:1403] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.002s
I20260709 12:16:46.888748 32164 ts_tablet_manager.cc:1403] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.000s
I20260709 12:16:46.889686 32363 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.890170 32363 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.891412 32164 raft_consensus.cc:359] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.892067 32164 raft_consensus.cc:385] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.892334 32164 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.893285 32164 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.897497 32363 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:46.897919 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:46.898276 32162 raft_consensus.cc:359] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.904946 32162 raft_consensus.cc:385] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.898985 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:46.905331 32162 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.905618 32121 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:46.906126 32162 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.906885 32013 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:46.905027 31968 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:46.907649 32363 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.908031 32363 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.908411 32363 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.914877 32163 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:46.915400 32163 ts_tablet_manager.cc:1403] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 12:16:46.907658 32162 ts_tablet_manager.cc:1434] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 12:16:46.917455 32162 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:46.921677 32162 tablet_bootstrap.cc:654] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.922446 32163 raft_consensus.cc:359] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.923120 32163 raft_consensus.cc:385] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.923398 32163 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.924142 32163 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.926159 32163 ts_tablet_manager.cc:1434] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:46.927408 32163 tablet_bootstrap.cc:492] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:46.927205 32363 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.929900 32363 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:46.930580 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:46.930887 32164 ts_tablet_manager.cc:1434] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.002s
I20260709 12:16:46.931224 31968 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.931797 32164 tablet_bootstrap.cc:492] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:46.934939 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:46.935462 32121 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.939931 32163 tablet_bootstrap.cc:654] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.942204 32164 tablet_bootstrap.cc:654] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.945056 32121 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:46.946692 32162 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:46.946805 32013 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:46.947168 32162 ts_tablet_manager.cc:1403] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.011s
I20260709 12:16:46.948249 32510 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.948732 32510 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:46.949497 32510 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.949787 32162 raft_consensus.cc:359] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.951861 32162 raft_consensus.cc:385] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.952163 32162 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.952813 32162 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:46.987428 32162 ts_tablet_manager.cc:1434] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.002s
I20260709 12:16:46.989209 32162 tablet_bootstrap.cc:492] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:46.998904 32164 tablet_bootstrap.cc:492] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:46.999421 32164 ts_tablet_manager.cc:1403] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.027s
I20260709 12:16:47.002967 32163 tablet_bootstrap.cc:492] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.003157 32045 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.003454 32163 ts_tablet_manager.cc:1403] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.076s	user 0.022s	sys 0.034s
I20260709 12:16:47.004451 31968 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:46.987581 32446 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.007040 32446 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.009130 32446 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.009760 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:47.011111 32084 leader_election.cc:343] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.014016 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.014287 32482 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.015573 32355 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:47.015650 32084 leader_election.cc:343] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.016260 32084 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.016125 32355 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.018451 32169 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.018302 32164 raft_consensus.cc:359] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.018978 32164 raft_consensus.cc:385] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.019261 32164 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.019615 32442 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.019922 32164 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.020083 32442 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.020848 32163 raft_consensus.cc:359] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.022074 32164 ts_tablet_manager.cc:1434] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.002s
I20260709 12:16:47.018129 32355 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.021535 32163 raft_consensus.cc:385] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.023255 32163 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.022029 32442 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.024116 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.023994 32163 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.025529 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.025523 32162 tablet_bootstrap.cc:654] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.026306 31968 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:47.027354 31868 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:47.027298 31938 leader_election.cc:343] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.031088 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.031991 32355 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.032334 32355 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.032675 32355 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.032889 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.033424 32163 ts_tablet_manager.cc:1434] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.003s
I20260709 12:16:47.034415 32163 tablet_bootstrap.cc:492] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
W20260709 12:16:47.036260 31935 leader_election.cc:343] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.036676 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.037518 32442 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.040118 32355 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.040135 32163 tablet_bootstrap.cc:654] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.043469 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.044167 32040 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:47.045333 32454 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.045789 32454 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.047125 32442 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.047835 32454 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.049065 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.049628 31968 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:47.050513 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.050940 32164 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.050830 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.051267 32121 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:47.047572 32442 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.054327 32355 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.056681 32162 tablet_bootstrap.cc:492] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.056985 32164 tablet_bootstrap.cc:654] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.057152 32162 ts_tablet_manager.cc:1403] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.001s
I20260709 12:16:47.058161 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.060112 32162 raft_consensus.cc:359] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.061087 32162 raft_consensus.cc:385] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.061416 32162 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.062114 32510 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.062599 32510 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.054282 32442 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.062189 32162 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.063002 32510 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.063580 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.081373 32510 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.084172 32482 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.084663 32482 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.086730 32482 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.091156 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.091467 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.092267 31967 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.092988 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.091863 32043 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:47.095439 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:47.096201 32040 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:16:47.097272 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:47.099347 32084 leader_election.cc:343] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.105664 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.104856 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.108074 32304 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.109666 31967 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:47.112222 32482 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.112649 32482 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.113051 32482 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.114817 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.115648 32362 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.116113 32362 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:47.117098 32362 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.125530 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.134819 32162 ts_tablet_manager.cc:1434] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.005s
I20260709 12:16:47.137184 32446 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.154604 32162 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.154604 32164 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.155625 32164 ts_tablet_manager.cc:1403] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.105s	user 0.027s	sys 0.025s
I20260709 12:16:47.166821 32163 tablet_bootstrap.cc:492] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.167532 32163 ts_tablet_manager.cc:1403] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.133s	user 0.021s	sys 0.042s
I20260709 12:16:47.166826 32510 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.175889 32163 raft_consensus.cc:359] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.176880 32163 raft_consensus.cc:385] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.177289 32163 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.183580 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:47.184643 32118 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:47.184232 32163 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.194077 32163 ts_tablet_manager.cc:1434] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:16:47.195137 32163 tablet_bootstrap.cc:492] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.194769 32164 raft_consensus.cc:359] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.198216 32164 raft_consensus.cc:385] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.198652 32164 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.200250 32164 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.206305 32162 tablet_bootstrap.cc:654] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.212587 32164 ts_tablet_manager.cc:1434] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 12:16:47.213635 32164 tablet_bootstrap.cc:492] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.216369 31870 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.225172 31971 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.240902 32164 tablet_bootstrap.cc:654] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.235800 32482 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.268163 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.268828 31968 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.277962 31968 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:47.278978 32482 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.279049 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:47.280081 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.281184 32039 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.282657 32162 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.283166 32162 ts_tablet_manager.cc:1403] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.033s
I20260709 12:16:47.284371 32482 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.284621 32490 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.285706 32482 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:47.286209 32163 tablet_bootstrap.cc:654] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.286603 32482 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.289484 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:47.289680 32162 raft_consensus.cc:359] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.290405 32162 raft_consensus.cc:385] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.290691 32162 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.291359 32162 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.294399 32164 tablet_bootstrap.cc:492] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.294831 32164 ts_tablet_manager.cc:1403] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.081s	user 0.007s	sys 0.017s
I20260709 12:16:47.297366 32164 raft_consensus.cc:359] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.298025 32164 raft_consensus.cc:385] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.298278 32164 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.300352 32164 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.301950 32163 tablet_bootstrap.cc:492] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.302770 32163 ts_tablet_manager.cc:1403] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.108s	user 0.009s	sys 0.003s
I20260709 12:16:47.302296 32164 ts_tablet_manager.cc:1434] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:47.302259 32162 ts_tablet_manager.cc:1434] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:16:47.303916 32162 tablet_bootstrap.cc:492] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.305975 32163 raft_consensus.cc:359] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.306655 32163 raft_consensus.cc:385] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.306957 32163 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.303917 32164 tablet_bootstrap.cc:492] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.306804 31870 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:47.307749 32163 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.309900 32163 ts_tablet_manager.cc:1434] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:47.310822 32163 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.316480 32163 tablet_bootstrap.cc:654] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.316773 32164 tablet_bootstrap.cc:654] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.320061 32162 tablet_bootstrap.cc:654] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.321895 32163 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.322386 32163 ts_tablet_manager.cc:1403] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:16:47.333478 32362 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.334075 32362 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.336313 32362 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.338861 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.339053 32163 raft_consensus.cc:359] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.339948 32163 raft_consensus.cc:385] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.340344 32163 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
W20260709 12:16:47.341069 32013 leader_election.cc:343] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.342046 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.341328 32163 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:47.343930 32010 leader_election.cc:343] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.344449 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.345504 32362 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.345707 32164 tablet_bootstrap.cc:492] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.346725 32164 ts_tablet_manager.cc:1403] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.010s
I20260709 12:16:47.348446 32163 ts_tablet_manager.cc:1434] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:16:47.349277 32164 raft_consensus.cc:359] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.351537 32164 raft_consensus.cc:385] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.351809 32164 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.349290 32163 tablet_bootstrap.cc:492] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.351256 32162 tablet_bootstrap.cc:492] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.352661 32162 ts_tablet_manager.cc:1403] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.004s
I20260709 12:16:47.355187 32162 raft_consensus.cc:359] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.355922 32162 raft_consensus.cc:385] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.356231 32162 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.356855 32162 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.358222 32164 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.360414 32164 ts_tablet_manager.cc:1434] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:16:47.361294 32164 tablet_bootstrap.cc:492] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.361364 32163 tablet_bootstrap.cc:654] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.363348 32162 ts_tablet_manager.cc:1434] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:47.364441 32162 tablet_bootstrap.cc:492] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.374785 32162 tablet_bootstrap.cc:654] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.381470 32164 tablet_bootstrap.cc:654] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.387897 32163 tablet_bootstrap.cc:492] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.388702 32163 ts_tablet_manager.cc:1403] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.011s
I20260709 12:16:47.391360 32362 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.391494 32164 tablet_bootstrap.cc:492] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.392009 32164 ts_tablet_manager.cc:1403] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.005s
I20260709 12:16:47.391868 32362 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.393959 32362 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.388742 32162 tablet_bootstrap.cc:492] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.394578 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.394762 32163 raft_consensus.cc:359] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.394614 32162 ts_tablet_manager.cc:1403] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 12:16:47.395525 32163 raft_consensus.cc:385] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.395438 31971 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:47.395977 32163 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.394614 32164 raft_consensus.cc:359] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.395059 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.396766 32163 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.397204 32164 raft_consensus.cc:385] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:47.398195 32013 leader_election.cc:343] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.398299 32164 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.398586 32162 raft_consensus.cc:359] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.399245 32162 raft_consensus.cc:385] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.399545 32162 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.399284 32164 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.399968 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.400241 32162 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.401319 32362 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.401715 32362 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.408525 32164 ts_tablet_manager.cc:1434] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:47.402096 32362 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.409592 32164 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.409857 32163 ts_tablet_manager.cc:1434] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:16:47.415930 32163 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.421671 32164 tablet_bootstrap.cc:654] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.421761 32162 ts_tablet_manager.cc:1434] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:16:47.422801 32162 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.429278 32162 tablet_bootstrap.cc:654] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.431576 32163 tablet_bootstrap.cc:654] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.431537 32454 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.433393 32164 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.433943 32164 ts_tablet_manager.cc:1403] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 12:16:47.433410 32304 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.434813 32494 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.435016 32304 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.435361 32494 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.437570 32494 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.439564 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.440368 32117 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:47.440176 32164 raft_consensus.cc:359] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.440982 32164 raft_consensus.cc:385] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.441293 32164 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.441481 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.441978 32164 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.443691 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.443997 32164 ts_tablet_manager.cc:1434] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
W20260709 12:16:47.444504 31938 leader_election.cc:343] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.444789 32162 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.445008 32164 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.445194 32162 ts_tablet_manager.cc:1403] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.003s
I20260709 12:16:47.445096 32494 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.448149 32494 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.448530 32494 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.452210 32163 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.452670 32163 ts_tablet_manager.cc:1403] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.004s
I20260709 12:16:47.452817 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.453312 32447 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.455312 32163 raft_consensus.cc:359] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.455942 32163 raft_consensus.cc:385] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.456195 32163 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.456809 32163 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.457520 32494 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.453507 32039 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:47.458398 32304 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.458865 32163 ts_tablet_manager.cc:1434] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:16:47.459246 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.459765 32045 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:47.460917 32162 raft_consensus.cc:359] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.461118 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.462440 32442 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.462839 32442 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.463165 32162 raft_consensus.cc:385] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.463229 32442 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.463514 32162 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.464437 32162 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.467031 32162 ts_tablet_manager.cc:1434] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:16:47.468442 32162 tablet_bootstrap.cc:492] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.479533 32362 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.482494 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:47.482955 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:47.482486 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:47.483922 32121 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.483129 32118 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.482952 32164 tablet_bootstrap.cc:654] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.490228 32121 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:47.490499 32163 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.490990 32494 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.493605 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:47.494298 32363 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.494786 32363 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:47.495520 32363 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.493124 32362 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.499547 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.501089 31970 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.492974 32039 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.499280 32442 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.516544 32162 tablet_bootstrap.cc:654] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.535920 31868 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.543068 32163 tablet_bootstrap.cc:654] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.538554 32490 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.545786 32490 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.547798 32490 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.548918 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:16:47.550272 32084 leader_election.cc:343] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.554159 31970 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:47.561803 32490 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.564255 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.565364 32490 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.566879 32039 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:47.568084 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.571691 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.577411 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:47.575706 32220 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.579486 32220 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.580663 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:47.581190 32045 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.581710 32220 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:16:47.568871 32084 leader_election.cc:343] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:47.578558 31938 leader_election.cc:343] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.582405 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.584000 32482 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.586488 32497 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.586938 32497 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:47.587077 32355 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.587644 32355 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.595409 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:47.596783 31935 leader_election.cc:343] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.597735 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.598520 31971 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:47.599126 32442 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.602914 31968 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:47.568166 32490 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.604696 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.605247 32121 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:47.587702 32497 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.609104 32363 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.609519 32363 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.610453 32263 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.611559 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.613023 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:47.615311 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.613125 32045 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:47.632133 32490 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.632642 32490 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.633105 32363 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.634084 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.635432 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.590071 32355 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.632978 32490 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.639225 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:47.564260 32118 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:47.639225 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:47.641686 32446 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:47.643848 32010 leader_election.cc:343] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.642210 32220 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.636958 32442 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.637229 32043 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 0.
I20260709 12:16:47.648298 32163 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.649057 32163 ts_tablet_manager.cc:1403] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.159s	user 0.021s	sys 0.012s
I20260709 12:16:47.648330 32442 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.642076 32446 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.649428 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:47.650482 32446 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.650808 32118 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:47.652122 32220 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:47.652973 32220 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:16:47.655313 32084 leader_election.cc:343] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.654000 32442 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.660272 32013 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.662114 32164 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.662757 32164 ts_tablet_manager.cc:1403] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.218s	user 0.011s	sys 0.022s
I20260709 12:16:47.661765 32454 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.668642 31868 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:47.674727 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:47.667258 32454 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.681885 32162 tablet_bootstrap.cc:492] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.682381 32162 ts_tablet_manager.cc:1403] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.214s	user 0.011s	sys 0.014s
I20260709 12:16:47.684880 32162 raft_consensus.cc:359] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.685515 32162 raft_consensus.cc:385] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.685765 32162 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.686447 32162 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.686659 32164 raft_consensus.cc:359] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.687263 32164 raft_consensus.cc:385] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.687551 32164 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.688174 32164 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.689111 32162 ts_tablet_manager.cc:1434] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:47.689989 32162 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.690091 32164 ts_tablet_manager.cc:1434] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.000s
I20260709 12:16:47.679644 32454 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.696115 32490 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.710103 32164 tablet_bootstrap.cc:492] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.711927 32490 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.724350 32164 tablet_bootstrap.cc:654] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.724215 32442 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.727866 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:47.727998 32163 raft_consensus.cc:359] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.728792 32163 raft_consensus.cc:385] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.729266 32163 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.729803 32454 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.728725 32043 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.694490 32446 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.730046 32163 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.734709 32446 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:47.735981 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:47.736320 32442 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:16:47.737800 32084 leader_election.cc:343] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.738716 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.739328 31970 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.739622 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:16:47.741777 31935 leader_election.cc:343] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.746227 31970 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:47.758721 32362 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.759841 32454 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.760143 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.760782 31971 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.761474 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:47.762166 31972 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:47.764835 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:47.766281 32560 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.767963 32560 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.768520 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:47.769164 32118 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.776063 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.778676 32263 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.779394 32263 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:47.780505 32263 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.770242 31971 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:16:47.795063 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:47.796329 32355 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.796976 32163 ts_tablet_manager.cc:1434] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.147s	user 0.005s	sys 0.003s
I20260709 12:16:47.796998 32355 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:16:47.798125 32163 tablet_bootstrap.cc:492] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.798055 32355 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.798950 32118 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:47.815940 32043 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
W20260709 12:16:47.827222 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 125904us
I20260709 12:16:47.832253 32162 tablet_bootstrap.cc:654] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.851015 31868 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.860062 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:47.861089 32118 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:47.863268 31867 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.841696 32164 tablet_bootstrap.cc:492] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.866842 32164 ts_tablet_manager.cc:1403] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.157s	user 0.012s	sys 0.000s
I20260709 12:16:47.843210 32163 tablet_bootstrap.cc:654] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.875088 32355 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.869319 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.879678 32497 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.882690 32163 tablet_bootstrap.cc:492] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.883122 32163 ts_tablet_manager.cc:1403] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.001s
I20260709 12:16:47.885751 32162 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.885540 32163 raft_consensus.cc:359] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.886165 32162 ts_tablet_manager.cc:1403] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.196s	user 0.010s	sys 0.001s
I20260709 12:16:47.887228 32164 raft_consensus.cc:359] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.887919 32164 raft_consensus.cc:385] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.888180 32164 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.888798 32164 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.886166 32163 raft_consensus.cc:385] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.890201 32163 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.890882 32163 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.892055 31868 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:47.893888 32164 ts_tablet_manager.cc:1434] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 12:16:47.894810 32164 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.895071 32162 raft_consensus.cc:359] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.895685 32162 raft_consensus.cc:385] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.895948 32162 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.896559 32162 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.899777 32162 ts_tablet_manager.cc:1434] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.002s
I20260709 12:16:47.902529 32163 ts_tablet_manager.cc:1434] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:16:47.903551 32163 tablet_bootstrap.cc:492] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.904142 32162 tablet_bootstrap.cc:492] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Bootstrap starting.
I20260709 12:16:47.912477 32163 tablet_bootstrap.cc:654] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.914746 32162 tablet_bootstrap.cc:654] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.916455 32164 tablet_bootstrap.cc:654] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.922904 32220 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.923400 32220 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.926065 32220 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:47.926571 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:16:47.927987 31938 leader_election.cc:343] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.934247 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:16:47.935571 31935 leader_election.cc:343] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.935997 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:16:47.937429 32497 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.947201 32163 tablet_bootstrap.cc:492] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:47.947872 32163 ts_tablet_manager.cc:1403] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.019s
I20260709 12:16:47.947227 32164 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:47.951287 32164 ts_tablet_manager.cc:1403] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.006s
I20260709 12:16:47.951187 32163 raft_consensus.cc:359] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.951936 32163 raft_consensus.cc:385] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.952257 32163 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.953228 32163 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.954442 32164 raft_consensus.cc:359] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.955080 32164 raft_consensus.cc:385] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.955384 32163 ts_tablet_manager.cc:1434] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:47.956969 32162 tablet_bootstrap.cc:492] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: No bootstrap required, opened a new log
I20260709 12:16:47.957226 32163 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Bootstrap starting.
I20260709 12:16:47.957391 32162 ts_tablet_manager.cc:1403] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.010s
I20260709 12:16:47.959978 32162 raft_consensus.cc:359] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.960703 32162 raft_consensus.cc:385] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.960973 32162 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.955333 32164 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.961695 32164 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.961592 32162 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:47.963829 32162 ts_tablet_manager.cc:1434] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:47.965229 32164 ts_tablet_manager.cc:1434] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:47.968798 32164 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Bootstrap starting.
I20260709 12:16:47.974340 32164 tablet_bootstrap.cc:654] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.974748 32163 tablet_bootstrap.cc:654] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.008939 32362 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.017315 32362 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.023080 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:48.024724 32497 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:48.024902 32010 leader_election.cc:343] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.025233 32497 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.025887 32362 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.027297 32497 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.028898 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:48.029155 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:48.029554 32117 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 0.
I20260709 12:16:48.030706 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:16:48.031198 31935 leader_election.cc:343] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.031415 32497 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.031934 32497 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.032267 32497 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.032382 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:48.033036 32118 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:48.034389 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:48.035050 32363 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.035444 32363 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.036211 32263 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.036875 32263 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.037248 32363 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.037355 32263 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.040467 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:48.040983 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:48.041258 31970 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:48.041642 32121 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:16:48.042222 32497 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.043653 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:48.044634 32497 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.045132 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:48.045286 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:48.046062 32454 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.045651 32121 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:48.047348 31935 leader_election.cc:343] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.052663 32454 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.054631 32121 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:48.055609 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:16:48.056654 32497 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.057333 32497 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:48.058139 32497 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.074373 32164 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: No bootstrap required, opened a new log
I20260709 12:16:48.074833 32164 ts_tablet_manager.cc:1403] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent bootstrapping tablet: real 0.106s	user 0.036s	sys 0.024s
I20260709 12:16:48.076390 32163 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: No bootstrap required, opened a new log
I20260709 12:16:48.077335 32164 raft_consensus.cc:359] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.077525 31868 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.078763 32163 ts_tablet_manager.cc:1403] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent bootstrapping tablet: real 0.122s	user 0.041s	sys 0.047s
I20260709 12:16:48.079205 32164 raft_consensus.cc:385] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.079532 32164 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.074630 32263 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.080170 32164 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.081852 32164 ts_tablet_manager.cc:1434] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:16:48.082060 32263 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.082646 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:48.083118 31970 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.082019 32163 raft_consensus.cc:359] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.083751 32163 raft_consensus.cc:385] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.084028 32163 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.084710 32163 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.086141 32163 ts_tablet_manager.cc:1434] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.001s
I20260709 12:16:48.090279 31970 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:48.092054 32362 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:48.095849 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:48.096252 32363 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.091084 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:48.098791 32121 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.100400 32362 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.100921 32362 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:48.101745 32362 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.127732 32490 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.128216 32490 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.130213 32490 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:48.132144 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:48.132675 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:48.133509 32039 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:48.132133 32121 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:48.135358 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:48.143394 32490 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.153614 32490 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.173452 31868 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.181411 32362 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.182058 32362 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.184480 32362 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.185475 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:48.185757 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:48.186048 32121 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:48.186380 31967 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 0.
I20260709 12:16:48.187101 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:16:48.187788 32362 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.188189 32362 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.188480 32362 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.190038 32510 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.190471 32510 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.192229 32510 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.193387 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:48.193477 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:16:48.194169 32121 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 1: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:16:48.195055 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:16:48.195189 32362 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.195637 32510 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.197239 32362 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:48.197917 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:16:48.198455 31967 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.198534 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:48.199141 32118 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.201960 32510 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.202518 31967 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:48.203751 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:48.204475 32510 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.204913 32118 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:16:48.204998 32510 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:16:48.205833 32510 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:48.213984 31868 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.215596 32220 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.231971 32564 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.248523 32564 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.257984 32497 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.323027 32510 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.333318 32362 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.405443 32490 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.418519 32489 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.459311 32564 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.470362 32497 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.536167 32362 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.545724 32510 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:48.714807 31918 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:48.714751 32454 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.725512 32510 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:49.496196 32220 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.496999 32220 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:49.499208 32220 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:49.500391 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:16:49.500188 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:16:49.501298 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:49.502676 32121 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:16:49.504566 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:49.506263 32220 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:49.507733 32220 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.508040 32220 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.513451 32220 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:49.515184 32220 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:49.515897 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:16:49.516372 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:16:49.516568 32043 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.516908 32118 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.524296 32118 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:16:49.524296 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:16:49.525734 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:16:49.526546 32220 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:49.527014 32220 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:16:49.527894 32220 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:16:49.537940 31867 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 0 to 2, leader changed from <none> to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 2 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.972910 32117 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:49.973985 32600 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:49.995684 32043 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:49.998232 32220 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:51.328317 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:51.364570 32617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:51.364987 32614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:51.365017 32615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:51.368037 30820 server_base.cc:1061] running on GCE node
I20260709 12:16:51.369722 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:51.370021 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:51.370244 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599411370212 us; error 0 us; skew 500 ppm
I20260709 12:16:51.371059 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:51.374037 30820 webserver.cc:533] Webserver started at http://127.30.25.4:43347/ using document root <none> and password file <none>
I20260709 12:16:51.374596 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:51.374823 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:51.375231 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:51.376842 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-3-root/instance:
uuid: "ec7f26ea46944ea498ad253b1b1819b0"
format_stamp: "Formatted at 2026-07-09 12:16:51 on dist-test-slave-rldg"
I20260709 12:16:51.382717 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:16:51.387934 32622 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:51.388942 30820 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:16:51.389423 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-3-root
uuid: "ec7f26ea46944ea498ad253b1b1819b0"
format_stamp: "Formatted at 2026-07-09 12:16:51 on dist-test-slave-rldg"
I20260709 12:16:51.389855 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:51.415190 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:51.416482 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:51.417977 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:51.431033 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:51.431388 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:51.431761 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:51.432051 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:51.495080 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.4:39399
I20260709 12:16:51.495180 32684 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.4:39399 every 8 connection(s)
I20260709 12:16:51.499714 30820 auto_leader_rebalancer-test.cc:439] add a tserver: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:51.518746 32685 heartbeater.cc:344] Connected to a master server at 127.30.25.62:33695
I20260709 12:16:51.519186 32685 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:51.519891 32685 heartbeater.cc:507] Master 127.30.25.62:33695 requested a full tablet report, sending...
I20260709 12:16:51.521945 31868 ts_manager.cc:194] Registered new tserver with Master: ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:16:51.523536 31868 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41188
I20260709 12:16:51.902220 31967 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:51.908792 32117 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.910418 32220 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.915881 32039 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.921051 32612 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.940871 32220 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:51.949432 32039 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:51.956084 32118 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:16:51.972766 31936 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71e0cf115549454c894335e70c43843a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.988189 31868 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.045141 32117 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.055754 32043 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.056361 31970 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.057188 32355 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:52.058082 32705 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:52.068094 32708 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.070704 32043 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:16:52.072621 32085 consensus_peers.cc:597] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c3ed5d095c544f0abfe4e2c375b06cc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:52.075531 31970 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.084072 31967 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.101329 31868 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:16:52.102651 31936 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e828882e6b1940ffb33d4c2f76b47fe9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:52.103524 32117 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.102856 32043 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.106964 32699 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:52.110411 32699 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:52.118870 32702 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.120316 32043 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.137889 32117 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.140810 31867 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.146759 32118 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.148121 32043 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.149901 31970 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.151441 32708 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:52.154484 32085 consensus_peers.cc:597] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 422b4d293d444b06bb00c3f5b66a6f25. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:52.155224 32712 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:52.166370 32708 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.168011 31970 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.173312 32043 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.183919 31867 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:52.526229 32685 heartbeater.cc:499] Master 127.30.25.62:33695 was elected leader, sending a full tablet report...
I20260709 12:16:52.536559 32719 ts_tablet_manager.cc:933] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:52.538837 32719 tablet_copy_client.cc:323] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:16:52.563966 31982 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 71e0cf115549454c894335e70c43843a from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:52.564582 31982 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 71e0cf115549454c894335e70c43843a from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-71e0cf115549454c894335e70c43843a
I20260709 12:16:52.566720 32723 ts_tablet_manager.cc:933] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:52.573338 31982 tablet_copy_source_session.cc:215] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.574738 32723 tablet_copy_client.cc:323] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:16:52.577330 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet e828882e6b1940ffb33d4c2f76b47fe9 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:52.577893 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet e828882e6b1940ffb33d4c2f76b47fe9 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-e828882e6b1940ffb33d4c2f76b47fe9
I20260709 12:16:52.578226 32719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e0cf115549454c894335e70c43843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.587972 31981 tablet_copy_source_session.cc:215] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.591372 32725 ts_tablet_manager.cc:933] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:52.597038 32723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e828882e6b1940ffb33d4c2f76b47fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.599723 32725 tablet_copy_client.cc:323] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:16:52.606346 32719 tablet_copy_client.cc:806] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.607158 32719 tablet_copy_client.cc:670] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.611908 32719 tablet_copy_client.cc:538] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.615919 32723 tablet_copy_client.cc:806] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.616591 32723 tablet_copy_client.cc:670] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.621915 32723 tablet_copy_client.cc:538] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.621618 32719 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:52.627420 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 1c3ed5d095c544f0abfe4e2c375b06cc from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:16:52.628036 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 1c3ed5d095c544f0abfe4e2c375b06cc from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-1c3ed5d095c544f0abfe4e2c375b06cc
I20260709 12:16:52.634344 32723 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:52.645390 32131 tablet_copy_source_session.cc:215] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.649793 32725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ed5d095c544f0abfe4e2c375b06cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.672783 32725 tablet_copy_client.cc:806] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.673446 32725 tablet_copy_client.cc:670] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.673869 32719 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.674914 32719 tablet_bootstrap.cc:492] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:52.675662 32719 ts_tablet_manager.cc:1403] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.019s
I20260709 12:16:52.680402 32719 raft_consensus.cc:359] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.681365 32719 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:52.681675 32725 tablet_copy_client.cc:538] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.682175 32719 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.686509 32719 ts_tablet_manager.cc:1434] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:16:52.689523 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-71e0cf115549454c894335e70c43843a received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:52.689733 32723 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.689996 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-71e0cf115549454c894335e70c43843a on tablet 71e0cf115549454c894335e70c43843a with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:52.690685 32723 tablet_bootstrap.cc:492] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:52.691663 32725 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:52.691830 32723 ts_tablet_manager.cc:1403] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.011s
I20260709 12:16:52.700649 32723 raft_consensus.cc:359] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.701577 32723 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:52.702236 32723 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.708678 32723 ts_tablet_manager.cc:1434] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:16:52.710698 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-e828882e6b1940ffb33d4c2f76b47fe9 received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:52.711185 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-e828882e6b1940ffb33d4c2f76b47fe9 on tablet e828882e6b1940ffb33d4c2f76b47fe9 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:52.722211 32725 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.723248 32725 tablet_bootstrap.cc:492] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:52.723963 32725 ts_tablet_manager.cc:1403] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.033s	user 0.024s	sys 0.003s
I20260709 12:16:52.726956 32725 raft_consensus.cc:359] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.727784 32725 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:52.728451 32725 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.730736 32725 ts_tablet_manager.cc:1434] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:52.732544 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-1c3ed5d095c544f0abfe4e2c375b06cc received from {username='slave'} at 127.0.0.1:34592
I20260709 12:16:52.732996 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-1c3ed5d095c544f0abfe4e2c375b06cc on tablet 1c3ed5d095c544f0abfe4e2c375b06cc with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:52.751536 32725 ts_tablet_manager.cc:933] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:16:52.755121 32725 tablet_copy_client.cc:323] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:16:52.756788 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 422b4d293d444b06bb00c3f5b66a6f25 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:16:52.757339 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 422b4d293d444b06bb00c3f5b66a6f25 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-422b4d293d444b06bb00c3f5b66a6f25
I20260709 12:16:52.764118 32131 tablet_copy_source_session.cc:215] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.770591 32725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b4d293d444b06bb00c3f5b66a6f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.791330 32725 tablet_copy_client.cc:806] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.792244 32725 tablet_copy_client.cc:670] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.796953 32725 tablet_copy_client.cc:538] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.807912 32725 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:52.837497 32725 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.838429 32725 tablet_bootstrap.cc:492] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:52.839155 32725 ts_tablet_manager.cc:1403] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.010s
I20260709 12:16:52.842447 32725 raft_consensus.cc:359] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.843357 32725 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:52.844095 32725 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:52.848615 32725 ts_tablet_manager.cc:1434] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.001s
I20260709 12:16:52.851784 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-422b4d293d444b06bb00c3f5b66a6f25 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:16:52.852237 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-422b4d293d444b06bb00c3f5b66a6f25 on tablet 422b4d293d444b06bb00c3f5b66a6f25 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:53.023538 32660 raft_consensus.cc:1217] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:53.036038 32660 raft_consensus.cc:1217] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:53.129312 32660 raft_consensus.cc:1217] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:53.152184 32660 raft_consensus.cc:1217] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:53.440198 32702 raft_consensus.cc:1064] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:53.442812 32702 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.451761 32117 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.452018 32660 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.453805 32700 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.457548 32699 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.458458 32043 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.471493 32737 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.473372 32117 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.479386 32702 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.483317 32660 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.499158 32043 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.505345 31868 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.540541 31967 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.545590 32660 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:53.546365 32117 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:53.547111 32702 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:53.547897 32699 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:53.555701 32699 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.558066 32117 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.561131 32660 raft_consensus.cc:2955] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.576681 32702 raft_consensus.cc:1064] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:53.575529 31868 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.579082 32702 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.583751 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e828882e6b1940ffb33d4c2f76b47fe9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:53.592908 32117 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.594463 32746 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.605587 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet e828882e6b1940ffb33d4c2f76b47fe9 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:16:53.611351 32660 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.613602 32750 tablet_replica.cc:333] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:16:53.613413 32746 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
W20260709 12:16:53.637950 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53258us
I20260709 12:16:53.638584 32750 raft_consensus.cc:2243] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:53.646797 32750 raft_consensus.cc:2272] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:53.648012 32043 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.651400 32750 ts_tablet_manager.cc:1916] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:53.653146 32700 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.655586 32734 raft_consensus.cc:1064] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:53.656960 32702 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.658288 32734 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.671974 32735 raft_consensus.cc:1064] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:53.676357 32043 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.676020 32660 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.677251 32117 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.678086 32705 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.680588 32735 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.689167 31867 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.699719 32039 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.707757 32750 ts_tablet_manager.cc:1929] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:53.727871 32750 log.cc:1199] T e828882e6b1940ffb33d4c2f76b47fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/e828882e6b1940ffb33d4c2f76b47fe9
I20260709 12:16:53.743754 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet e828882e6b1940ffb33d4c2f76b47fe9 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:53.748456 32660 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.753919 31967 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.755532 32734 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.759781 32660 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.761531 32043 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.763381 31970 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.767452 32757 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.769222 32757 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.770541 32734 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.773628 32757 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:16:53.775485 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 51654.3us
I20260709 12:16:53.802040 32734 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.804143 32660 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.804862 31970 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.810734 32755 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.814746 32043 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.815958 32039 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.823880 32659 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.840996 31868 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.844756 31867 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.864218 31967 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.874624 32118 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.892093 32118 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:53.900187 32659 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:53.903293 32755 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:53.907379 32045 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:53.907773 32658 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:53.910524 32735 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:53.912380 32759 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:53.914273 32760 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.927883 32043 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:53.937147 32758 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.946719 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c3ed5d095c544f0abfe4e2c375b06cc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:53.945556 31867 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.946777 32659 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.949319 32759 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:53.954998 32658 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.956673 32043 raft_consensus.cc:2955] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.961948 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c3ed5d095c544f0abfe4e2c375b06cc with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:16:53.964926 32039 raft_consensus.cc:2955] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:53.970544 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 422b4d293d444b06bb00c3f5b66a6f25 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:53.977059 31867 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.983817 32766 tablet_replica.cc:333] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:16:53.985157 32766 raft_consensus.cc:2243] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:53.985963 32766 raft_consensus.cc:2272] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:53.994369 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 422b4d293d444b06bb00c3f5b66a6f25 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:16:54.013732 32766 ts_tablet_manager.cc:1916] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.032047 32766 ts_tablet_manager.cc:1929] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.032569 32766 log.cc:1199] T 1c3ed5d095c544f0abfe4e2c375b06cc P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/1c3ed5d095c544f0abfe4e2c375b06cc
I20260709 12:16:54.033993 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 1c3ed5d095c544f0abfe4e2c375b06cc (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:54.034189 32766 tablet_replica.cc:333] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:16:54.035233 32766 raft_consensus.cc:2243] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.035998 32766 raft_consensus.cc:2272] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.040021 32766 ts_tablet_manager.cc:1916] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.056339 32766 ts_tablet_manager.cc:1929] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.056733 32766 log.cc:1199] T 422b4d293d444b06bb00c3f5b66a6f25 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/422b4d293d444b06bb00c3f5b66a6f25
I20260709 12:16:54.058271 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 422b4d293d444b06bb00c3f5b66a6f25 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:54.224530 31967 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:54.230463 32658 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.231500 32043 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.232141 32745 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.233081 32702 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.240031 32702 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:54.247900 32043 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:54.250663 32658 raft_consensus.cc:2955] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:54.264664 31867 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:54.284264 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 71e0cf115549454c894335e70c43843a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.289364 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 71e0cf115549454c894335e70c43843a with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:16:54.303539   303 tablet_replica.cc:333] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:16:54.304615   303 raft_consensus.cc:2243] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.305289   303 raft_consensus.cc:2272] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.308549   303 ts_tablet_manager.cc:1916] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.319254   303 ts_tablet_manager.cc:1929] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.319541   303 log.cc:1199] T 71e0cf115549454c894335e70c43843a P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/71e0cf115549454c894335e70c43843a
I20260709 12:16:54.320817 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 71e0cf115549454c894335e70c43843a (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:55.359313 31967 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:55.365229 32121 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:55.365670 31936 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3695041c497f47468894b0baf52fcfd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:55.365731 32043 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.366734   301 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.367671 32745 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.375975   301 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.376827 32121 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.385010 32043 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.392572 31967 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:55.401038 31870 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:55.414214 31936 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e910ef3c0af41bf8d4fba5315d68eef. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:55.414456 32121 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.416013   307 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.417742 32043 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.421092   307 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.431108   311 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.434818 32121 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.446357 32043 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.454948 31967 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:55.455991 31868 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:55.483801 31936 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8709b6ab6ebf4e279d1fe057a1ac58b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:55.484588 32121 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.485273 32039 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.486701   307 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.487454 32745 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.496794   307 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.499794 32117 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.502240 32039 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.514590 31870 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.578509 32039 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:55.589696 31967 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.591910   306 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:55.596899 32121 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.599493 32767 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:55.616091 31970 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.614930   306 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.621229 32121 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:55.632572 31868 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:55.639214 32011 consensus_peers.cc:597] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e4bfb2be45e4fc38e6622722ab10735. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:55.917308   329 ts_tablet_manager.cc:933] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:55.919486   329 tablet_copy_client.cc:323] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:16:55.921208 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 3695041c497f47468894b0baf52fcfd0 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:55.921718 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 3695041c497f47468894b0baf52fcfd0 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-3695041c497f47468894b0baf52fcfd0
I20260709 12:16:55.927340 31981 tablet_copy_source_session.cc:215] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:55.929991   329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3695041c497f47468894b0baf52fcfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:55.941756   329 tablet_copy_client.cc:806] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:55.942389   329 tablet_copy_client.cc:670] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:55.946529   329 tablet_copy_client.cc:538] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:55.955371   329 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:55.972627   329 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:55.973364   329 tablet_bootstrap.cc:492] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:55.973927   329 ts_tablet_manager.cc:1403] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:16:55.976101   329 raft_consensus.cc:359] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:55.976680   329 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:55.977321   329 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:55.983394   329 ts_tablet_manager.cc:1434] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.006s
I20260709 12:16:55.986608 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-3695041c497f47468894b0baf52fcfd0 received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:55.987039 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-3695041c497f47468894b0baf52fcfd0 on tablet 3695041c497f47468894b0baf52fcfd0 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:55.998822   329 ts_tablet_manager.cc:933] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:56.000402   329 tablet_copy_client.cc:323] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:16:56.002302 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 0e910ef3c0af41bf8d4fba5315d68eef from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:56.002897 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 0e910ef3c0af41bf8d4fba5315d68eef from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-0e910ef3c0af41bf8d4fba5315d68eef
I20260709 12:16:56.010383 31981 tablet_copy_source_session.cc:215] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.014672   329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e910ef3c0af41bf8d4fba5315d68eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.026026   329 tablet_copy_client.cc:806] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.026662   329 tablet_copy_client.cc:670] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.030165   329 tablet_copy_client.cc:538] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.039208   329 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:56.082731   334 ts_tablet_manager.cc:933] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:56.121088   329 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:56.126931   329 tablet_bootstrap.cc:492] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:56.128121   329 ts_tablet_manager.cc:1403] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.014s
I20260709 12:16:56.122488   334 tablet_copy_client.cc:323] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
W20260709 12:16:56.130462 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47009.7us
I20260709 12:16:56.135586   342 ts_tablet_manager.cc:933] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:56.135978 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:56.136587 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-8709b6ab6ebf4e279d1fe057a1ac58b5
I20260709 12:16:56.142128   329 raft_consensus.cc:359] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:56.144749   329 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:56.145453   329 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:56.147894   329 ts_tablet_manager.cc:1434] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 12:16:56.149984 31982 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0e910ef3c0af41bf8d4fba5315d68eef received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:56.150477 31982 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0e910ef3c0af41bf8d4fba5315d68eef on tablet 0e910ef3c0af41bf8d4fba5315d68eef with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:56.150655   342 tablet_copy_client.cc:323] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:16:56.150977 31981 tablet_copy_source_session.cc:215] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.155958   334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.172421 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 3e4bfb2be45e4fc38e6622722ab10735 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:16:56.173143 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 3e4bfb2be45e4fc38e6622722ab10735 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-3e4bfb2be45e4fc38e6622722ab10735
I20260709 12:16:56.179929   334 tablet_copy_client.cc:806] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.181255   334 tablet_copy_client.cc:670] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.183066 32057 tablet_copy_source_session.cc:215] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.185976   334 tablet_copy_client.cc:538] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.186825   342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4bfb2be45e4fc38e6622722ab10735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.195577   334 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:56.201617   342 tablet_copy_client.cc:806] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.202323   342 tablet_copy_client.cc:670] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.208683   342 tablet_copy_client.cc:538] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.220944   342 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:56.235742   334 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:56.236778   334 tablet_bootstrap.cc:492] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:56.237560   334 ts_tablet_manager.cc:1403] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.042s	user 0.025s	sys 0.009s
I20260709 12:16:56.240813   334 raft_consensus.cc:359] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:56.241719   334 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:56.242508   334 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:56.252408   342 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:56.252614   334 ts_tablet_manager.cc:1434] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.003s
I20260709 12:16:56.253518   342 tablet_bootstrap.cc:492] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:56.254277   342 ts_tablet_manager.cc:1403] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.034s	user 0.024s	sys 0.009s
I20260709 12:16:56.257261   342 raft_consensus.cc:359] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:56.257715 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-8709b6ab6ebf4e279d1fe057a1ac58b5 received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:56.257995   342 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:56.258276 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-8709b6ab6ebf4e279d1fe057a1ac58b5 on tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:56.258701   342 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:56.261410   342 ts_tablet_manager.cc:1434] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:56.262981 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-3e4bfb2be45e4fc38e6622722ab10735 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:16:56.263392 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-3e4bfb2be45e4fc38e6622722ab10735 on tablet 3e4bfb2be45e4fc38e6622722ab10735 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:56.440580 32659 raft_consensus.cc:1217] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:56.466944 32659 raft_consensus.cc:1217] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:56.527660 32659 raft_consensus.cc:1217] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:56.625190 32659 raft_consensus.cc:1217] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:56.887620   349 raft_consensus.cc:1064] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:56.890383   349 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:56.905933 32659 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:56.907377 32045 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:56.907582   307 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:56.910521   307 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:56.912144 32117 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:56.915717   307 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:56.945588   311 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:56.953466 32117 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:56.962564 32659 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:56.967415 31870 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:56.970065 32045 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:56.992790   349 raft_consensus.cc:1064] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:56.995821   349 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:57.013846   337 raft_consensus.cc:1064] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:57.016319   337 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:16:57.035825 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56750.7us
I20260709 12:16:57.046634 32121 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.047343 32039 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.048866 32659 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.049741   307 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:57.052783   307 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.055099   307 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.066372   360 raft_consensus.cc:1064] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:16:57.070593   360 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:57.075726 32121 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.075865   348 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.087514 31971 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.082284 32039 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.088941   368 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:57.088862 31970 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:57.094632 32660 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.094625 32121 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.098726   306 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.097772   361 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:16:57.101341   361 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.117506 31867 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.118165   361 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.082077 32659 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.085819 32658 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.132277 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0e910ef3c0af41bf8d4fba5315d68eef with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:57.133518   365 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:57.122793 32660 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.130446   368 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.146962 32121 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.158828 31867 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.161974 31971 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.171250 32660 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.171554 32043 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.171867 31869 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.175840   349 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.179060   307 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:16:57.188354 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 70447.5us
I20260709 12:16:57.192309 32119 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.206452 32658 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.241519 32120 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.245483 32120 raft_consensus.cc:2955] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.257958   361 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:16:57.263914   373 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.060s	user 0.001s	sys 0.001s
I20260709 12:16:57.339548 32043 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.344260 32658 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.355916 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e910ef3c0af41bf8d4fba5315d68eef with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:16:57.365329 31867 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.369946   372 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:57.420403 31970 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:57.423928 32117 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.431159 31967 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:57.433811 32658 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.435804   311 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.441752 32659 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.443323   377 tablet_replica.cc:333] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:16:57.444730   377 raft_consensus.cc:2243] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:57.445566   377 raft_consensus.cc:2272] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:57.447059   311 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.452518 32040 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.454573   311 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.457180 32039 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.458786   371 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.462664 32658 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.464900   307 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.472874 32659 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.480772   349 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.488292 31867 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.489303 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3695041c497f47468894b0baf52fcfd0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:57.497922 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:57.489504   377 ts_tablet_manager.cc:1916] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:57.491228 32039 raft_consensus.cc:2955] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.494288 32043 raft_consensus.cc:2955] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.511602 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 3695041c497f47468894b0baf52fcfd0 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:16:57.517472   377 ts_tablet_manager.cc:1929] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:57.518076   377 log.cc:1199] T 0e910ef3c0af41bf8d4fba5315d68eef P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/0e910ef3c0af41bf8d4fba5315d68eef
I20260709 12:16:57.517678 31869 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.520958 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 0e910ef3c0af41bf8d4fba5315d68eef (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:57.535295   379 tablet_replica.cc:333] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:16:57.536607   379 raft_consensus.cc:2243] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:57.537520   379 raft_consensus.cc:2272] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:57.539816 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:16:57.542608   379 ts_tablet_manager.cc:1916] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:57.564555   379 ts_tablet_manager.cc:1929] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:57.565074   379 log.cc:1199] T 3695041c497f47468894b0baf52fcfd0 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/3695041c497f47468894b0baf52fcfd0
I20260709 12:16:57.566989   379 tablet_replica.cc:333] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:16:57.567363 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 3695041c497f47468894b0baf52fcfd0 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:57.568341   379 raft_consensus.cc:2243] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:57.569268   379 raft_consensus.cc:2272] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:57.573261   379 ts_tablet_manager.cc:1916] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:57.589479   379 ts_tablet_manager.cc:1929] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:57.589893   379 log.cc:1199] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/8709b6ab6ebf4e279d1fe057a1ac58b5
I20260709 12:16:57.591776 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 8709b6ab6ebf4e279d1fe057a1ac58b5 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:57.652463 32043 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:16:57.658493 32658 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.659929 32117 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.660151   357 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:57.661525   337 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.666918   357 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.669574 32121 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.670030 32658 raft_consensus.cc:2955] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.685746 31867 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:16:57.694340 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e4bfb2be45e4fc38e6622722ab10735 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:57.700577 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e4bfb2be45e4fc38e6622722ab10735 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:16:57.702448   380 tablet_replica.cc:333] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:16:57.703408   380 raft_consensus.cc:2243] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:57.704151   380 raft_consensus.cc:2272] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:57.706990   380 ts_tablet_manager.cc:1916] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:57.721418   380 ts_tablet_manager.cc:1929] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:57.721848   380 log.cc:1199] T 3e4bfb2be45e4fc38e6622722ab10735 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/3e4bfb2be45e4fc38e6622722ab10735
I20260709 12:16:57.723656 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 3e4bfb2be45e4fc38e6622722ab10735 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:16:58.768674 31970 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:16:58.774845 31936 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d27358a7bd3493a8877bd9ce8ae9055. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:58.775080 32039 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:58.775591 32119 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:58.776340   371 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:58.777551   307 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:58.784278   371 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.785718 32039 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.787199 32119 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.801403 31869 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.797641 32043 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:58.813403 32119 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:58.814463 32011 consensus_peers.cc:597] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfedf707307443deb5a0681825db83cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:58.814958   357 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:58.819307 31967 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:58.820776   357 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:16:58.827675   357 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.830166 32121 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.845197 31970 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:58.855283 31967 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.862289 32121 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:58.863440 31936 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c59a3d3077c4b8db6380a834f6ce3c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:58.864033 32043 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:58.863796   371 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:58.865761   307 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:58.881809   371 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.906639 32043 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:58.908759 31870 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.931169 32121 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:16:58.935457 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62049.9us
I20260709 12:16:58.935192 32040 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.963604 32119 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:58.965636   396 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:58.968770 32011 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fbb363b96dbf430998ab5641d3b03f46. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:58.970913 31967 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:58.973914   393 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:58.980298 31867 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.987287   393 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:58.990515 32120 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:59.005999 31967 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:59.021318 31870 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:16:59.235209   408 ts_tablet_manager.cc:933] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:59.245151   407 ts_tablet_manager.cc:933] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:59.275080   407 tablet_copy_client.cc:323] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:16:59.278147 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet bfedf707307443deb5a0681825db83cb from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:16:59.278944 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet bfedf707307443deb5a0681825db83cb from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-bfedf707307443deb5a0681825db83cb
I20260709 12:16:59.291134   408 tablet_copy_client.cc:323] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:16:59.294205 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 6d27358a7bd3493a8877bd9ce8ae9055 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:59.295148 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 6d27358a7bd3493a8877bd9ce8ae9055 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-6d27358a7bd3493a8877bd9ce8ae9055
I20260709 12:16:59.356765 31981 tablet_copy_source_session.cc:215] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:59.359038 32057 tablet_copy_source_session.cc:215] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:59.361282   408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d27358a7bd3493a8877bd9ce8ae9055. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:59.365255 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47147.5us
I20260709 12:16:59.366732   407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfedf707307443deb5a0681825db83cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:59.436719   413 ts_tablet_manager.cc:933] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:16:59.461449   413 tablet_copy_client.cc:323] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:16:59.463044   407 tablet_copy_client.cc:806] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:59.463512 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 2c59a3d3077c4b8db6380a834f6ce3c0 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:16:59.463830   407 tablet_copy_client.cc:670] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:59.464177 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 2c59a3d3077c4b8db6380a834f6ce3c0 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-2c59a3d3077c4b8db6380a834f6ce3c0
I20260709 12:16:59.467499   408 tablet_copy_client.cc:806] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:59.468109   408 tablet_copy_client.cc:670] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:59.473865   408 tablet_copy_client.cc:538] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:59.476130 31981 tablet_copy_source_session.cc:215] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:59.477783   407 tablet_copy_client.cc:538] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:59.487126   413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c59a3d3077c4b8db6380a834f6ce3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:59.503479   408 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:59.503896   407 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:59.527010   413 tablet_copy_client.cc:806] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:59.527629   413 tablet_copy_client.cc:670] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:59.536619   413 tablet_copy_client.cc:538] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:59.552603   407 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:59.553635   407 tablet_bootstrap.cc:492] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:59.554402   407 ts_tablet_manager.cc:1403] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.052s	user 0.021s	sys 0.007s
I20260709 12:16:59.557091   413 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:59.557331   417 ts_tablet_manager.cc:933] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:16:59.559312   407 raft_consensus.cc:359] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.569952   417 tablet_copy_client.cc:323] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:16:59.568653   408 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:59.571959   407 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:59.572964 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet fbb363b96dbf430998ab5641d3b03f46 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:16:59.573541 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet fbb363b96dbf430998ab5641d3b03f46 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-fbb363b96dbf430998ab5641d3b03f46
I20260709 12:16:59.573449   407 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.584537   407 ts_tablet_manager.cc:1434] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.002s
I20260709 12:16:59.592236 32056 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-bfedf707307443deb5a0681825db83cb received from {username='slave'} at 127.0.0.1:54754
I20260709 12:16:59.592788 32056 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-bfedf707307443deb5a0681825db83cb on tablet bfedf707307443deb5a0681825db83cb with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:59.601105 32057 tablet_copy_source_session.cc:215] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:59.572994   408 tablet_bootstrap.cc:492] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:59.609314   408 ts_tablet_manager.cc:1403] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.112s	user 0.016s	sys 0.017s
I20260709 12:16:59.611446   417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb363b96dbf430998ab5641d3b03f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:59.612282   408 raft_consensus.cc:359] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.613103   408 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:59.613750   408 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.616365   408 ts_tablet_manager.cc:1434] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:59.622910 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-6d27358a7bd3493a8877bd9ce8ae9055 received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:59.623373 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-6d27358a7bd3493a8877bd9ce8ae9055 on tablet 6d27358a7bd3493a8877bd9ce8ae9055 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:59.641880   413 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:59.641579   417 tablet_copy_client.cc:806] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:59.642916   417 tablet_copy_client.cc:670] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:59.643095   413 tablet_bootstrap.cc:492] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:59.644127   413 ts_tablet_manager.cc:1403] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.087s	user 0.020s	sys 0.021s
I20260709 12:16:59.648555   413 raft_consensus.cc:359] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.649421   413 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:59.650127   413 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.653352   417 tablet_copy_client.cc:538] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:59.663621   413 ts_tablet_manager.cc:1434] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 12:16:59.664891   417 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:16:59.672560 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-2c59a3d3077c4b8db6380a834f6ce3c0 received from {username='slave'} at 127.0.0.1:58058
I20260709 12:16:59.672982 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-2c59a3d3077c4b8db6380a834f6ce3c0 on tablet 2c59a3d3077c4b8db6380a834f6ce3c0 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:59.685516   417 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:59.686514   417 tablet_bootstrap.cc:492] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:16:59.687170   417 ts_tablet_manager.cc:1403] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.003s
I20260709 12:16:59.690791   417 raft_consensus.cc:359] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.691705   417 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:16:59.692508   417 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:16:59.694911   417 ts_tablet_manager.cc:1434] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:16:59.696632 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-fbb363b96dbf430998ab5641d3b03f46 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:16:59.697053 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-fbb363b96dbf430998ab5641d3b03f46 on tablet fbb363b96dbf430998ab5641d3b03f46 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:16:59.699153 32659 raft_consensus.cc:1217] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:59.798033 32659 raft_consensus.cc:1217] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:59.829820 32659 raft_consensus.cc:1217] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:00.081264 32659 raft_consensus.cc:1217] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:00.290088   396 raft_consensus.cc:1064] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:00.292481   396 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.301931 32121 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.303454   393 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:00.305548 32659 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.305933 31970 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.306916   393 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.310482   357 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.355585   357 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.361958 31970 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.367105   371 raft_consensus.cc:1064] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:00.367743 32120 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.369469   371 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.381724 31867 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.390121 32658 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.405254 32660 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.407610   432 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.409811 32121 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.412573   432 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.438771 32043 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.441665   432 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.447309   431 raft_consensus.cc:1064] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:00.450109   431 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.449470   432 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.466980 32659 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.487560 32120 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.493994 32121 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.494609 32039 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.496116   432 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.497012   371 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.497300 32658 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.499171   371 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.502219 32043 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.556128 32043 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.561214 32658 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.565943   396 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.560525 31870 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.562407 32117 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.569269   424 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.582947   424 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.588171   428 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.611498 32658 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.599539 32119 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.603305 32660 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.604305 31968 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.630450 31869 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.633741 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bfedf707307443deb5a0681825db83cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:00.635849 32121 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.637722   402 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.645082 32043 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.649501   432 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.650295 32117 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.659333 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet bfedf707307443deb5a0681825db83cb with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:17:00.673290 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d27358a7bd3493a8877bd9ce8ae9055 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:00.677655   393 raft_consensus.cc:1064] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:00.680033   445 tablet_replica.cc:333] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:17:00.680760 32658 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.680451   393 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.681128   445 raft_consensus.cc:2243] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:00.681871   445 raft_consensus.cc:2272] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:00.682152   371 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.692282   445 ts_tablet_manager.cc:1916] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:00.694535 32659 raft_consensus.cc:2955] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.696911 31867 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.707273   445 ts_tablet_manager.cc:1929] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:00.707855   445 log.cc:1199] T bfedf707307443deb5a0681825db83cb P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/bfedf707307443deb5a0681825db83cb
I20260709 12:17:00.657691 32120 raft_consensus.cc:2955] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.712067 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet bfedf707307443deb5a0681825db83cb (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:00.724462 32658 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.724833 31967 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.726241   396 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.728083   396 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:17:00.728149 31968 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.739760 31869 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.749548 32120 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:00.751060   393 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:00.755023 32658 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.757021 32117 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.758213 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d27358a7bd3493a8877bd9ce8ae9055 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:00.762202   371 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.763204   402 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.772114   431 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.774364 32657 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.789417   449 tablet_replica.cc:333] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:17:00.789673 32117 raft_consensus.cc:2955] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.808480 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2c59a3d3077c4b8db6380a834f6ce3c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:00.810735   396 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.812129   449 raft_consensus.cc:2243] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:00.812953   449 raft_consensus.cc:2272] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:00.832331   449 ts_tablet_manager.cc:1916] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:00.836867 32657 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.840521 31868 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:17:00.871594 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43710.4us
I20260709 12:17:00.882541 31967 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.888833   449 ts_tablet_manager.cc:1929] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:00.889482   449 log.cc:1199] T 6d27358a7bd3493a8877bd9ce8ae9055 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/6d27358a7bd3493a8877bd9ce8ae9055
I20260709 12:17:00.894645 32119 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.901532 31869 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.907429 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 6d27358a7bd3493a8877bd9ce8ae9055 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:00.908365 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c59a3d3077c4b8db6380a834f6ce3c0 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:00.910770   449 tablet_replica.cc:333] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:17:00.911938   449 raft_consensus.cc:2243] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:00.916080   449 raft_consensus.cc:2272] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:00.919878   449 ts_tablet_manager.cc:1916] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:00.921414 32043 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:00.947311 31967 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.949117 32657 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:00.952591   449 ts_tablet_manager.cc:1929] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:00.953027   449 log.cc:1199] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/2c59a3d3077c4b8db6380a834f6ce3c0
I20260709 12:17:00.954651   357 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:00.955658   452 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:00.956118 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 2c59a3d3077c4b8db6380a834f6ce3c0 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:00.965898   396 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.969852 32660 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.977964 31967 raft_consensus.cc:2955] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:00.986207 31868 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:01.000319 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fbb363b96dbf430998ab5641d3b03f46 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:01.001808 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet fbb363b96dbf430998ab5641d3b03f46 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:17:01.003865   453 tablet_replica.cc:333] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:17:01.005050   453 raft_consensus.cc:2243] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:01.005862   453 raft_consensus.cc:2272] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:01.010833   453 ts_tablet_manager.cc:1916] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:01.024163   453 ts_tablet_manager.cc:1929] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:01.024551   453 log.cc:1199] T fbb363b96dbf430998ab5641d3b03f46 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/fbb363b96dbf430998ab5641d3b03f46
I20260709 12:17:01.026218 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet fbb363b96dbf430998ab5641d3b03f46 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:02.073807 32120 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.080104 31968 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:02.080091 32043 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:02.080986 32085 consensus_peers.cc:597] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d02c13b00af439e946e76ef880668d9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:02.081740   381 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:02.082549   411 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:02.090969   381 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.092653 32043 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.095201 31968 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.108062 31867 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:02.119108 32039 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.125823 32120 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:02.127228   393 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:02.133144 31968 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:02.134982   357 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:02.137106 32011 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78b16558212549a98cd68f4e744eb410. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:02.145507   393 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.147696 31968 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.175179 32121 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.185334 32119 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.191937 32039 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:02.194058 31868 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:02.199801   381 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:02.205572 31970 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:02.207763 32085 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d2cc7077f01430da5a31385d674e266. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:02.211802   454 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:02.239835   454 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.250291 32039 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.261294 32040 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.265297 31869 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:02.276536 32011 consensus_peers.cc:597] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15d2c5dc46254473902eca585b82958f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:02.280356 32121 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:02.283725 31967 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:02.282541   469 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:02.285327 31970 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.286787   464 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:02.314426   469 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.318817 31970 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.319890 32121 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.338547 31869 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:02.525766   476 ts_tablet_manager.cc:933] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:02.528335   476 tablet_copy_client.cc:323] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:17:02.530651 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 6d02c13b00af439e946e76ef880668d9 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:17:02.531221 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 6d02c13b00af439e946e76ef880668d9 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-6d02c13b00af439e946e76ef880668d9
I20260709 12:17:02.539052 32131 tablet_copy_source_session.cc:215] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:02.542752   476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02c13b00af439e946e76ef880668d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:02.557757   476 tablet_copy_client.cc:806] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:02.558456   476 tablet_copy_client.cc:670] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:02.562197   476 tablet_copy_client.cc:538] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:02.570825   476 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:02.591400   476 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:02.595697   476 tablet_bootstrap.cc:492] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:02.596303   479 ts_tablet_manager.cc:933] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:02.596526   476 ts_tablet_manager.cc:1403] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.007s
I20260709 12:17:02.602088   479 tablet_copy_client.cc:323] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:02.601583   476 raft_consensus.cc:359] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.602658   476 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:02.603410   476 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.604012 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 78b16558212549a98cd68f4e744eb410 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:02.604609 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 78b16558212549a98cd68f4e744eb410 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-78b16558212549a98cd68f4e744eb410
I20260709 12:17:02.606668   476 ts_tablet_manager.cc:1434] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 12:17:02.610343 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-6d02c13b00af439e946e76ef880668d9 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:17:02.610749 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-6d02c13b00af439e946e76ef880668d9 on tablet 6d02c13b00af439e946e76ef880668d9 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:02.624332 32057 tablet_copy_source_session.cc:215] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:02.627661   479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b16558212549a98cd68f4e744eb410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:02.643857   479 tablet_copy_client.cc:806] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:02.644551   479 tablet_copy_client.cc:670] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:02.648337   479 tablet_copy_client.cc:538] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:02.658022   479 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:02.689980   479 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:02.695284   479 tablet_bootstrap.cc:492] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:02.696081   479 ts_tablet_manager.cc:1403] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.038s	user 0.027s	sys 0.001s
I20260709 12:17:02.700376   476 ts_tablet_manager.cc:933] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:02.707333   479 raft_consensus.cc:359] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.708858   476 tablet_copy_client.cc:323] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:17:02.709774   479 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:02.710556   479 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.712188 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 9d2cc7077f01430da5a31385d674e266 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:17:02.712752 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 9d2cc7077f01430da5a31385d674e266 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-9d2cc7077f01430da5a31385d674e266
I20260709 12:17:02.719374   479 ts_tablet_manager.cc:1434] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:17:02.720916 32131 tablet_copy_source_session.cc:215] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:02.722249 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-78b16558212549a98cd68f4e744eb410 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:02.722743 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-78b16558212549a98cd68f4e744eb410 on tablet 78b16558212549a98cd68f4e744eb410 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:02.727916   476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2cc7077f01430da5a31385d674e266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:02.739980   476 tablet_copy_client.cc:806] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:02.740620   476 tablet_copy_client.cc:670] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:02.744805   476 tablet_copy_client.cc:538] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:02.752453   476 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:02.756304   479 ts_tablet_manager.cc:933] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:02.758967   479 tablet_copy_client.cc:323] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:02.760735 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 15d2c5dc46254473902eca585b82958f from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:02.761338 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 15d2c5dc46254473902eca585b82958f from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-15d2c5dc46254473902eca585b82958f
I20260709 12:17:02.771376 32057 tablet_copy_source_session.cc:215] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:02.775418   479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d2c5dc46254473902eca585b82958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:02.780910   476 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:02.781867   476 tablet_bootstrap.cc:492] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:02.782586   476 ts_tablet_manager.cc:1403] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.012s
I20260709 12:17:02.785841   476 raft_consensus.cc:359] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.786854   476 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:02.787593   476 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.793346   476 ts_tablet_manager.cc:1434] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:17:02.795219 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-9d2cc7077f01430da5a31385d674e266 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:17:02.795619 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-9d2cc7077f01430da5a31385d674e266 on tablet 9d2cc7077f01430da5a31385d674e266 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:02.805624   479 tablet_copy_client.cc:806] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:02.806342   479 tablet_copy_client.cc:670] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:02.810292   479 tablet_copy_client.cc:538] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:02.824244   479 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:02.842912   479 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:02.843954   479 tablet_bootstrap.cc:492] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:02.844751   479 ts_tablet_manager.cc:1403] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.008s
I20260709 12:17:02.846951   479 raft_consensus.cc:359] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.847607   479 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:02.848124   479 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:02.850088   479 ts_tablet_manager.cc:1434] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:17:02.852407 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-15d2c5dc46254473902eca585b82958f received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:02.852777 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-15d2c5dc46254473902eca585b82958f on tablet 15d2c5dc46254473902eca585b82958f with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:03.004091 32657 raft_consensus.cc:1217] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:03.035480 32657 raft_consensus.cc:1217] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:03.281008 32657 raft_consensus.cc:1217] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:03.294754 32660 raft_consensus.cc:1217] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:03.412181   487 raft_consensus.cc:1064] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:03.415326   487 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.424052 32119 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.426898 31970 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.427376 32657 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.427734   488 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.429234   487 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.430321   393 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.440657 32657 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.438828   393 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.463760 32119 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.468189 31868 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.519426 31970 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.555022 32040 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.559237 32660 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.559887 31970 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.560526   469 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.562397   487 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.585506   487 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.589460 32660 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.591437 31967 raft_consensus.cc:2955] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.603463 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78b16558212549a98cd68f4e744eb410 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:03.606779 31867 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.618538 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 78b16558212549a98cd68f4e744eb410 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:17:03.632596   498 tablet_replica.cc:333] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:17:03.634125   498 raft_consensus.cc:2243] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:03.634964   498 raft_consensus.cc:2272] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:03.638657   498 ts_tablet_manager.cc:1916] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:03.651985   498 ts_tablet_manager.cc:1929] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:03.652449   498 log.cc:1199] T 78b16558212549a98cd68f4e744eb410 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/78b16558212549a98cd68f4e744eb410
I20260709 12:17:03.653949 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 78b16558212549a98cd68f4e744eb410 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:03.661985   473 raft_consensus.cc:1064] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:03.664484   473 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.671425 32040 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.673691 32660 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.674124 31970 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.675164   472 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.676203   473 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.677004   381 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.694489   472 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.699021 32660 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.711493 31967 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.712316 32039 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.721482   500 raft_consensus.cc:1064] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:03.720887 31868 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:03.724123   500 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.742527 32039 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.743800   500 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.745445 31967 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.748989   500 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.758380 32660 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.758852 32119 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.759996   472 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.769337   473 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.775527 32040 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.783059 32657 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.798090 31870 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.813373 32043 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.817909   381 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.815158 32660 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.822686   393 raft_consensus.cc:1064] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:03.824556   472 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.824904   393 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.829582   381 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.831250 32040 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.845616 32660 raft_consensus.cc:2955] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.860628 32657 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.860194 31967 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.861558 31970 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.862010   488 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:03.863502 32119 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.871315 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d02c13b00af439e946e76ef880668d9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:03.877977   464 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.885047   469 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.888345 31867 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.893544 32121 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:03.903960 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d02c13b00af439e946e76ef880668d9 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:17:03.906096   464 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.911424 31970 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.917424 32657 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.920284   500 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.935456 31970 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.937558   381 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.936900   473 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.938238 32659 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.943843 32660 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.945573 31967 raft_consensus.cc:2955] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.953712 31870 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.958369   515 tablet_replica.cc:333] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:17:03.959595   515 raft_consensus.cc:2243] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:03.973688   515 raft_consensus.cc:2272] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:03.976584 32119 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:03.979215 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9d2cc7077f01430da5a31385d674e266 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:03.984983 31867 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:04.007196   515 ts_tablet_manager.cc:1916] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:04.009315 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d2cc7077f01430da5a31385d674e266 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:04.028249   515 ts_tablet_manager.cc:1929] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:04.028824   515 log.cc:1199] T 6d02c13b00af439e946e76ef880668d9 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/6d02c13b00af439e946e76ef880668d9
I20260709 12:17:04.031999 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 6d02c13b00af439e946e76ef880668d9 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:04.038859   516 tablet_replica.cc:333] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:17:04.042636 32039 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:04.050231 32660 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:04.050220 32119 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:04.051766   488 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:04.052619   469 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:04.060808   464 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:04.061888 32117 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:04.072938   516 raft_consensus.cc:2243] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.073796   516 raft_consensus.cc:2272] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.073880 32660 raft_consensus.cc:2955] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:04.077544   516 ts_tablet_manager.cc:1916] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:04.098619 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15d2c5dc46254473902eca585b82958f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:04.111250 31868 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:04.129113 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 15d2c5dc46254473902eca585b82958f with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:17:04.130043   515 tablet_replica.cc:333] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:17:04.131129   515 raft_consensus.cc:2243] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.131785   515 raft_consensus.cc:2272] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.134197   516 ts_tablet_manager.cc:1929] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:04.134690   516 log.cc:1199] T 9d2cc7077f01430da5a31385d674e266 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/9d2cc7077f01430da5a31385d674e266
I20260709 12:17:04.134725   515 ts_tablet_manager.cc:1916] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:04.136471 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 9d2cc7077f01430da5a31385d674e266 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:04.147543   515 ts_tablet_manager.cc:1929] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:04.147989   515 log.cc:1199] T 15d2c5dc46254473902eca585b82958f P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/15d2c5dc46254473902eca585b82958f
I20260709 12:17:04.149348 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 15d2c5dc46254473902eca585b82958f (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:05.199126 31970 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.215756 32117 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.217378 32040 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:05.217999 31936 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f953fa305a42492da9de0fa8c18c43eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:05.218380   431 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.219125   517 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.237540   431 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.242967 32039 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.243887 32117 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.252336 32040 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.254874 31870 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.271226 32117 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:05.271556 32011 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ca59f0a11104ab882fac419c003e8b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:05.272909   464 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.272378 31970 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.275807   464 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.281448   518 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.290158 32121 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.308722 31968 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.310346 32039 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.310600 31868 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.319353 31970 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.320559 32121 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.320873   531 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.322425   518 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:05.323324 32011 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c52a06827f24bb6bc1851474fc80e53. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:05.331928   464 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.343717 31968 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.344390 32119 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.359512 32121 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.361794 31869 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.374145 32040 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.376524   511 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:05.378422 32085 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3f78e3424864d319ee8da87f4848fe9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:05.378468 31970 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.379844   511 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.391784   532 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.402058 32039 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.408926 31970 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.424396 31870 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:05.651317   541 ts_tablet_manager.cc:933] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:05.653275   541 tablet_copy_client.cc:323] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:17:05.655479 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet f953fa305a42492da9de0fa8c18c43eb from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:17:05.656008 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet f953fa305a42492da9de0fa8c18c43eb from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-f953fa305a42492da9de0fa8c18c43eb
I20260709 12:17:05.661933 31981 tablet_copy_source_session.cc:215] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.664654   541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f953fa305a42492da9de0fa8c18c43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.678743   541 tablet_copy_client.cc:806] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.679342   541 tablet_copy_client.cc:670] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.682839   541 tablet_copy_client.cc:538] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.690939   541 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:05.715854   544 ts_tablet_manager.cc:933] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:05.719058   544 tablet_copy_client.cc:323] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:05.720687 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 8ca59f0a11104ab882fac419c003e8b7 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:05.721201 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 8ca59f0a11104ab882fac419c003e8b7 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-8ca59f0a11104ab882fac419c003e8b7
I20260709 12:17:05.721370   541 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.722256   541 tablet_bootstrap.cc:492] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:05.722988   541 ts_tablet_manager.cc:1403] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.013s
I20260709 12:17:05.725934   541 raft_consensus.cc:359] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.726780   541 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:05.727478   541 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.734788 32057 tablet_copy_source_session.cc:215] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.738530   544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca59f0a11104ab882fac419c003e8b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.739216   541 ts_tablet_manager.cc:1434] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.005s
I20260709 12:17:05.741151 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-f953fa305a42492da9de0fa8c18c43eb received from {username='slave'} at 127.0.0.1:58058
I20260709 12:17:05.741546 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-f953fa305a42492da9de0fa8c18c43eb on tablet f953fa305a42492da9de0fa8c18c43eb with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:05.760999   544 tablet_copy_client.cc:806] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.761565   544 tablet_copy_client.cc:670] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.764772   544 tablet_copy_client.cc:538] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.773416   544 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:05.793435   544 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.794349   544 tablet_bootstrap.cc:492] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:05.795059   544 ts_tablet_manager.cc:1403] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.008s
I20260709 12:17:05.798381   544 raft_consensus.cc:359] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.799196   544 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:05.799870   544 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.803409   544 ts_tablet_manager.cc:1434] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:05.807756 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-8ca59f0a11104ab882fac419c003e8b7 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:05.808168 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-8ca59f0a11104ab882fac419c003e8b7 on tablet 8ca59f0a11104ab882fac419c003e8b7 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:05.852883   544 ts_tablet_manager.cc:933] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:05.855352   544 tablet_copy_client.cc:323] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:17:05.857211 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet f3f78e3424864d319ee8da87f4848fe9 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:17:05.857750 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet f3f78e3424864d319ee8da87f4848fe9 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-f3f78e3424864d319ee8da87f4848fe9
I20260709 12:17:05.876902   541 ts_tablet_manager.cc:933] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:05.888471 32131 tablet_copy_source_session.cc:215] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.891317   541 tablet_copy_client.cc:323] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:05.894073   544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f78e3424864d319ee8da87f4848fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.897786 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 7c52a06827f24bb6bc1851474fc80e53 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:05.898387 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 7c52a06827f24bb6bc1851474fc80e53 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-7c52a06827f24bb6bc1851474fc80e53
I20260709 12:17:05.905438 32057 tablet_copy_source_session.cc:215] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.908152   541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c52a06827f24bb6bc1851474fc80e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.910799   544 tablet_copy_client.cc:806] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.911422   544 tablet_copy_client.cc:670] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.915215   544 tablet_copy_client.cc:538] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.922887   541 tablet_copy_client.cc:806] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.923530   541 tablet_copy_client.cc:670] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.926337   544 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:05.931932   541 tablet_copy_client.cc:538] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.949499   541 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:05.956084   544 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.957039   544 tablet_bootstrap.cc:492] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:05.957746   544 ts_tablet_manager.cc:1403] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.012s
I20260709 12:17:05.972522   544 raft_consensus.cc:359] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.973469   544 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:05.974165   544 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.979408   541 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.980796   541 tablet_bootstrap.cc:492] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:05.981467   544 ts_tablet_manager.cc:1434] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.021s	user 0.001s	sys 0.005s
I20260709 12:17:05.981714   541 ts_tablet_manager.cc:1403] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.016s
I20260709 12:17:05.983994 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-f3f78e3424864d319ee8da87f4848fe9 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:17:05.984411 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-f3f78e3424864d319ee8da87f4848fe9 on tablet f3f78e3424864d319ee8da87f4848fe9 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:05.988715   541 raft_consensus.cc:359] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.990118   541 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:05.990821   541 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:05.993273   541 ts_tablet_manager.cc:1434] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:17:05.994863 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-7c52a06827f24bb6bc1851474fc80e53 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:05.995237 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-7c52a06827f24bb6bc1851474fc80e53 on tablet 7c52a06827f24bb6bc1851474fc80e53 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:06.143062 32659 raft_consensus.cc:1217] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:06.285523 32660 raft_consensus.cc:1217] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:06.289844 32659 raft_consensus.cc:1217] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:06.308710 32659 raft_consensus.cc:1217] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:06.651407   548 raft_consensus.cc:1064] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:06.654088   548 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:06.665920 32121 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.666023 32659 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.667351   554 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.668300   548 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.672191 32039 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.678887   554 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.695482   548 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.699172 32121 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.706435 32040 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.710374 32659 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.728917 31870 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.788770 31968 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:06.789899   553 raft_consensus.cc:1064] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:06.794703 32659 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.795955 32121 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.796527   548 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:06.802383   554 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:06.807705   548 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.808547   553 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:06.817265 32117 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.820685 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f953fa305a42492da9de0fa8c18c43eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:06.809325 32659 raft_consensus.cc:2955] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.829856 31868 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.848335   488 raft_consensus.cc:1064] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:06.850539 32659 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.851552   488 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:06.854015   540 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:06.862099 31968 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.862846 32039 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.865356   540 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.869067 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet f953fa305a42492da9de0fa8c18c43eb with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:06.878677   540 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:06.892405   568 tablet_replica.cc:333] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:17:06.893731   568 raft_consensus.cc:2243] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:06.894762   568 raft_consensus.cc:2272] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:06.902606   568 ts_tablet_manager.cc:1916] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:06.903131   464 raft_consensus.cc:1064] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:06.908841   464 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:06.917063 32117 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.918919 32660 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.919199 31970 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.920631   531 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.920809   553 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.922676   531 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.923141 31970 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.925941 32660 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.935807 32040 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:06.941881   488 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:06.945150 31868 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.976567 32117 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:06.985249 32657 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.987703 31967 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.990732   553 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:06.993018   565 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:17:06.999464 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 78912.2us
I20260709 12:17:06.998842   540 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.000599 32660 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.041368 31870 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:07.077013 32660 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:07.079191 32117 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:07.081583 31968 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:07.081079   572 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:07.099444   488 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:07.106899 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f3f78e3424864d319ee8da87f4848fe9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:07.106871   531 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:07.116057 31967 raft_consensus.cc:2955] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.127954   488 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.130431 31968 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.131817 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet f3f78e3424864d319ee8da87f4848fe9 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:07.139916   531 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.164772 32660 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.170465 31869 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.171294 32121 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.175017 31967 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.177690   568 ts_tablet_manager.cc:1929] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.178112   568 log.cc:1199] T f953fa305a42492da9de0fa8c18c43eb P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/f953fa305a42492da9de0fa8c18c43eb
I20260709 12:17:07.184710 32659 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.193236 31867 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.198031 32117 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.215678   568 tablet_replica.cc:333] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:17:07.216538 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet f953fa305a42492da9de0fa8c18c43eb (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:07.216748   568 raft_consensus.cc:2243] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:07.226866   568 raft_consensus.cc:2272] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:07.234725   568 ts_tablet_manager.cc:1916] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:07.240103 32040 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:07.241048 32045 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:07.246672 32659 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:07.249631   464 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:07.250020 32659 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:07.246635 31968 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:07.250047 31967 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:07.253837   531 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:07.254438   572 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:07.255776   531 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:07.264997   488 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.274534   568 ts_tablet_manager.cc:1929] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.282995 32660 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.289376 31968 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.285326   574 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.294898 32657 raft_consensus.cc:2955] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.300806 31971 raft_consensus.cc:2955] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.303059   568 log.cc:1199] T f3f78e3424864d319ee8da87f4848fe9 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/f3f78e3424864d319ee8da87f4848fe9
I20260709 12:17:07.304821 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet f3f78e3424864d319ee8da87f4848fe9 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:07.312765 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7c52a06827f24bb6bc1851474fc80e53 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:07.321523 31869 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.323765 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8ca59f0a11104ab882fac419c003e8b7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:07.332177 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c52a06827f24bb6bc1851474fc80e53 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:17:07.344419 31867 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:07.351176   582 tablet_replica.cc:333] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:17:07.360539   582 raft_consensus.cc:2243] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:07.361760   582 raft_consensus.cc:2272] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:07.362237 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ca59f0a11104ab882fac419c003e8b7 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:17:07.365675   582 ts_tablet_manager.cc:1916] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:07.377573   582 ts_tablet_manager.cc:1929] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.377938   582 log.cc:1199] T 7c52a06827f24bb6bc1851474fc80e53 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/7c52a06827f24bb6bc1851474fc80e53
I20260709 12:17:07.379359   582 tablet_replica.cc:333] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:17:07.379388 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 7c52a06827f24bb6bc1851474fc80e53 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:07.380149   582 raft_consensus.cc:2243] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:07.380686   582 raft_consensus.cc:2272] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:07.383653   582 ts_tablet_manager.cc:1916] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:07.396263   582 ts_tablet_manager.cc:1929] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.396641   582 log.cc:1199] T 8ca59f0a11104ab882fac419c003e8b7 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/8ca59f0a11104ab882fac419c003e8b7
I20260709 12:17:07.398255 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 8ca59f0a11104ab882fac419c003e8b7 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:08.428347 32040 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:08.434602 32117 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.436080 31968 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.436133   573 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:08.437119 32011 consensus_peers.cc:597] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c82ba0f778d4ff98ba9cc8255f28e9d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.437558   488 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.444548   583 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.446911 32117 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.463027 32121 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:08.465785 31866 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.471030 31968 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.475415 32040 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.477061   539 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:08.477825 31971 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:08.479187 32085 consensus_peers.cc:597] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b2dd1839ae848ae849e1ec74ce14112. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.482079   577 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.504415 32045 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.506194   539 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.548372 31971 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.560577 32117 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:08.572856 31866 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:17:08.580229 32085 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f93a3ac5b02416a8d1d2547d9a9c726. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.580289 32045 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.580744 31971 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.581708   577 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.582564   539 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.594924   539 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.596714 32045 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.598879 31971 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.612170 31869 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.613500 32045 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:08.621575 31971 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:08.621877 32011 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0292f21aab4a4843b2b424c3497ddced. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.623031   587 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.634737 32117 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.636906   587 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:08.656183   488 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.658437 31971 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.663267 32121 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.685878 31867 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:08.881973   606 ts_tablet_manager.cc:933] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:08.885737   606 tablet_copy_client.cc:323] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:17:08.887418 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 5b2dd1839ae848ae849e1ec74ce14112 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:17:08.887952 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 5b2dd1839ae848ae849e1ec74ce14112 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-5b2dd1839ae848ae849e1ec74ce14112
I20260709 12:17:08.895265 32131 tablet_copy_source_session.cc:215] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:08.898080   606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b2dd1839ae848ae849e1ec74ce14112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:08.909024   606 tablet_copy_client.cc:806] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:08.909539   606 tablet_copy_client.cc:670] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:08.913147   606 tablet_copy_client.cc:538] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:08.919469   606 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:08.941304   606 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:08.942173   606 tablet_bootstrap.cc:492] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:08.942829   606 ts_tablet_manager.cc:1403] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.003s
I20260709 12:17:08.945751   606 raft_consensus.cc:359] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:08.946573   606 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:08.947178   606 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:08.955402   606 ts_tablet_manager.cc:1434] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 12:17:08.957075 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-5b2dd1839ae848ae849e1ec74ce14112 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:17:08.957482 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-5b2dd1839ae848ae849e1ec74ce14112 on tablet 5b2dd1839ae848ae849e1ec74ce14112 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:09.010421   606 ts_tablet_manager.cc:933] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:09.011960   606 tablet_copy_client.cc:323] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:09.013532 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:09.013968 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 0c82ba0f778d4ff98ba9cc8255f28e9d from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-0c82ba0f778d4ff98ba9cc8255f28e9d
I20260709 12:17:09.019410 32057 tablet_copy_source_session.cc:215] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:09.022269   606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.037081   606 tablet_copy_client.cc:806] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:09.037673   606 tablet_copy_client.cc:670] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:09.041152   606 tablet_copy_client.cc:538] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:09.050727   606 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:09.072440   606 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:09.073390   606 tablet_bootstrap.cc:492] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:09.074112   606 ts_tablet_manager.cc:1403] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 12:17:09.077127   606 raft_consensus.cc:359] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:09.077956   606 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:09.078855   606 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:09.080734   606 ts_tablet_manager.cc:1434] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:09.083200 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0c82ba0f778d4ff98ba9cc8255f28e9d received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:09.083628 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0c82ba0f778d4ff98ba9cc8255f28e9d on tablet 0c82ba0f778d4ff98ba9cc8255f28e9d with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:09.088961   613 ts_tablet_manager.cc:933] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:09.094506   613 tablet_copy_client.cc:323] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:09.096091 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 0292f21aab4a4843b2b424c3497ddced from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:09.096637 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 0292f21aab4a4843b2b424c3497ddced from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-0292f21aab4a4843b2b424c3497ddced
I20260709 12:17:09.104945 32057 tablet_copy_source_session.cc:215] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:09.107985   613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0292f21aab4a4843b2b424c3497ddced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.124161   613 tablet_copy_client.cc:806] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:09.124761   613 tablet_copy_client.cc:670] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:09.128145   606 ts_tablet_manager.cc:933] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:09.133692   613 tablet_copy_client.cc:538] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:09.135710   606 tablet_copy_client.cc:323] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:17:09.137576 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 2f93a3ac5b02416a8d1d2547d9a9c726 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:17:09.138036 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 2f93a3ac5b02416a8d1d2547d9a9c726 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-2f93a3ac5b02416a8d1d2547d9a9c726
I20260709 12:17:09.145475 32131 tablet_copy_source_session.cc:215] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:09.145478   613 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:09.148885   606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f93a3ac5b02416a8d1d2547d9a9c726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.167505   606 tablet_copy_client.cc:806] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:09.168396   606 tablet_copy_client.cc:670] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:09.178423   606 tablet_copy_client.cc:538] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:09.186254   613 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:09.187469   613 tablet_bootstrap.cc:492] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:09.188665   613 ts_tablet_manager.cc:1403] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.011s
I20260709 12:17:09.191936   613 raft_consensus.cc:359] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:09.193113   606 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:09.193994   613 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:09.194782   613 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:09.197263   613 ts_tablet_manager.cc:1434] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:17:09.223452 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0292f21aab4a4843b2b424c3497ddced received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:09.223934 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0292f21aab4a4843b2b424c3497ddced on tablet 0292f21aab4a4843b2b424c3497ddced with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:09.228752   606 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:09.229598   606 tablet_bootstrap.cc:492] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:09.230212   606 ts_tablet_manager.cc:1403] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.009s
I20260709 12:17:09.232926   606 raft_consensus.cc:359] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:09.233664   606 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:09.234212   606 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:09.244046   606 ts_tablet_manager.cc:1434] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:17:09.245708 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-2f93a3ac5b02416a8d1d2547d9a9c726 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:17:09.246119 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-2f93a3ac5b02416a8d1d2547d9a9c726 on tablet 2f93a3ac5b02416a8d1d2547d9a9c726 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:09.514295 32659 raft_consensus.cc:1217] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.530974 32660 raft_consensus.cc:1217] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.546429 32660 raft_consensus.cc:1217] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.733964 32660 raft_consensus.cc:1217] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:10.044302   577 raft_consensus.cc:1064] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:10.047566   577 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.056066 32660 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.056185 32039 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.057696   539 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.059476   577 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.058218 31971 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.069813   595 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.071365   488 raft_consensus.cc:1064] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:10.073808   488 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.102350 32121 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.103070 32660 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.104674 31967 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.104103   595 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.107715   620 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.111321   630 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.110737 31967 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.114612   620 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.131098 32660 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.146809 32039 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.159863   620 raft_consensus.cc:1064] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:10.161252 31866 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.168272   488 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.176057   620 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.180866 32119 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.198058 32121 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.200369   630 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.202824 32660 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.208338   630 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.216229 31967 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.220000   632 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.223528 31971 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.251000 32657 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:17:10.269626 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66487.9us
I20260709 12:17:10.275352   620 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.282675 31971 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.295375 32119 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.298072   539 raft_consensus.cc:1064] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:10.301540   539 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.306082 32659 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.330411 32659 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.331893   595 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.332777 31971 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.332913 32119 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.335130 32039 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.334336 31870 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.336841 31870 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.341152   595 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.342113   577 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.350140 32657 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.351790   595 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.361187 32657 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.361163   577 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:17:10.379611 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54849.6us
I20260709 12:17:10.402527 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5b2dd1839ae848ae849e1ec74ce14112 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.408845 31869 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.414027   649 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.424330 31967 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.426946 32046 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.427313 32039 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.445730 32044 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.447284   577 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.453379 32039 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.454283 32659 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.458159 32120 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.458587 32046 raft_consensus.cc:2955] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.456785 32660 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.461355   488 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.466739 31870 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.475459 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b2dd1839ae848ae849e1ec74ce14112 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:17:10.476199   633 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.476729 31967 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.480297   633 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.486500   620 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.481376 32660 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.489266 32119 raft_consensus.cc:2955] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.493762   632 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.496687 32659 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.500984   488 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.505771 31968 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.513846 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0292f21aab4a4843b2b424c3497ddced with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.514883 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.517376 32659 raft_consensus.cc:2955] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.534510   654 tablet_replica.cc:333] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:17:10.536949 31866 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.540815   654 raft_consensus.cc:2243] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.543031   654 raft_consensus.cc:2272] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.543406 31869 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 3 to 4, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.553723 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c82ba0f778d4ff98ba9cc8255f28e9d with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56068
I20260709 12:17:10.555598 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 0292f21aab4a4843b2b424c3497ddced with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47762
I20260709 12:17:10.563680   654 ts_tablet_manager.cc:1916] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.570461 32119 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:10.589548 32659 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.593755 31967 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.594705   650 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.595655   577 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.602737   647 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.609501 32660 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.612159   654 ts_tablet_manager.cc:1929] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.612509   655 tablet_replica.cc:333] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:17:10.612764   654 log.cc:1199] T 5b2dd1839ae848ae849e1ec74ce14112 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/5b2dd1839ae848ae849e1ec74ce14112
I20260709 12:17:10.614945 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 5b2dd1839ae848ae849e1ec74ce14112 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:10.615379   654 tablet_replica.cc:333] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:17:10.616475   654 raft_consensus.cc:2243] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.616098 31967 raft_consensus.cc:2955] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.617126   654 raft_consensus.cc:2272] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.617348   655 raft_consensus.cc:2243] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.618211   655 raft_consensus.cc:2272] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.621829   655 ts_tablet_manager.cc:1916] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.636418   654 ts_tablet_manager.cc:1916] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.652006   655 ts_tablet_manager.cc:1929] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.653026   655 log.cc:1199] T 0292f21aab4a4843b2b424c3497ddced P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/0292f21aab4a4843b2b424c3497ddced
I20260709 12:17:10.652006   654 ts_tablet_manager.cc:1929] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.654268   654 log.cc:1199] T 0c82ba0f778d4ff98ba9cc8255f28e9d P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/0c82ba0f778d4ff98ba9cc8255f28e9d
I20260709 12:17:10.656004 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 0c82ba0f778d4ff98ba9cc8255f28e9d (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:10.657014 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 0292f21aab4a4843b2b424c3497ddced (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:10.657706 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f93a3ac5b02416a8d1d2547d9a9c726 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.658535 31870 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 3 to 4, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:10.670020 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f93a3ac5b02416a8d1d2547d9a9c726 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:10.672171   656 tablet_replica.cc:333] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:17:10.675604   656 raft_consensus.cc:2243] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.676496   656 raft_consensus.cc:2272] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.680020   656 ts_tablet_manager.cc:1916] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.690815   656 ts_tablet_manager.cc:1929] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.691185   656 log.cc:1199] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/2f93a3ac5b02416a8d1d2547d9a9c726
I20260709 12:17:10.692770 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 2f93a3ac5b02416a8d1d2547d9a9c726 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:17:11.501015 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:11.557523 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d2c5dc46254473902eca585b82958f"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:11.558100 32039 raft_consensus.cc:606] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:11.574288 32120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:11.574844 32120 raft_consensus.cc:606] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:11.577235 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:11.577684 32039 raft_consensus.cc:606] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:11.579821 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:11.580313 32039 raft_consensus.cc:606] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:11.582880 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:11.583379 32039 raft_consensus.cc:606] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:11.586992 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:11.587543 32039 raft_consensus.cc:606] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:11.593008   633 raft_consensus.cc:993] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:11.593480   620 raft_consensus.cc:1081] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:11.595191 32660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:11.595753 32660 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.596035 32660 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.603163 32660 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:11.604981 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:17:11.605530 31967 raft_consensus.cc:606] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:11.607637 32660 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:11.607914 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:11.608214 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:11.608700 32039 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.608774 32117 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.609067 32039 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:11.609817 32039 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:11.610860 32039 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.614166 32117 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:11.607185 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:17:11.615356 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:11.615715 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:11.615885 32039 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:11.616868 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:11.617964   661 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.619297   661 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:11.620049   661 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:11.628786 31869 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.664160   633 raft_consensus.cc:993] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:11.664657   620 raft_consensus.cc:1081] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:11.666129 32660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:11.666659 32660 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.667057 32660 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.673723 32660 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:11.677603 32660 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:11.678576 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:11.678663 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:11.679251 31968 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.679299 32046 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.679713 32046 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:11.680445 32046 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:11.681460 32046 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.685613 31968 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:11.686829 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:11.687657   661 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.688118   661 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:11.688265 32046 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:11.688997   661 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:11.696098   623 raft_consensus.cc:993] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:11.696647   623 raft_consensus.cc:1081] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:11.698843 32046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:17:11.700330 32046 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.700670 32046 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.707156 32046 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.710193 32046 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:11.711223 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:11.711905 32120 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.714480 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:11.713944 31869 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.715140 31968 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.715529 31968 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:11.716307 31968 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.717473 31968 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.721652 32120 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:11.723097 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:11.724041   620 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.725015   620 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:11.725816   620 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.729415 31968 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:11.740052   577 raft_consensus.cc:993] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:11.740680   650 raft_consensus.cc:1081] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:11.742615 32046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:17:11.745311 32046 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.745708 32046 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.751864 32046 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.754189 32046 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:11.754240 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:11.753450 31869 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 2, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.755381 31968 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.754864 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:11.757344 32120 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.757653 32120 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:11.758503 32120 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.759646 32120 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.765677 31968 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:11.766805 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:11.767444   620 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.767912   620 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:11.767979 32120 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:11.768610   620 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.779196 31869 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 2, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.789829 32046 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:11.796627 32120 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:11.797891   620 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:11.798915 32011 consensus_peers.cc:597] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b83e7300b3fe4d21a97d941dee98fa81. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:11.799042 31968 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:11.800254   620 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:11.808627   667 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.811205 31968 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.815363 32120 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.822921 31869 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.825021 32117 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:11.846653 32046 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:11.847352 32085 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1612f8e575bd46cf8069fa1c21ca192b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:11.850343   650 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.875602   650 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.877830 32039 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.879356 31968 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:11.880625   676 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:11.889261 31968 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.893926 31870 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.908457 32046 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:11.912204   620 raft_consensus.cc:993] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:11.912634   620 raft_consensus.cc:1081] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:11.913961 32117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:17:11.914561 32117 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.915460 32117 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.924477 32117 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.926918 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:11.927475 31967 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.935438 32117 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:11.936455 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:11.936934 32039 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.937184 32039 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:11.937841 32039 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.938630 32039 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.935469 31967 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:11.947263 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:11.948125   650 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.948644   650 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:11.949674   650 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:11.951231 32117 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:11.952713 31967 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:11.952795   667 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.954475 32039 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:11.954689   667 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:11.957559 32011 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 826cffce8bb540c8beb56cc00a702782. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:11.967283   633 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.969492 32117 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:11.973008 31870 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 2 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.973626 31967 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:17:11.984615 31918 auto_rebalancer.cc:629] Failed to schedule move for tablet 1a06d9856d134874b3144fd2abdbfc71: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:11.992951 31867 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 2, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 1 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:12.026227 32117 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.027768   661 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.033519   667 raft_consensus.cc:993] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:12.034047   633 raft_consensus.cc:1081] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:12.036281 32119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:17:12.036996 32119 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.037480 32119 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.040964 32039 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.042658   662 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.051110 32119 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:12.053803 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:12.054431 31967 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.055249 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:12.056071 32046 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.056416 32046 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:12.057246 32046 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:12.059127 32046 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.066406 32119 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:12.075700 32046 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:12.077122 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:12.079159   650 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.079638   650 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:12.080472   650 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:12.091064 31967 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:12.103225 31869 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 2 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.136247   667 raft_consensus.cc:993] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:12.136745   633 raft_consensus.cc:1081] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:12.138147 32660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15d2c5dc46254473902eca585b82958f"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:12.138692 32660 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.139017 32660 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.145354 32660 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.147867 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:12.148428 32039 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.148707 32039 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:12.149466 32039 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.150890 32039 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.158818 32039 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:12.159415 32660 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:12.159821 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:12.159946 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:12.160712 32119 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.160777   662 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.161382   662 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:12.162223   662 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.170773 32119 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:12.175773 31967 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.176620 31870 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.183159   633 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.208204 32119 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.211745   633 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.277016 31967 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.278281   662 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.296012 32039 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.298283   661 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.313205   698 ts_tablet_manager.cc:933] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:12.316941   698 tablet_copy_client.cc:323] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:12.318742 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet b83e7300b3fe4d21a97d941dee98fa81 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:12.319319 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet b83e7300b3fe4d21a97d941dee98fa81 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-b83e7300b3fe4d21a97d941dee98fa81
I20260709 12:17:12.324182 31967 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.325843   633 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.327636 32057 tablet_copy_source_session.cc:215] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.331888   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b83e7300b3fe4d21a97d941dee98fa81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.337426 32119 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.339248   667 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.384172   698 tablet_copy_client.cc:806] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:12.384835   698 tablet_copy_client.cc:670] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:12.388823   698 tablet_copy_client.cc:538] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:12.402014   698 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:12.426340   698 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:12.427340   698 tablet_bootstrap.cc:492] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:12.428049   698 ts_tablet_manager.cc:1403] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.010s
I20260709 12:17:12.431509   698 raft_consensus.cc:359] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:12.432351   704 ts_tablet_manager.cc:933] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:12.432447   698 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:12.433307   698 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:12.436693   698 ts_tablet_manager.cc:1434] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:12.439234 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-b83e7300b3fe4d21a97d941dee98fa81 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:12.439693 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-b83e7300b3fe4d21a97d941dee98fa81 on tablet b83e7300b3fe4d21a97d941dee98fa81 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.446835   704 tablet_copy_client.cc:323] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:17:12.448419 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 826cffce8bb540c8beb56cc00a702782 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:17:12.448939 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 826cffce8bb540c8beb56cc00a702782 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-826cffce8bb540c8beb56cc00a702782
I20260709 12:17:12.460542 32057 tablet_copy_source_session.cc:215] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.461864   698 ts_tablet_manager.cc:933] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:12.464707   698 tablet_copy_client.cc:323] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:17:12.466290   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 826cffce8bb540c8beb56cc00a702782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.467940 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 1612f8e575bd46cf8069fa1c21ca192b from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:17:12.468604 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 1612f8e575bd46cf8069fa1c21ca192b from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-1612f8e575bd46cf8069fa1c21ca192b
I20260709 12:17:12.480082 31968 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.482879   675 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.490705 32131 tablet_copy_source_session.cc:215] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.494362   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1612f8e575bd46cf8069fa1c21ca192b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.516644   704 tablet_copy_client.cc:806] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:12.517246   704 tablet_copy_client.cc:670] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:12.522960   704 tablet_copy_client.cc:538] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:12.526275 32039 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.527902   675 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.533954   698 tablet_copy_client.cc:806] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:12.534770   698 tablet_copy_client.cc:670] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:12.542696   698 tablet_copy_client.cc:538] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:12.537781   704 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:12.560822   698 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:17:12.578691 32117 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.578958 31968 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.580204   662 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.586439   675 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.600229   704 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:12.601850 32039 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.605769   662 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.612828   698 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:12.613884   698 tablet_bootstrap.cc:492] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:12.614679   698 ts_tablet_manager.cc:1403] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.014s
I20260709 12:17:12.615334   704 tablet_bootstrap.cc:492] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:17:12.615995   704 ts_tablet_manager.cc:1403] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.006s
I20260709 12:17:12.623253   698 raft_consensus.cc:359] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:12.634775   698 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:12.635437   698 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:12.637928   704 raft_consensus.cc:359] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:12.645843   704 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:17:12.646873   704 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:12.649255   704 ts_tablet_manager.cc:1434] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:17:12.638360   698 ts_tablet_manager.cc:1434] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:17:12.654765 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-826cffce8bb540c8beb56cc00a702782 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:17:12.655254 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-826cffce8bb540c8beb56cc00a702782 on tablet 826cffce8bb540c8beb56cc00a702782 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.670219 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-1612f8e575bd46cf8069fa1c21ca192b received from {username='slave'} at 127.0.0.1:34592
I20260709 12:17:12.670694 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-1612f8e575bd46cf8069fa1c21ca192b on tablet 1612f8e575bd46cf8069fa1c21ca192b with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.675772 32039 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.676994 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:12.687273   712 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.759518 31968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71e0cf115549454c894335e70c43843a"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:17:12.760033 31968 raft_consensus.cc:606] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.762688 32046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:12.763188 32046 raft_consensus.cc:606] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.765741 32117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:12.766207 32117 raft_consensus.cc:606] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:12.768882 32046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfedf707307443deb5a0681825db83cb"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:12.769471 32046 raft_consensus.cc:606] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.772217 32046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:12.772957 32046 raft_consensus.cc:606] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:12.775502 32046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:12.776098 32046 raft_consensus.cc:606] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:12.779675 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:12.781075 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:12.781716 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:12.799726   633 raft_consensus.cc:993] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:12.800274   667 raft_consensus.cc:1081] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:12.801769 32657 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfedf707307443deb5a0681825db83cb"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:12.802270 32657 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.802627 32657 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.808947 32657 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.812132 32657 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:12.812649 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:12.813186 32117 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.814735 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:12.815291 32046 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.815585 32046 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:12.817683 32046 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.819007 32046 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.819283 32117 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:12.820426 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:12.821372   662 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.821817   662 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:12.822695   662 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.835918 32046 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:12.843098 32657 raft_consensus.cc:1217] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:12.861577   702 raft_consensus.cc:993] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:12.862123   667 raft_consensus.cc:1081] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:12.864606 32657 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:12.865128 32657 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.865454 32657 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.865608 31870 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.870671 32657 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.876462 32657 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:12.878149 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:12.878759 32039 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.879125 32039 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:12.879823 32039 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.880995 32039 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.878649 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:12.882365 31967 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.888195 32039 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:12.889840 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:12.890775   662 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.891223   662 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:12.892012   662 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:12.900187 31967 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:12.946147   577 raft_consensus.cc:993] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:12.946713   577 raft_consensus.cc:1081] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:12.948202 32039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:17:12.948736 32039 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.949074 32039 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.949362 32657 raft_consensus.cc:1217] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:12.967411 32039 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:12.971184 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:12.971885 32117 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.972205 32117 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:12.973352 32117 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:12.978827 32117 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.979859 32039 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:12.990630 32117 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:12.991971 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:12.992547 31967 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.009780 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:13.010689   667 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.011169   667 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:13.011978   667 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:13.016014 31870 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:13.049805 32657 raft_consensus.cc:1217] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:13.041499 31867 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 2, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.024858 31967 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:13.065707   633 raft_consensus.cc:993] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:13.066241   633 raft_consensus.cc:1081] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:13.068977 32117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:17:13.069581 32117 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:13.070073 32117 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.086287 32117 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:13.089916 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:13.090298 32117 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:13.090570 31968 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.096190 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:13.097224 32039 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.097779 32039 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:13.098821 32039 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:13.101188 32039 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.114298 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:13.115736 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:13.118445   650 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.119001   650 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:13.120102   650 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:13.127413 32039 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
W20260709 12:17:14.271446 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080003bae0 after lost signal to thread 30823
I20260709 12:17:14.655797   623 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.656580   623 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.659915   623 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.690125   726 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:14.692376 31901 debug-util.cc:398] Leaking SignalData structure 0x7b0800091880 after lost signal to thread 31904
I20260709 12:17:14.703111   727 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.704068   727 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.703675   726 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.703579   734 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.707362   729 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.707470   734 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.708103   729 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.712417   738 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.713120   738 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.697867   730 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.718843   730 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.721485   730 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.723825   738 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:14.716163   667 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:14.730434   667 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.733021   667 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:14.716765   633 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.734004   633 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.736733   633 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.738268   738 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.738998   738 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.741578   738 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.697701   728 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.727663   730 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.728529   737 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.744590   730 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.744735   737 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.746913   730 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.747666   728 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:14.751403   728 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.718048   739 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.758265   728 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.729234   734 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.758469   739 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.763170   739 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.766275   737 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.774148   735 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.774976   735 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.777793   735 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:14.957706   633 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.958451   667 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.959014   667 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.960729   692 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.961186   667 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:14.961292   692 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.961752   667 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.767310   731 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.789271   740 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.755671   732 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.962553   731 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.965569   735 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.965724   684 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.966187   735 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.702303   623 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.963359   740 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.967305   623 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.968828   740 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.882928   737 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.969305   623 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.969487   737 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.970710   577 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.971153   577 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.971616   737 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.972118   737 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.972537   737 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.972965   577 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:14.973438   577 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.975153   731 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:14.975448   623 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.975999   623 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.977728   740 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.978240   623 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.978327   740 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.962303   667 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:14.795552   726 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.922895   738 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.982717   738 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.984892   738 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.981345   733 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.986394   733 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.929728   662 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.758924   728 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.943295   739 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.963649   732 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.991890   739 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.993848   739 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.994024   747 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.993816   732 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.994700   747 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.948818   650 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:14.995908   650 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:14.714344   729 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:14.957003   730 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.966223   684 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.988076   725 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:14.988512   738 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.988827   737 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.949180   734 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.956365   742 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.963773   633 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.990273   667 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.724238   727 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.916893   711 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:14.997748   650 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.012375 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.023871   577 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:15.046715 32117 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:15.016192 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.002825 31867 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 2 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:15.052716 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:15.008688 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.053411 32043 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
W20260709 12:17:15.016490 31901 debug-util.cc:398] Leaking SignalData structure 0x7b0800100000 after lost signal to thread 31996
I20260709 12:17:15.054531 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:14.999418 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 51614.6us
I20260709 12:17:15.008606   753 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.055820   753 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.043020   732 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.057276   732 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.054402 32040 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:15.058066   753 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.058871 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:15.007615   676 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.035367   761 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.059998   676 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.060309   761 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.056025 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:15.063900 32046 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:15.067981   623 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.176616 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:15.178399 32657 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:15.180032   623 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.181334 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.181912 31967 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:15.182597 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.183650   749 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:15.184168   749 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:14.997268   747 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.185837 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:15.185990   747 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.186473   747 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.187816   739 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:15.188236   764 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.188553   739 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.188843   764 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.191236   739 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.192538   739 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.192585   747 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.193161   747 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.012053   729 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:14.991482   740 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.011408   728 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.193080   739 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.035193   743 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.035022   751 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.034838   744 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.219391 31967 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:15.221221 32046 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:14.963604   692 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.034658   750 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.013234   684 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:14.991525   662 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.224774   731 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.230790   623 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.011197   760 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.232354 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.016752   738 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.011809   730 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.017254   734 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:15.223372 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080031d360 after lost signal to thread 32071
I20260709 12:17:15.043951   711 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.219226   758 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.229431   633 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.244012 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.244608   759 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.245290   749 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.243570   740 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.239375   746 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.250654 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.228169   765 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.228057   757 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.253400   623 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.017189   725 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.222977   729 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.240357   676 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.295011 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.022711   742 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.216447   747 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.244855   760 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.239974   751 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.241132   744 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.243428   750 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.332831 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.486966   623 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.248576   692 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.488828 31967 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:15.493407 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.490587   662 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.493464   755 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.489854   765 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.473135 32118 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:15.248368   731 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.333222   736 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.328709   728 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.543787 32660 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:15.330758   691 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.436170   752 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.452765   735 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.436508   754 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.242523   743 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.333134   767 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.464653   761 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.323204   748 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.550367   760 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.550557   742 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.491829   749 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.489388   739 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.503746   692 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.314267   633 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.323626   726 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.437196   768 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.330921   762 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.514634   732 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.514364   711 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.498348   676 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.514047   738 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.576781   756 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.423259 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:15.584262 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:15.558553 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080033b400 after lost signal to thread 32145
I20260709 12:17:15.591266 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.553454 32115 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:15.506108   740 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.632364   743 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.496779   746 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.634102 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.643523   751 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.633358   726 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.513573   769 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.558293   728 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.584175   744 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.635100   730 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:15.654637 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080033b360 after lost signal to thread 32146
W20260709 12:17:15.676591 32086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.007s	sys 0.009s
I20260709 12:17:15.642011   633 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.568748   755 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.329411   758 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:15.684870 32086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.010s
I20260709 12:17:15.553862   757 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.666440   738 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:15.633404   768 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.655470   676 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:15.726179 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.013s	sys 0.067s
W20260709 12:17:15.746155 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.013s	sys 0.067s
I20260709 12:17:15.581105   691 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:15.726028   742 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:15.663319 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 237656us
I20260709 12:17:15.558035   736 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.725229   711 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.725354   751 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.626994   752 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.716183   738 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.626361   748 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:15.781656   753 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.782001 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.657119   734 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.694134 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.783107 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.787714   650 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.804432   650 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:15.787765 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.806066 31968 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:15.640879   749 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.684331 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.505865   759 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.656006   762 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.664875   756 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.814941   770 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.815151   731 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.820607   746 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.777765   729 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:15.830636 31901 debug-util.cc:398] Leaking SignalData structure 0x7b0800159340 after lost signal to thread 32684
I20260709 12:17:15.830935   650 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.609805   767 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.846727 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.819803   728 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.683470   769 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.831408 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.839600   747 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:15.840085   776 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:17:15.838863   745 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:15.785264   735 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:15.845356   633 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.791392   738 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.842859   732 raft_consensus.cc:993] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:15.784773   742 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.855897   691 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.857935 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.860091   736 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.792321   751 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.816005   753 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:15.863763 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 271473us
I20260709 12:17:15.876639   758 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.877679 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:15.864916 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.074s	sys 0.066s
I20260709 12:17:15.836216   725 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.818658   711 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.846280   692 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.878872 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.648177   754 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.880290 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.881691   711 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:15.882162 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.075s	sys 0.069s
W20260709 12:17:15.883194 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.054s	sys 0.106s
W20260709 12:17:15.883606 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.054s	sys 0.106s
I20260709 12:17:15.886976   577 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.888890   577 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.889544   577 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:15.872016   746 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:15.916885   756 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.887768   729 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:17:15.871619   755 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.919404   752 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.920421 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.920979 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:15.924889 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.925952 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:15.927450   733 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.929235   736 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:15.871446   757 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.940361 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:15.942149 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55006us
I20260709 12:17:15.964159 32046 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:15.888584   731 raft_consensus.cc:1081] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:15.870306   735 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.848150   770 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:15.971904 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46076.3us
W20260709 12:17:15.938575 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52756.6us
I20260709 12:17:15.940441   740 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:15.887948   691 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:15.915512   711 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.963840   676 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.943028   748 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:15.990347 32118 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:15.985601 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.989313 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:15.990825   711 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.991330 32121 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:15.992475 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.022428   749 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.019167   691 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.030798   770 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.958443   746 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:15.943967   762 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:15.939050 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:15.993216 32121 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:15.986901 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:15.942513 32114 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
W20260709 12:17:16.033114 32012 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.026s	sys 0.026s
I20260709 12:17:16.036367 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.874334   745 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.038796   769 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.101074 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.040884 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.048707 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.048833 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:15.993788 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.054437 31971 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:16.055588   623 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:16.056895 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 70102.3us
I20260709 12:17:16.055847 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.071768 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:16.100047 32012 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.027s	sys 0.027s
I20260709 12:17:16.108657 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.046015   742 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.112248   623 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.100296 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.104655 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.104813 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.053682   711 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.122872 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.112077 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.124328 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.112654   623 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.125172   623 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.105580 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.103981 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.051064   759 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.112088 31938 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.055945   754 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.111794   729 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.115913 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.112972   775 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.118885 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.060135   757 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.113711 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.135138   777 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.137117 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.139084 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.140486 32120 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:16.141980 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.144316   745 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.104668   767 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.158463 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.159137 32120 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:16.160068 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.160637 32120 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:16.153743 32117 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:16.151160 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:16.156942 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.003s	sys 0.025s
W20260709 12:17:16.163278 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.003s	sys 0.026s
W20260709 12:17:16.163292 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.066s	sys 0.153s
W20260709 12:17:16.163673 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.066s	sys 0.153s
W20260709 12:17:16.163836 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:16.163729 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 618032us
I20260709 12:17:16.109756 32116 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:16.155817 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:16.118042 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 227198us
I20260709 12:17:16.122924   751 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.166361 32043 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.140132   729 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:16.117926 32113 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:16.167014 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.180753 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.151605   757 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.186609 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.159008   775 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.146833 32654 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.135125   738 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.200165 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.200402 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.203408 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.165700   777 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.216162 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.183523   729 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.170928 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.209674 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.220778 31969 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.226227 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.226857 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.227785 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.228166 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.228627 31969 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.229431 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.213913 32043 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.229822 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.232843 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.217087 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.212045 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.189908   757 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.216605 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.187376 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.197510   735 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.222399   754 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:16.222334 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 172283us
I20260709 12:17:16.225379 31970 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:16.190160   736 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.237134 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.237385   782 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.220094   769 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.251102   765 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.251494   769 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.253237 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.249155   767 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.254261   767 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:16.254788   767 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.191356   730 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.207448 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.221863   777 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.248221 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.265369 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.267863 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.268549 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:16.245568 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 78112us
I20260709 12:17:16.212795   747 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:17:16.253530   769 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.254137 31963 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.241761 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.199532   775 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.258071 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.235255   746 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.261539 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.286386 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.262850 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.264043   691 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.271831 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.272440 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.248670 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:16.275131 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.273212   730 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.242820 31969 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.248986 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.284585 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.278575 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.278759   667 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.270630 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.249455   754 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.292047 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.293111 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:16.295382 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.294108 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.286937 31970 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:16.257341 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:16.307490 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.318166 32040 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.318606 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.278431   781 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.320942 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.303763 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.260159   782 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.307951 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.303218   667 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.458695 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.460692   667 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.456439 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.461251 31964 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:16.461664 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.462097 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.401680 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.462982 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.321045 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.464566 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.465246 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.405967   623 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:17:16.467214 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 174162us
I20260709 12:17:16.404392 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.468902 31972 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:16.304661 32113 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:16.366863 32047 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.305251   691 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.503885 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.320464   751 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.305831   769 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.476929 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.463517 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.320326   759 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.468277 31968 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.504866 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.466423   738 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:16.462422 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.475260   711 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.414016   764 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.415616 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.327170   746 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.507230 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.557013 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.557909 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:16.307574 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 64929.1us
I20260709 12:17:16.509349 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.391945   773 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.456404   729 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.529678 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:16.511545 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 193649us
I20260709 12:17:16.507221 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.365669 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.365041 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.529253 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:16.538973 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.469002   726 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.556277 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:16.366029 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 121950us
I20260709 12:17:16.561363 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.530347 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.532835   778 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.562902 32039 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:16.563804 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.531738   782 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.564069   767 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.564670   768 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.565362 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.565866 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.585398 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.585551 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.477787 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.504876   691 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:16.478127   781 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.508984   751 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.563874   741 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.537942   759 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.533715 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.534693 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.590782 32658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71e0cf115549454c894335e70c43843a"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47544
I20260709 12:17:16.591526 32658 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:16.591872 32658 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.532698   667 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.538213   711 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.581692 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.538808 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.584024 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.437541   777 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.561420 31964 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:16.590771   741 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.600914 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.601378   741 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.582643 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:16.602283 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.510492   769 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.605116 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.592988 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.605472 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:16.604254 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47347.9us
I20260709 12:17:16.594815 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.579300   773 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.576114   785 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.600221 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.588017   691 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.614539 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.615178 32039 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:16.600659   754 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.585251   778 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.585647   782 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:16.601012 32113 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:16.624125 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.623976   782 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.624629 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.625211 32113 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:16.625279 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.625893 32040 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:16.625993   782 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.574241   770 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.565917   746 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.644325   691 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.510444 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.604020 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.606361 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.588320   781 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.590605 32112 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:16.607105   769 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.607601 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.597194   711 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.587904 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.590368 31963 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:16.608632 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:16.608913   742 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.595626   759 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:16.751032 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 142463us
I20260709 12:17:16.751885 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:16.749995 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.609620   777 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.609524 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:16.610574 32656 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.591233   751 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.611707 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.613257 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.613483 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.763800 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.613817   773 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.603861 32044 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:16.616155 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.595361 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1201) took 1004 ms. Trace:
I20260709 12:17:16.623395 32658 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.765991 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.626710   785 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.641482   767 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.635046   778 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.642531 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.607125   738 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.628679   729 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.643097 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.616636   757 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.643965   770 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.715515 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.716444   727 raft_consensus.cc:1064] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:16.717082   768 raft_consensus.cc:1064] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:16.717859   623 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.652709 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:16.710574 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.696231 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.703946 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.645301 31968 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:16.707322 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.700872   577 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.684098 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.692575 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.720503 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.658252 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.663857 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.654590 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.654436 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.739780 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 240) took 3574 ms. Trace:
W20260709 12:17:16.739990 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 133750us
I20260709 12:17:16.628142 32042 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.679754   750 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.680358 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.745637   732 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.664173   775 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.685292 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.628396 32113 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.660480   667 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.632028   747 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
W20260709 12:17:16.744468 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 113739us
I20260709 12:17:16.682860   753 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.750965   758 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.752596   650 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.803495   650 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.803900   650 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.782876 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.805145 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.752945 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:16.806170   753 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.758215 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.803368   758 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.807247   758 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.807552   757 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:16.667611 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.549s (SENT) [suppressed 3 similar messages]
I20260709 12:17:16.718009   691 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.661816 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.667009   746 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.592283   730 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:16.809871 31966 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.769363 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.652470 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.772099 32658 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:16.763536   731 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.818240   781 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:16.820958   727 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:16.821871 32039 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:16.775419 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.778187 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.767413   748 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:16.770294   738 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:16.786360 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:16.787098 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 94606us
I20260709 12:17:16.773177 32652 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 3. Ops: []
I20260709 12:17:16.788697   577 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:16.843102   577 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.843483   577 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:16.828554   731 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.795138 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:16.766371 32118 rpcz_store.cc:276] 0709 12:17:15.590745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.835015 (+244270us) service_pool.cc:224] Handling call
0709 12:17:16.595222 (+760207us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540013a954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:16.595308 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328506112}
I20260709 12:17:16.788184 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:16.797556 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:16.775738 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 170482us
I20260709 12:17:16.799434 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.867304 32040 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:16.868484 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:16.808084 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55210.6us
I20260709 12:17:16.796459 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:16.791402   732 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.870363 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.765985 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:16.871608 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 105689us
I20260709 12:17:16.873212 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.814373 31970 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:16.814301 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:16.817330 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:16.815232 32116 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.823812   772 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.882571   772 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:16.882980   772 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.808120 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:16.827016 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.824635 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.808283 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.794371 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.866488 32046 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:16.875501 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.870875   732 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.880367   790 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.876734   691 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.826267 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:16.859467   731 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.835529   738 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.887703 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:16.784927 31867 rpcz_store.cc:276] 0709 12:17:13.147449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.147609 (+   160us) service_pool.cc:224] Handling call
0709 12:17:14.999625 (+1852016us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1d7f3a 00007f2291f8de9c 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:17:16.721846 (+1722221us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:15.203286 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:15.204160 (+   874us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:15.204209 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:17:15.204236 (+    27us) tablet.cc:662] Decoding operations
0709 12:17:15.204709 (+   473us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:15.204801 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:15.204848 (+    47us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:15.205130 (+   282us) tablet.cc:701] Row locks acquired
0709 12:17:15.205155 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:17:15.205345 (+   190us) write_op.cc:270] Start()
0709 12:17:15.205497 (+   152us) write_op.cc:276] Timestamp: P: 1783599435205292 usec, L: 0
0709 12:17:15.205532 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:17:15.225908 (+ 20376us) log.cc:844] Serialized 535 byte log entry
0709 12:17:16.430666 (+1204758us) op_driver.cc:464] REPLICATION: finished
0709 12:17:16.593214 (+162548us) write_op.cc:301] APPLY: starting
0709 12:17:16.593299 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:16.593404 (+   105us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:16.593526 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:16.593777 (+   251us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:16.593928 (+   151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:16.593983 (+    55us) write_op.cc:312] APPLY: finished
0709 12:17:16.594169 (+   186us) log.cc:844] Serialized 23 byte log entry
0709 12:17:16.594356 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:16.594435 (+    79us) write_op.cc:454] Released schema lock
0709 12:17:16.594473 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:16.600060 (+  5587us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:16.600103 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":4251988864,"child_traces":[["op",{"apply.queue_time_us":162294,"apply.run_cpu_time_us":1589,"apply.run_wall_time_us":3607,"num_ops":1,"prepare.queue_time_us":150264,"prepare.run_cpu_time_us":12766,"prepare.run_wall_time_us":273772,"raft.queue_time_us":427211,"raft.run_cpu_time_us":42572,"raft.run_wall_time_us":185029,"replication_time_us":1224917,"thread_start_us":865610,"threads_started":4,"wal-append.queue_time_us":330447}]]}
W20260709 12:17:16.909240 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 121174us
W20260709 12:17:16.906055 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 135379us
I20260709 12:17:16.889183 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.884143   742 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.892443 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1339) took 1050 ms. Trace:
I20260709 12:17:16.827625 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.902544 32044 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.884228   746 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:16.908097 31965 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:16.911293 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 83741.1us
I20260709 12:17:16.941387 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:16.911069 32118 rpcz_store.cc:276] 0709 12:17:15.841699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.891592 (+1049893us) service_pool.cc:224] Handling call
0709 12:17:16.891956 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.892151 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:16.892188 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:16.892239 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:16.892272 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.892314 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.892398 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:16.913573 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:16.884826 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.917577 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:16.944254 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:16.939460 31971 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.939049   650 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:16.946197 31969 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.946424   650 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.946888   775 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:16.947153 32121 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:16.947405 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.947708   745 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.948072   745 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.948236 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:16.948455   745 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.948621   757 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.951004 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:16.887204   785 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:16.890328   729 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:17:16.956277 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 288656us
I20260709 12:17:17.023721   650 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.044902   650 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.897257 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:16.957126 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:16.918430   691 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:17.048295 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.056392 32658 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.908608   731 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.958402   623 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:17.058454 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.920156   749 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:17.058789 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 101736us
I20260709 12:17:16.966840 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:16.958802 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 69693.1us
I20260709 12:17:16.909111   790 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:16.938685   736 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.045290   650 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:17.070520 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 103785us
I20260709 12:17:17.063810 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.063843 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:16.918643   754 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:17.066299 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.076058 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.077520 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:16.957798   676 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.949075   757 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:17.056005 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:16.903914 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:16.949348 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1026) took 1035 ms. Trace:
I20260709 12:17:16.957858   750 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.958380 32042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:17.060714 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1032) took 1133 ms. Trace:
I20260709 12:17:17.065158   782 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.094288 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.095934 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1197) took 1506 ms. Trace:
I20260709 12:17:17.099565   767 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.957942   730 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.068557   736 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:17.093192   676 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:17.094436   757 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.248157   623 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.069660   790 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.071755   753 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.080492 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.078414   726 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.073499   754 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:17.057561   691 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:17.081229   746 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.239406 32117 rpcz_store.cc:276] 0709 12:17:15.589424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.166383 (+576959us) service_pool.cc:224] Handling call
0709 12:17:16.697015 (+530632us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:16.697272 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.764791 (+ 67519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:16.764819 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:16.764856 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:16.764882 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.095721 (+330839us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b54002b57d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:17.095764 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:17.095878 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162028288}
W20260709 12:17:17.246057 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 151801us
I20260709 12:17:17.273913 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.097262   577 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.275095   577 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:17.252301 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 171915us
I20260709 12:17:17.276760 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.236747 32652 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:17.250563   676 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.237577   750 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:17.278095 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.247195   767 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.282277 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.246373   781 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:17.238924 31965 rpcz_store.cc:276] 0709 12:17:15.922880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.886617 (+963737us) service_pool.cc:224] Handling call
0709 12:17:16.907905 (+ 21288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.941364 (+ 33459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.056152 (+114788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:17.250978   736 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.248129 32042 raft_consensus.cc:606] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.286511 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.273535   691 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.250744   729 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.288326 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.255541   754 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.274621   765 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.288038   623 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.304734   623 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:17.285036   750 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.298826 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:17.272976 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 379671us
I20260709 12:17:17.302088 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.282332 31963 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.305287   623 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.298055 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.283919   760 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.285782   774 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.284005   782 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.292414   789 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.290403   767 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.318853   650 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.342192 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.305593   775 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:17.350786 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 277878us
I20260709 12:17:17.350880 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:17.117827 31964 rpcz_store.cc:276] 0709 12:17:15.913598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.819623 (+906025us) service_pool.cc:224] Handling call
0709 12:17:16.885045 (+ 65422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.895411 (+ 10366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:16.899123 (+  3712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:16.903549 (+  4426us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:16.908440 (+  4891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.915365 (+  6925us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.942409 (+ 27044us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:17.336217   741 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.000s
I20260709 12:17:17.355561 32117 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:17.304579   765 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:17.306221   746 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.360545 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.363240 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:17.367058   770 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:17.371148 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:17.372951 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1335) took 1654 ms. Trace:
I20260709 12:17:17.379457 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:17.384907   768 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:17.405763 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49316.3us
I20260709 12:17:17.336153   749 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.338572   789 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:17.408167 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47694.1us
I20260709 12:17:17.351517   748 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.423880   650 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:17.404359 31938 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.409713 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:17.354681   775 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:17.389074   783 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.337242   732 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.438141   794 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:17.442837 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 123381us
I20260709 12:17:17.417582   623 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.444554 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:17.445247 32119 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:17.426071 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.439245   800 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.444077 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:17.427953 32114 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:17.419431   725 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.401176   765 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.403375 32112 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:17.419237 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72213.3us
I20260709 12:17:17.440085 31965 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.452034 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.452271 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.402477   741 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.452785   650 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.463809   650 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.464176   650 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.422734   789 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.340704   762 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
W20260709 12:17:17.456586 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52355.2us
I20260709 12:17:17.421252   749 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:17.457803 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1349) took 1410 ms. Trace:
I20260709 12:17:17.458084 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:17.466991 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.435245   797 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.446717   794 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:17.471508   623 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:17.459993 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:17.457810   667 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.451128 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.441488   783 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.479415 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.453222 32045 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.468381 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.467052 32120 rpcz_store.cc:276] 0709 12:17:16.047039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.457084 (+1410045us) service_pool.cc:224] Handling call
0709 12:17:17.457368 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.457519 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:17.457543 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:17.457581 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:17.457608 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.457657 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:17.457747 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:17.482009 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1359) took 1251 ms. Trace:
I20260709 12:17:17.472422 32117 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.437211   775 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.466639   782 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.451701   748 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:17.398509 32116 rpcz_store.cc:276] 0709 12:17:15.719075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.602632 (+883557us) service_pool.cc:224] Handling call
0709 12:17:16.608327 (+  5695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.882508 (+274181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.372716 (+490208us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540013a454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:17.372893 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459563776}
I20260709 12:17:17.475538 31972 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.499783   727 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.459187   725 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:17.467348   769 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.706s	user 0.005s	sys 0.005s
I20260709 12:17:17.464113 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.480382   783 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.485579   770 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.482439 32120 rpcz_store.cc:276] 0709 12:17:16.230693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.481306 (+1250613us) service_pool.cc:224] Handling call
0709 12:17:17.481579 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.481732 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:17.481756 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:17.481795 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:17.481823 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.481871 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:17.481960 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:17.468111   749 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.471697 31969 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.474917   797 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:17.598585   797 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.473028   763 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:17.543145 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.476070   794 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:17.488699   778 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.001s
I20260709 12:17:17.486428   577 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.521896   725 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.557605   769 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.561986   741 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:17.475809   774 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:17:17.618769   774 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.619231   774 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:17.620261   774 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.621682   774 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.523113   748 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.568282   760 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:17.630431 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 162125us
I20260709 12:17:17.486059 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.631644 32040 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:17.632263 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 168214us
I20260709 12:17:17.631214 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.637356 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:17.638127 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 169099us
I20260709 12:17:17.624365   732 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:17.639180 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.643383 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.628276   782 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:17.644912 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.617113   778 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:17.652009 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.653856 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:17.654230 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 174874us
I20260709 12:17:17.661442 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:17.661502 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:17.640599   747 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.646221   667 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.643371   727 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.667263 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:17.670130 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.003s	sys 0.024s
W20260709 12:17:17.670477 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.004s	sys 0.024s
W20260709 12:17:17.670965 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 607169us
I20260709 12:17:17.646004   763 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:17:17.673663   667 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.674435   667 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.674602   742 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:17.674809   667 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.675105   742 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.675598 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:17.676402 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:17.651737 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 176543us
I20260709 12:17:17.679483 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:17.682631 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:17.723026 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:17.724375 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.725425 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.651742   782 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.729938 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.675280   667 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.770771 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.771754 32039 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.764744 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1341) took 1896 ms. Trace:
I20260709 12:17:17.671854   795 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.773626 32118 rpcz_store.cc:276] 0709 12:17:15.868175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.956678 (+1088503us) service_pool.cc:224] Handling call
0709 12:17:17.763915 (+807237us) spinlock_profiling.cc:243] Waited 873 us on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:17.764130 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.764336 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:17.764373 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:17.764426 (+    53us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:17.764480 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.764549 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:17.764680 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006784}
I20260709 12:17:17.773450 31965 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:17.763521 31964 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:17.771129   748 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:17.829305 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 184450us
I20260709 12:17:17.735170   732 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:17:17.773636 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:17.833149 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.837814   577 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.849112 32117 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:17.849958 32120 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.855574 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:17.856499   577 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:17.857030   577 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.830636   788 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.862483 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1181) took 1188 ms. Trace:
I20260709 12:17:17.773675   804 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:17.842294   769 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:17.863914 32047 rpcz_store.cc:276] 0709 12:17:16.671731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.807465 (+135734us) service_pool.cc:224] Handling call
0709 12:17:17.855393 (+1047928us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:17.859689 (+  4296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3457920}
I20260709 12:17:17.881531 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.885645 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.886638 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.870115   650 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.891604 32039 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:17:17.899135   731 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.830s	user 0.024s	sys 0.013s
I20260709 12:17:17.899842   650 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:17.902518 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.903358   577 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:17.907516   788 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:17.908336   772 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.025s	user 0.005s	sys 0.006s
I20260709 12:17:17.899557   731 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.908768   772 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.914515 31971 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.914561   772 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:17.915402   772 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.915778   772 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.916149   772 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:17.899685   751 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.135s	user 0.024s	sys 0.002s
W20260709 12:17:17.900378   758 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.093s	user 0.024s	sys 0.015s
I20260709 12:17:17.900923   769 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.880165   804 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.928927 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1157) took 1613 ms. Trace:
I20260709 12:17:17.910216   778 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:17.929441 32040 rpcz_store.cc:276] 0709 12:17:16.315860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.885555 (+569695us) service_pool.cc:224] Handling call
0709 12:17:16.885848 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.632992 (+747144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.928623 (+295631us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540037a354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:17.928854 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714769280}
W20260709 12:17:17.937028   773 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.166s	user 0.014s	sys 0.007s
I20260709 12:17:17.937449   773 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.939126 32659 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.939868   650 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.924666   725 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:17.942389   738 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.000s	user 0.013s	sys 0.013s
I20260709 12:17:17.938701 32041 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:17.908214   788 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.947877 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:17:17.933725   753 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.678s	user 0.004s	sys 0.007s
I20260709 12:17:17.924650   751 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.942757   738 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:17.926890   758 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:17.940363   731 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:17.956825 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 226982us
W20260709 12:17:17.958559 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 125544us
I20260709 12:17:17.959235 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.945259   782 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:17.961750 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:17.963459 32113 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:17:17.972784 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 503505us
I20260709 12:17:17.952065   753 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:17.973835   711 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.212s	user 0.006s	sys 0.019s
I20260709 12:17:17.985332 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:17.976981 32655 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.994518   650 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.977847   748 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:17.995034   650 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.995505   650 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:17.996662 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48876.2us
I20260709 12:17:17.989850 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.012558 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1173) took 1473 ms. Trace:
I20260709 12:17:18.012495 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1336) took 2085 ms. Trace:
I20260709 12:17:18.015429 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1363) took 1704 ms. Trace:
I20260709 12:17:18.019095 32120 rpcz_store.cc:276] 0709 12:17:16.303804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.574926 (+1271122us) service_pool.cc:224] Handling call
0709 12:17:17.849730 (+274804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.855189 (+  5459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.008275 (+153086us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540043add0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.008473 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631060736}
I20260709 12:17:18.020682 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:18.020835 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.018213 31972 rpcz_store.cc:276] 0709 12:17:15.912314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.632845 (+720531us) service_pool.cc:224] Handling call
0709 12:17:16.652150 (+ 19305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.511866 (+859716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.997162 (+485296us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54001ea0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:17.997402 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893045120}
I20260709 12:17:17.988938 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 50) took 1081 ms. Trace:
W20260709 12:17:18.011835   759 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.260s	user 0.009s	sys 0.005s
I20260709 12:17:18.016664   667 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.028573   759 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.031478 32660 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.032326 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.033761 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:17.993850   725 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.023394 32045 rpcz_store.cc:276] 0709 12:17:16.529411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.902724 (+373313us) service_pool.cc:224] Handling call
0709 12:17:17.451085 (+548361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.480186 (+ 29101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.002508 (+522322us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54001950d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.002698 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1260471552}
I20260709 12:17:18.037155 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.035554 32039 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:18.026929 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.038286 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.035058   804 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.043350   667 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.044065   763 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:18.047204 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.016462   748 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:18.009429   807 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:18.070125 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.007123   711 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:18.073442   789 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.005s
I20260709 12:17:18.077342   650 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.039175   731 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.027832   773 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:18.070734 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:18.074954   750 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.011s
I20260709 12:17:18.052116   738 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:18.104010   757 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.831s	user 0.001s	sys 0.009s
I20260709 12:17:18.050607   770 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 12:17:18.163259 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85431.1us
I20260709 12:17:18.043386   809 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:18.178751 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:18.180711   736 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.874s	user 0.018s	sys 0.014s
I20260709 12:17:18.181114   738 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:18.181557   738 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:18.181281   736 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.182053   738 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.184126   810 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.203s
I20260709 12:17:18.186461   810 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.186782   810 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:18.187180   691 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.876s	user 0.013s	sys 0.017s
I20260709 12:17:18.038818   768 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.169019 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:18.028358 32113 rpcz_store.cc:276] 0709 12:17:16.907565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.962850 (+1055285us) service_pool.cc:224] Handling call
0709 12:17:17.963207 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:17.977058 (+ 13851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.988883 (+ 11825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.117856   751 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.171368 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.172653 32118 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.173396   577 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.202836   577 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.203202 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.176630   650 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:18.180776   775 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.666s	user 0.004s	sys 0.012s
W20260709 12:17:18.038852 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49120.2us
I20260709 12:17:18.050686   795 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:18.183557 32116 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.184495   770 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.053119   753 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.190208 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:18.214458 32658 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:18.187572   691 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.207340   807 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:18.215536   783 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.614s	user 0.000s	sys 0.010s
I20260709 12:17:18.215678 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.216013   783 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.216482   691 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.194550   802 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:17:18.194595 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:18.217792 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1359) took 1998 ms. Trace:
I20260709 12:17:18.218945   577 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.118258   789 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.217854   748 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.220917   789 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.204123   667 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:18.198135   742 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.005s
I20260709 12:17:18.222196 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.218986   738 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:18.222837   797 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.014s
I20260709 12:17:18.222752   742 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.223308   797 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.224450 32046 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.224697   742 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:18.172266 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 410066us
W20260709 12:17:18.226043   745 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.277s	user 0.013s	sys 0.022s
I20260709 12:17:18.226500   745 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.228678   745 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.229038 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.229316 32117 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.217690   802 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.230191 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.230533   802 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:18.218396 31971 rpcz_store.cc:276] 0709 12:17:16.221558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.956683 (+735125us) service_pool.cc:224] Handling call
0709 12:17:17.065463 (+108780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.915126 (+849663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.217574 (+302448us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000b6b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.217733 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952194432}
W20260709 12:17:18.201359 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 159658us
I20260709 12:17:18.208935 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.209285 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.209685 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.206254 31938 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.177028   750 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.207242   795 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:18.210659   775 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.211265   736 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.185006   757 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.236493   813 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:17:18.210692 32654 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:17:18.211772   790 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.938s	user 0.014s	sys 0.018s
W20260709 12:17:18.212499   749 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.597s	user 0.019s	sys 0.000s
I20260709 12:17:18.255210   759 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.245740 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 330) took 1488 ms. Trace:
I20260709 12:17:18.255153   749 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.213145 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.255903   750 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:18.256623   794 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.013s
I20260709 12:17:18.256971 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.257017   794 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:18.257639   749 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.258370 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.259238   794 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.259411 32113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:18.259910 32047 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.259967 32113 raft_consensus.cc:606] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:18.214126 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.261524 32120 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.262857 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1218) took 1312 ms. Trace:
I20260709 12:17:18.214130   753 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.216816 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.242086 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.262046 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1362) took 1965 ms. Trace:
I20260709 12:17:18.261974 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:18.187588   810 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.270007 32653 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:18.269524 32116 rpcz_store.cc:276] 0709 12:17:16.297092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.511177 (+1214085us) service_pool.cc:224] Handling call
0709 12:17:17.511527 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.209877 (+698350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.247356 (+ 37479us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400139a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.250946 (+  3590us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545068800}
I20260709 12:17:18.270394   810 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.266757 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.250205   738 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.271718 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1219) took 1320 ms. Trace:
W20260709 12:17:18.267505   676 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.974s	user 0.022s	sys 0.010s
I20260709 12:17:18.272096 32116 rpcz_store.cc:276] 0709 12:17:16.950983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.270911 (+1319928us) service_pool.cc:224] Handling call
0709 12:17:18.271227 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.271393 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.271422 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.271473 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.271505 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.271561 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.271663 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.272506   676 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.274189 32119 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.230525 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1374) took 1589 ms. Trace:
I20260709 12:17:18.210162   770 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.235194 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:18.275522 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:18.193485 32121 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:18.236620   745 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.279681   676 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.279452   770 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.280506   757 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.280616 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1358) took 2098 ms. Trace:
I20260709 12:17:18.280856 32113 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.281031   770 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.281503 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1223) took 1329 ms. Trace:
I20260709 12:17:18.281857 32113 rpcz_store.cc:276] 0709 12:17:16.951959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.280386 (+1328427us) service_pool.cc:224] Handling call
0709 12:17:18.280677 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.281272 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.281453 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.282330   775 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.283231 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1213) took 1669 ms. Trace:
I20260709 12:17:18.283360 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1224) took 1330 ms. Trace:
I20260709 12:17:18.283581 32118 rpcz_store.cc:276] 0709 12:17:16.613538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.797671 (+1184133us) service_pool.cc:224] Handling call
0709 12:17:17.830837 (+ 33166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.202713 (+371876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.283044 (+ 80331us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b540012b6d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.283181 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785633664}
I20260709 12:17:18.262497   802 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.284904   802 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.239117 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:18.277411 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.285808 32658 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:18.203750   788 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.246629 31968 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.248600 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.251665   730 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.244652 32115 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.252560 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1350) took 2212 ms. Trace:
I20260709 12:17:18.263554   797 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.240692   795 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.253895   741 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.243618 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:18.254375 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1340) took 2391 ms. Trace:
I20260709 12:17:18.244539 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1230) took 1290 ms. Trace:
I20260709 12:17:18.255066   790 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.218166   783 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.241565 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.263311 32120 rpcz_store.cc:276] 0709 12:17:16.950704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.022250 (+1071546us) service_pool.cc:224] Handling call
0709 12:17:18.025441 (+  3191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.261971 (+236530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.262620 (+   649us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54002f9254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.262797 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542386944}
W20260709 12:17:18.300441   765 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.014s
I20260709 12:17:18.265120 32047 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.265705 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.213104 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.267979 32040 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.207456   758 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.236546   691 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.275349 32117 rpcz_store.cc:276] 0709 12:17:16.641572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.857117 (+1215545us) service_pool.cc:224] Handling call
0709 12:17:17.857446 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.229704 (+372258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.230283 (+   579us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54001482d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.230453 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854796672}
I20260709 12:17:18.185232   772 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.278123   751 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.317405   765 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.317957   808 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.278651 32121 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.283732 32113 rpcz_store.cc:276] 0709 12:17:16.952505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.282630 (+1330125us) service_pool.cc:224] Handling call
0709 12:17:18.282907 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.283073 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.283098 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.283135 (+    37us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.283168 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.283223 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.283314 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.278451 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1220) took 1327 ms. Trace:
I20260709 12:17:18.286739 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:18.241851 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.188448   773 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:18.285486 32041 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1115) took 1694 ms (client timeout 2190 ms). Trace:
W20260709 12:17:18.299182 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 64040.2us
W20260709 12:17:18.292335   767 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.902s	user 0.009s	sys 0.018s
I20260709 12:17:18.253364 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:18.332710   772 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.333101 32044 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:18.301113   650 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:18.333658   650 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.333308   773 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.334122   650 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.335640   773 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.323972   711 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.337119 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1231) took 1382 ms. Trace:
I20260709 12:17:18.285950   749 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:18.304919 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1352) took 2162 ms. Trace:
I20260709 12:17:18.308465 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1367) took 1796 ms. Trace:
I20260709 12:17:18.309973 32120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:18.264258 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.310869 32039 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:18.300153   726 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:18.314756   811 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
W20260709 12:17:18.319079   814 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:17:18.305212   741 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:18.284996 32118 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.320104 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1222) took 1368 ms. Trace:
I20260709 12:17:18.324836 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.326512 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:18.363483 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1225) took 1410 ms. Trace:
I20260709 12:17:18.256003 32655 rpcz_store.cc:276] 0709 12:17:16.757406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.806017 (+ 48611us) service_pool.cc:224] Handling call
0709 12:17:16.806330 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.010071 (+1203741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.245471 (+235400us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003bae54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.245651 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684443520}
I20260709 12:17:18.323864 32116 rpcz_store.cc:276] 0709 12:17:16.951248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.273883 (+1322635us) service_pool.cc:224] Handling call
0709 12:17:18.277931 (+  4048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.278121 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.278150 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.278192 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:18.278235 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.278289 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.278402 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.358727   772 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.309032 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.307793 32114 rpcz_store.cc:276] 0709 12:17:16.954271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.022828 (+ 68557us) service_pool.cc:224] Handling call
0709 12:17:18.244327 (+1221499us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400139054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.244464 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245859072}
I20260709 12:17:18.367385 32660 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:18.328599 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.363273 32121 rpcz_store.cc:276] 0709 12:17:16.951721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.276536 (+1324815us) service_pool.cc:224] Handling call
0709 12:17:18.278476 (+  1940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.319840 (+ 41364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.320051 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.326683 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:18.329844 32040 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:18.327867   758 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:18.333756 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.330880   794 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:17:18.305581   577 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.307677 32011 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
W20260709 12:17:18.353737 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89561.8us
I20260709 12:17:18.351379 32120 raft_consensus.cc:606] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.382876 32011 leader_election.cc:403] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:18.351133 31968 rpcz_store.cc:276] 0709 12:17:16.536131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.113623 (+577492us) service_pool.cc:224] Handling call
0709 12:17:17.113940 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.294343 (+1180403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.308267 (+ 13924us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001e82d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.308409 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604436736}
I20260709 12:17:18.307968   667 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:17:18.323899 32651 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:18.329885 32041 rpcz_store.cc:269] 0709 12:17:16.590994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.826257 (+235263us) service_pool.cc:224] Handling call
0709 12:17:18.285285 (+1459028us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540003a054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.285422 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967435008}
I20260709 12:17:18.281064 31969 rpcz_store.cc:276] 0709 12:17:16.182242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.947913 (+765671us) service_pool.cc:224] Handling call
0709 12:17:16.948181 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.598179 (+649998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.280393 (+682214us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54000799d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.280561 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183141760}
I20260709 12:17:18.351141 32119 rpcz_store.cc:276] 0709 12:17:16.142964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.460056 (+1317092us) service_pool.cc:224] Handling call
0709 12:17:17.460299 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.274605 (+814306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.304753 (+ 30148us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54000807d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.304872 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1871354624}
I20260709 12:17:18.360911   799 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:17:18.389637 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.360976   650 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.363950 32118 rpcz_store.cc:276] 0709 12:17:16.952797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.284621 (+1331824us) service_pool.cc:224] Handling call
0709 12:17:18.284851 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.363232 (+ 78381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.363424 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.369163   711 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.330256   767 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.300578 31963 rpcz_store.cc:276] 0709 12:17:16.040227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.906871 (+866644us) service_pool.cc:224] Handling call
0709 12:17:17.114206 (+207335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.320995 (+206789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.252371 (+931376us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b5400313b50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.252502 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453647232}
W20260709 12:17:18.392287 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 63787.5us
I20260709 12:17:18.367024   790 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.371131   730 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:18.337836 32113 rpcz_store.cc:276] 0709 12:17:16.954554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.336134 (+1381580us) service_pool.cc:224] Handling call
0709 12:17:18.336437 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.336622 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.336661 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.336711 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.336758 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.336814 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.337019 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.403121   787 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.403350   730 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:18.403481   797 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:18.403855 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.404158 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.404896 31967 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:18.404637   730 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.405349 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.405920   789 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.400166   802 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:17:18.406636 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.407416   802 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.407526   789 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.400990 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.407816   802 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:18.407931   789 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.408139 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.408488 31964 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:18.370899   769 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:18.366001   741 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.418452   769 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:18.377319   751 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:18.419284   751 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:18.419701   751 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.419726 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:18.420434 32657 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.420889   751 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:18.410400   754 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.094s	user 0.019s	sys 0.014s
I20260709 12:17:18.406028 32113 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.423415   754 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.424661   753 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.425577   754 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.379727   758 raft_consensus.cc:993] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:18.380056   765 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:18.352995   726 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.435415   726 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.436643 32657 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:18.436941 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1228) took 1483 ms. Trace:
W20260709 12:17:18.384863 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81807.6us
I20260709 12:17:18.377385   798 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:17:18.390836   788 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.390902 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:18.318886   817 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.163s
I20260709 12:17:18.393040 31938 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.395314 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.395969 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.397879 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.397957 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.397967 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.399325 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.400071 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.306583 32112 rpcz_store.cc:276] 0709 12:17:15.862718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.901391 (+1038673us) service_pool.cc:224] Handling call
0709 12:17:17.394084 (+492693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.465876 (+ 71792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.254140 (+788264us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54000802d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.254284 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123222272}
I20260709 12:17:18.401798 31965 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:18.449838   787 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.402261   775 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.402475 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.402691 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.404738   797 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.338129   749 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.452386 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.452877 32657 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:18.411273   650 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.412501 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.331619   781 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:18.409451 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.416676 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.417598 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.426527 32045 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.432672 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.418860   769 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:18.433459 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.434584 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.437817 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:18.437384 32117 rpcz_store.cc:276] 0709 12:17:16.953797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.316423 (+1362626us) service_pool.cc:224] Handling call
0709 12:17:18.316750 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.384406 (+ 67656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.384426 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.384452 (+    26us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:18.384470 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.436708 (+ 52238us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b540010ead4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:18.436754 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.436882 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155192704}
I20260709 12:17:18.439791 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:18.440856 32119 raft_consensus.cc:2368] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:18.481341 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.408805   789 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.444325 31970 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:17:18.444988 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52029.4us
I20260709 12:17:18.446077 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.446128 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.446658 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.447680 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:18.484935 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50415.6us
I20260709 12:17:18.448752   796 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:17:18.449234   768 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.449277 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:18.427423   751 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.440222   691 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.454231 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.455927 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.440618   742 raft_consensus.cc:1081] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:17:18.457612 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:18.452409   797 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:18.433902   805 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:17:18.426244   753 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:18.486362 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48621.9us
I20260709 12:17:18.482820 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.488808 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.489998 32047 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.492386 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.510562 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.512012 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.512637   805 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:18.512995   805 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:18.513955 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:18.495196 32013 leader_election.cc:400] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:18.515791 32013 leader_election.cc:403] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:18.502033 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.462002   749 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.503779 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1233) took 1548 ms. Trace:
I20260709 12:17:18.489343   745 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.517993 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.517879 32119 rpcz_store.cc:276] 0709 12:17:16.955008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.502989 (+1547981us) service_pool.cc:224] Handling call
0709 12:17:18.503255 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.503433 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.503463 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.503505 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:18.503537 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.503596 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.503726 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.520107   810 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:18.522272   676 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.531348   810 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:18.521061   787 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.532286   758 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.492569   768 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:18.532693   758 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.533396   768 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.533576   758 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.521760   799 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:18.532316   810 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.508720 31966 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:18.529232   650 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.528836 32119 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.530628 32039 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:18.492161   736 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:17:18.540298 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.513765   805 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.541348 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:18.537103 32112 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:18.556448 32043 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:18.524884   754 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.524963   774 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.552114 32119 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.522588   778 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:18.547369 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.506623   727 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:17:18.565280 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.566708 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.576355 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.576884   805 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.563515 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.577477 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.567278 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.579257   650 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.567572   758 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.568615 31966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:17:18.581764 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.582252 31966 raft_consensus.cc:606] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:18.583547 31966 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.569479   789 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.571990   775 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:18.545683 32116 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.546072   745 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:18.580480 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.567204 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.506090   797 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.562835   768 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.593784 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.595201 31968 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:18.595911 31967 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:18.581048 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:18.597486 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.624109 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.623721 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:18.605965 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.623248 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:18.640197 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 124727us
I20260709 12:17:18.561241   796 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:17:18.506213   807 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
W20260709 12:17:18.649622 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.528867   769 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.632357   788 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:18.649010   770 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.597607   811 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
W20260709 12:17:18.666855 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85885.9us
I20260709 12:17:18.675668 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.677335   650 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:18.678763   815 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.668507 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1235) took 1604 ms. Trace:
I20260709 12:17:18.679553 32116 rpcz_store.cc:276] 0709 12:17:17.062333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.494012 (+1431679us) service_pool.cc:224] Handling call
0709 12:17:18.526247 (+ 32235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.631973 (+105726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.667098 (+ 35125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.680799 32116 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.681877 31970 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:18.669448   775 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.687109 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.688442 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.689240 31964 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:18.690941 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.691493 31971 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.693308 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:18.698093 32120 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:18.592159   806 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:18.700402 32047 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:18.677843 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.641682   745 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.690127 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:18.724558   650 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:18.724704 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47065.8us
I20260709 12:17:18.725526 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.726989 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:18.728274   808 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.728709   806 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:18.727667   785 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.727308 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:18.729935   806 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.731446 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.732054   822 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:17:18.732791   820 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.733275 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.733755 31970 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.670137   809 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:17:18.702651 31966 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.750535   820 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.750072 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:18.778177 32047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:18.779042 32047 raft_consensus.cc:606] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:18.782853 32047 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:17:18.783937 32047 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:18.784466 32047 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.788951   757 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:18.789635   757 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.794241 32047 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.772282 31964 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:18.800843   667 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.803548 32047 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.813344 32047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f4287254496472398f69d3a10b48d77"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:18.814306 32047 raft_consensus.cc:606] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.816205 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:18.818202 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:18.819573 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:18.794862   731 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.809211 32116 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:18.770381 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:18.820955   795 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.003s
I20260709 12:17:18.811939   667 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:18.825173 32113 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:18.830446   731 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:18.858839   790 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.865552 32117 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.782997   745 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.867197 32121 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:17:18.870292 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 334751us
I20260709 12:17:18.850667   726 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:18.864648   767 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:18.861864   757 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:18.854159   795 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:18.866472 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103033us
I20260709 12:17:18.883104 32040 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:18.876788 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1356) took 2673 ms. Trace:
I20260709 12:17:18.887420 31972 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:17:18.891003 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 220152us
I20260709 12:17:18.895483 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:18.892102 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:18.892038 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 262472us
I20260709 12:17:19.071400 32651 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:18.893479   795 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.072671   827 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:19.085139   754 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.000s
I20260709 12:17:19.085604   754 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:18.887972 32115 rpcz_store.cc:276] 0709 12:17:16.201785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.463715 (+1261930us) service_pool.cc:224] Handling call
0709 12:17:18.244447 (+780732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.300460 (+ 56013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.874966 (+574506us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400139050. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:18.875133 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697623296}
I20260709 12:17:18.882835   667 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.082412   820 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:18.886854   731 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:18.897043   828 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.244s
I20260709 12:17:18.893250   826 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.251s
W20260709 12:17:19.172834 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:19.172866 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:19.173570 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.179196   726 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.179308   827 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.180053   826 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.169529   810 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:17:19.180475   826 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.183161 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.184028 32121 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:19.073464 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 173502us
I20260709 12:17:19.185268 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:19.186834 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.114853   790 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:17:19.143404   757 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:17:19.155531   754 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:19.193975 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.195042 32121 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:19.187330 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.169073   820 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:19.197781 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 296433us
I20260709 12:17:19.200392 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.193735   731 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.206393 32116 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.199101 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.234026 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:19.239580   796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.047s
W20260709 12:17:19.244287   796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.000s	sys 0.047s
I20260709 12:17:19.277633   788 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:17:19.303172   830 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:19.303709 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1236) took 1988 ms. Trace:
I20260709 12:17:19.303927   830 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.304286 32117 rpcz_store.cc:276] 0709 12:17:17.314831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.500425 (+1185594us) service_pool.cc:224] Handling call
0709 12:17:18.865377 (+364952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.865995 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.303397 (+437402us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b540010efd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:19.303609 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784059264}
I20260709 12:17:19.307158   796 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.306492 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.326875 32117 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:19.206784   826 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:19.243830   758 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 12:17:19.244722 32655 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.246722 32041 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.269021 32115 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:19.478092 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1197) took 1495 ms. Trace:
I20260709 12:17:19.482736 32045 rpcz_store.cc:276] 0709 12:17:17.983154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.169807 (+186653us) service_pool.cc:224] Handling call
0709 12:17:18.202642 (+ 32835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.458492 (+255850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.477847 (+1019355us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400216954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:19.478029 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514439040}
I20260709 12:17:19.249234   769 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.480314 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.488935 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 64) took 1035 ms. Trace:
I20260709 12:17:19.489326 32040 rpcz_store.cc:276] 0709 12:17:18.453180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.486545 (+ 33365us) service_pool.cc:224] Handling call
0709 12:17:18.882782 (+396237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.884413 (+  1631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.488769 (+604356us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b5400033ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:19.488884 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910489472}
I20260709 12:17:19.268707   711 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:19.325613   753 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.803s	user 0.010s	sys 0.000s
I20260709 12:17:19.495050   823 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.499289 32114 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:19.499442   817 raft_consensus.cc:993] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:19.499284   823 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.500169   817 raft_consensus.cc:1081] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:19.500799 32114 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.501566   823 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:19.508769 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.509780 32117 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:19.495167   816 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.502092 32114 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.503358   830 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.512358 32119 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:19.507331   826 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:19.516510   806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.001s	sys 0.014s
I20260709 12:17:19.511034   816 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:19.498162   711 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.524114   820 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.524500   748 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.524956   748 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:19.526348   748 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.527509   748 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.523357   790 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:17:19.530941   781 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.531342   781 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.531509   816 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.532100   781 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.532862   816 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.533063   781 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:19.521485   806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.002s	sys 0.015s
I20260709 12:17:19.575158 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:19.578961 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 378513us
I20260709 12:17:19.590902 32660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:19.591322   769 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:19.592712   769 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.595402   711 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:19.597831 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:19.601522   805 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.602403   769 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:19.603868 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:19.605367   843 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:19.605976 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:19.605971 32660 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:19.607132 32660 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.606405   843 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:19.519016 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 312855us
I20260709 12:17:19.610704 32038 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.610994 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:19.611673 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1158) took 1071 ms. Trace:
I20260709 12:17:19.612169 32044 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:19.614850 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.621325 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.614919   810 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.622646 31964 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:19.617277 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.623091 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:19.624902 31972 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1070) took 1415 ms (client timeout 1599 ms). Trace:
I20260709 12:17:19.625168 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:19.625381 31972 rpcz_store.cc:269] 0709 12:17:18.209597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.212797 (+  3200us) service_pool.cc:224] Handling call
0709 12:17:19.624490 (+1411693us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400313b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:19.624722 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323323264}
I20260709 12:17:19.625885 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1157) took 1086 ms. Trace:
I20260709 12:17:19.626509 32116 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:19.626430 32039 rpcz_store.cc:276] 0709 12:17:18.539647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.540129 (+   482us) service_pool.cc:224] Handling call
0709 12:17:19.603385 (+1063256us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:19.625798 (+ 22413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3357056}
I20260709 12:17:19.627525 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:19.628746 31969 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:19.630524 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:19.601640   757 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:19.634445   757 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.634819 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:19.636978 31963 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:19.637503   757 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.640772 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1382) took 1255 ms. Trace:
I20260709 12:17:19.641220 32120 rpcz_store.cc:276] 0709 12:17:18.385578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.416436 (+ 30858us) service_pool.cc:224] Handling call
0709 12:17:19.640319 (+1223883us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54002fa654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:19.640535 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438123776}
I20260709 12:17:19.602092   805 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:19.618281 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.618346 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.643543 31967 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:19.642227 32118 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.645699 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:19.646778   803 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:19.647696 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.648667   789 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.649041   781 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.649550   781 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.648648   803 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:19.646567 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 56) took 1264 ms. Trace:
I20260709 12:17:19.650615   803 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:19.651238 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.651085   781 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.653800   781 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.615450   731 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:19.612236 32038 rpcz_store.cc:276] 0709 12:17:18.539978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.542034 (+  2056us) service_pool.cc:224] Handling call
0709 12:17:19.609789 (+1067755us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:19.610189 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.611323 (+  1134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.611552 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26893568}
I20260709 12:17:19.651100   803 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.658852   803 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.659538 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:19.660169 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.660635 31967 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:19.660552 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:19.656155   826 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:19.662139   826 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.662835 31967 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.664237 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.664784   826 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.665840 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:19.656675 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43727.7us
I20260709 12:17:19.667001   802 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.668506 32112 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.667039 32120 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:19.670035   803 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.670442   797 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.672502   816 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:19.673144 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:19.673287   816 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.675307 32112 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.675855   816 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.677767 32046 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:19.678349 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 75) took 1053 ms. Trace:
I20260709 12:17:19.678692 32046 rpcz_store.cc:276] 0709 12:17:18.624632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.635223 (+ 10591us) service_pool.cc:224] Handling call
0709 12:17:19.605529 (+970306us) spinlock_profiling.cc:243] Waited 6.56 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:19.678247 (+ 72718us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15092992}
I20260709 12:17:19.680456 32660 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.580624 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.683670 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:19.684405 31972 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.654008   732 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:19.689781   818 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.690061   732 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:19.690459   818 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.690685   807 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.691586   732 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:19.693254 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 112920us
I20260709 12:17:19.694377 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.695791   837 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.695873 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:19.696285   837 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:19.697005 32655 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:19.693341   822 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.692101   843 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.650429 32118 rpcz_store.cc:276] 0709 12:17:18.381699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.569981 (+188282us) service_pool.cc:224] Handling call
0709 12:17:19.641742 (+1071761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.644531 (+  2789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.646331 (+  1800us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002fa650. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:19.646478 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2463909632}
I20260709 12:17:19.697688   837 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.691870   818 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.707835   818 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.704273   809 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:19.704978   831 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.699133   822 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:19.648638 32113 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.693250   732 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:19.708292   730 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.726761   730 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.727753   831 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:19.727921   833 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.728444   833 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:19.728089   730 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.711814   822 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.714421 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.715606 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.716234 32660 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:19.718712 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.731907 31968 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:19.719137 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 55) took 1339 ms. Trace:
I20260709 12:17:19.649551   789 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.712514 32113 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:19.723266 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.689790   802 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.728423   831 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.727579   726 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:17:19.725710 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 80260.1us
I20260709 12:17:19.733354 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.732275 32119 rpcz_store.cc:276] 0709 12:17:18.380111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.566548 (+186437us) service_pool.cc:224] Handling call
0709 12:17:18.566840 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:19.516116 (+949276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.718559 (+202443us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54004344d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:19.718954 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173666560}
I20260709 12:17:19.730532   730 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.741204   760 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.741729   760 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.743268 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.744046   816 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.743990   760 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:19.745324   816 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.735245   726 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:19.747292   726 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.751845 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:19.752045 31965 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.755944   667 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.743482   741 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.748188 32044 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.754151 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1377) took 1186 ms. Trace:
I20260709 12:17:19.759740   822 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.735634   837 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.769608 31972 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:19.729997   833 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.763238 32113 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.756772   831 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.764966 31965 rpcz_store.cc:276] 0709 12:17:18.573796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.576584 (+  2788us) service_pool.cc:224] Handling call
0709 12:17:18.576935 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.752595 (+1175660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.753802 (+  1207us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400068e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:19.754067 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701467904}
I20260709 12:17:19.775655 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:19.767117   726 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.782085 32660 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:19.775347   833 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.790819   811 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.791170 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:19.791467   811 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:19.792645 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.776996   789 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:19.797842 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:19.797775   795 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:19.798297   806 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.799233   768 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.799072   806 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.799639   768 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:19.769656 31969 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:19.800706   768 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.803673   768 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.804009   806 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:19.806699 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.801103 31965 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:19.805826 32113 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.812518   843 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:19.816819 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 51506us
I20260709 12:17:19.818590 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:19.818123 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:19.792785   811 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.827801   811 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.821106 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:19.820631   691 raft_consensus.cc:2764] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:19.828969 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:19.849752 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83183us
I20260709 12:17:19.826860 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:19.852635 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 144) took 1345 ms. Trace:
I20260709 12:17:19.853171 31867 rpcz_store.cc:276] 0709 12:17:18.506657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.506793 (+   136us) service_pool.cc:224] Handling call
0709 12:17:19.852382 (+1345589us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7,"reported_tablets":3}
I20260709 12:17:19.854614   767 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:19.855073   767 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:19.855432   767 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:19.863023 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.871856   799 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.872659   799 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.874629 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:19.877871   816 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:19.880393   816 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.880735   816 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:19.883478   845 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.883867   789 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.884083   845 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.884536   789 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.881680   816 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.885824   816 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.888398 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:19.889223 32121 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:19.890906 32121 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.890975 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:19.891702 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.894255 32121 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:19.818569   805 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.895632   805 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.891619 32660 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:19.900859 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
W20260709 12:17:19.903105 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 84577.8us
I20260709 12:17:19.850606 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.905208 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.915930 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:19.915478 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:19.919543   691 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.921207   806 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:19.921367   691 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.922811 32652 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:17:19.923228 32652 pending_rounds.cc:85] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 4
I20260709 12:17:19.913383 31867 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:19.924681 32652 pending_rounds.cc:107] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:19.926482   691 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:19.926764 32652 raft_consensus.cc:2889] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:19.924947 32040 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:19.928099   790 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.930167   789 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:19.902372   799 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:19.932313 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 105644us
I20260709 12:17:19.934046 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:19.934263   845 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.923486 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:19.936311   789 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:19.936740   789 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:19.937110   789 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.937706   806 raft_consensus.cc:993] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:19.938277   806 raft_consensus.cc:1081] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:19.939484 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:19.940169 32655 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:19.940963 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.942335 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.943653 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:19.944288   845 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.945004   845 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:19.936419   775 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:19.946015 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:19.951157 32121 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.952536   837 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:19.952877 32659 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:19.953217   837 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.954561   802 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:19.957255   767 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.958279 32655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47552
W20260709 12:17:19.958621 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 161941us
W20260709 12:17:19.960572 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:19.967767 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 159609us
I20260709 12:17:19.971673   802 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:19.975608 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.932993   803 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.975841 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:19.981498 32655 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:19.989367   845 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.989720 32655 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.989960   845 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:19.991142 32044 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:19.992097   845 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:19.992674 32039 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:19.992962 32118 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.993693 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:19.993920 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:19.994659 32040 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:19.971187   767 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:19.994381 32045 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:19.981379   802 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.007370   691 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.008667 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.009248   803 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.010425   803 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.011391 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.012408   760 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.013113   760 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:20.019861 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 76369.9us
I20260709 12:17:20.022884 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:19.995226   781 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.980929   789 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:19.974223   799 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:17:20.032783 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.036484 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.037792   787 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:20.040043 32623 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:20.034469 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:20.049710 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 252892us
I20260709 12:17:20.040378 31964 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:20.051118 32040 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:20.051942 32652 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.040421 32623 leader_election.cc:403] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:20.057873 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:20.062414 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.049263   806 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.066951 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.047560   822 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.071362 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.078172   783 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:20.080658 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:20.085139   765 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.085563   765 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.085937   765 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.082690   783 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.089049   783 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.090456   824 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:17:20.091724 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.095767   837 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.096815 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.097539 32118 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:20.069571   767 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:20.073310   822 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:20.107460 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.103979 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:20.111267 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.070477   781 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 12:17:20.111706 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.116369 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.117304 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.118053 32119 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.106169   760 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:17:20.120268   760 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.120663   760 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:20.123543 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.124267 32112 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.106995   822 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.106555   799 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.121665   760 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:20.129472 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.110884   768 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.143092 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:20.144645 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 77380.3us
I20260709 12:17:20.144753 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.145658 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.146595   741 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.147084 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.147974   741 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.148780 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.149645   741 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.153081   843 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.154198   843 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.110617   837 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:17:20.154333 32657 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.155050   837 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.155416 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.155710   837 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.156065   837 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:20.147102 32653 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.156955   837 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.158438   770 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.109172   767 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:20.161711 32652 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:20.125792   803 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:20.137203 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.171883   803 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:20.137914   840 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:17:20.163332 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:20.163131   760 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.174237   803 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.177920   808 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.163439 31869 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 2, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:20.186543   808 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.187003   808 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.167937   837 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.135356 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.163333   788 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:17:20.168756   767 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.181634   790 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:20.182664   775 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:20.188707 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53445.6us
I20260709 12:17:20.200913 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.170763   765 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:20.179437   849 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:17:20.208770 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.217202 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.198508   803 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.222506 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.224438 32121 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:20.226444 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.227385 32112 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:20.227680 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.228387   768 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.229022   768 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.229285   809 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.230141   809 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.231315   768 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.232352   770 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.234633 32046 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:17:20.226145 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 162679us
I20260709 12:17:20.235603 31972 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.150105 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.212772   805 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.237510 32655 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.244931   789 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.252156   822 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.252911 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.253888 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.255694 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.257478   806 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:20.258046   788 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:20.258247   765 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.258816   775 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.259816   815 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.260574   809 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.261312 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.265911 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:20.293105 32660 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.274755 31972 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.234633   803 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.267961 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.240288 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:20.289502 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 139481us
I20260709 12:17:20.266471   805 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.281705   711 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.284324   732 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.282485 32113 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.364084   768 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.364639   768 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.366600   811 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.367302   811 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.369213 32655 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:20.371028 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 103563us
I20260709 12:17:20.371353 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.371917   775 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.372726   817 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:20.373299   817 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.373582 32010 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:20.373431 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.374567 32113 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.374431   817 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:20.375785   730 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:20.376303   730 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.376897 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.377221   730 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.378408   811 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.379280   768 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.381091   802 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.382159 32121 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:20.384235 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.385568 32113 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
W20260709 12:17:20.303232 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.296527   726 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.510s	user 0.002s	sys 0.008s
I20260709 12:17:20.387581   726 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:20.388200   726 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.348415 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.391435 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.392367   787 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.345966   805 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.353602 31972 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.286387   808 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.364092 32656 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.394723   808 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.355531   781 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.395718   781 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.358140 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:20.345350   818 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.006s
I20260709 12:17:20.362443   833 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.384907   822 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:20.327267 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.407141 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.396860   781 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.418489 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:20.428994 32660 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.430584 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:20.287683 32112 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:20.436465 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.440730 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.441483 31972 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.286674   806 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.295118   803 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.418195   808 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:17:20.448226   789 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.448578   808 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.448733   789 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:20.449011   808 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:20.449174   805 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.449769   789 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.418385   824 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.451524   824 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:20.449934   808 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.453838   824 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.454022   808 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:20.455128 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.455839 31968 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:20.419066   845 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.416700   783 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.455118 32046 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:20.412566   788 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:20.463814 31967 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.434717   726 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:20.469300 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.297755   815 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.467521   818 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:20.467875 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.468441 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:20.439604   781 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.460539   845 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.445871 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:20.454247 31966 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:20.409325   711 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.473271 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.493525 31968 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:17:20.466070 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.433938   833 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.475785 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:20.464119 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.461498 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:20.465632   809 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.470216   768 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:20.510710 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.478300   845 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.514994 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.497555 32655 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:20.528951 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65036.5us
I20260709 12:17:20.499253   802 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.562979 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.563860 31966 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.566429 31972 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.531705 31971 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.532125   768 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.535423 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.536626 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:20.537611 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:20.514240   811 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.538671 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:20.505228   818 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.505954   809 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.543195 32656 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:20.514544 31963 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:20.546164 32652 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:20.551638   806 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.552588 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.553926   815 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.554356   775 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.554687   788 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:20.557334   805 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.559928   711 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.511715   726 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:20.530767 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.566668 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.567381 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:20.528087 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 383405us
I20260709 12:17:20.565112 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.568243 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.561826 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.575372 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:20.588606 31967 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.584268 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:20.611145 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81281.2us
I20260709 12:17:20.611188 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.612514 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.613085 32043 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.613622   732 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.614177   822 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.614192 31966 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:20.585575 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.619498 32044 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:20.620563   811 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.621474   768 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:20.592856 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81851.8us
I20260709 12:17:20.599023   803 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.627440   805 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:20.629335   803 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.593708   789 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:20.630642   789 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.597419 31969 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:20.602741 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:20.596385 32651 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.613575 31965 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:20.644271 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.644968 31963 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.646970 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.647621 31969 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.647670 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:20.648551 32113 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.650463   833 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.652346   818 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:20.653424 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.654652   862 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.636637   815 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.656075   815 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.659214 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:20.659869   806 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:20.664207   789 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.664949 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.608640   775 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.667089 32119 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:20.667637   775 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.587702   730 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:17:20.638299 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.668570 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:20.669358 32118 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.672101 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:20.672714 32038 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.652387   711 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.640686   765 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.674860   765 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.676808   765 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.665796 32121 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:20.677874 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.678577 31964 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.679662 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.680783 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.681560 31964 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:20.601802 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.683121 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.683727 32039 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:20.684404   726 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:20.685420   726 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.673686   711 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:20.688700   730 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.689152   711 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.689149   730 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:20.690346   730 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.691479 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.692903   831 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:20.694347   831 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:20.694710   831 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.695029   831 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.603052   788 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.702642   788 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.706241 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:20.706578 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.731742 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.733870 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.736930 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:20.737738 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.739533 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:20.653486   859 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:17:20.742367 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.743160 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:20.743468 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 129676us
I20260709 12:17:20.746585 31972 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.745968 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.747466 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.747817 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.708027 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.748390 31970 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.705147   815 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.752053 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:20.752053 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.766712 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.767912 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.740013 32655 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.739931 31963 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.745791   765 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:20.772132 31972 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
W20260709 12:17:20.760064   865 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:17:20.778429 32043 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.747534   870 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:20.743484 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 130607us
I20260709 12:17:20.793774 32113 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:20.784584 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.776062 31965 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:20.786486   873 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.786923 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.830672 31963 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:20.830839   873 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.751103   809 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.794890   760 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.832484   760 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.833091   873 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.795288   822 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.830057 32119 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:20.834470   760 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.783779 31969 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:20.835541 31971 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:20.771191   869 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.799391 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.777926 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:20.839082 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50858.6us
I20260709 12:17:20.839951 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:20.841063 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.800061 32118 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:20.809515   753 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.842202   753 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.842540   858 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.843281   858 raft_consensus.cc:2764] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:17:20.843757   858 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.800853 32651 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:20.759601   775 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.852622 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:20.853190 31966 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.856379 32118 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:20.858554 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.859555   870 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:20.859999 32120 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 12:17:20.767235 31964 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.777190   765 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.806015   815 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.809463   860 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:20.810007 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56220.6us
I20260709 12:17:20.811866 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:20.860353 32120 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 2
I20260709 12:17:20.786183 32042 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:20.848052 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:20.797859   811 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.876794 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.877876 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.878736 32043 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:20.852380 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.801824   845 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:20.853801 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72460.6us
I20260709 12:17:20.854643 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:20.879674 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 67883.7us
I20260709 12:17:20.865280 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.812789 32116 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:20.868287 32658 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:20.863384   765 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.887272 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:20.887997 32118 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:20.869119 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.869903   870 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.864023 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.892711 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.870540   860 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.828939   781 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.873674 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.874078 32038 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:20.875502 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.875633 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.908010 32047 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:20.908057   732 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.884932 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:20.908664   732 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:20.844275   753 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:20.909566   732 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.886494   871 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.891862   817 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.916621 31969 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:20.917145   871 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:20.916635   817 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.918054   871 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.896368   788 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.897353   845 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:20.899351 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.892069   870 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.893647 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.864562 32120 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:20.922848 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.923669 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.910499   799 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:20.911412   869 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:20.910501   803 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.889065 32045 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:20.928122 32120 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:20.928704 32120 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:20.880776   815 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:20.859568   831 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.912302 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:20.912657   808 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.986204   808 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.986689   808 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:20.987108 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 146154us
I20260709 12:17:20.905452   789 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.918278   817 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.925284 32119 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.926510 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:20.989522 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63101.8us
I20260709 12:17:20.927958   753 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.929320 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:20.930493 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.930608 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:20.884804   730 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:20.932361 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:20.913187   860 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.000968   765 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.003930 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:21.005456   765 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:21.006009   877 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:17:21.001843   783 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:17:21.007802 32658 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.007967   870 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:21.009260   874 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
I20260709 12:17:21.009322   870 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.985775   803 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:20.932775   770 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:20.994272 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 95009.1us
I20260709 12:17:20.994966 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.998250 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:20.924355 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:21.000459 32118 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:21.000623   833 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.931473   775 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:21.000470 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71220.5us
I20260709 12:17:20.992666 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:20.985569   845 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.014632 32658 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 12:17:21.018222 32045 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.001600   711 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.018603 32117 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:21.016196 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:20.999568   753 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.995455   789 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.993999 32116 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.021005   806 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.033207 32040 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.033908 32039 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.027105   783 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.043206 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:21.037644 31970 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:21.045646 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:21.047276 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 114988us
I20260709 12:17:21.000030 31966 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:21.035933 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:21.066394 31971 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.025982 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:21.014287   788 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.026368   833 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.032734   775 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.037434 32658 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 2
I20260709 12:17:21.062613   837 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:21.082774 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.049540   711 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.081530 32114 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
W20260709 12:17:21.095451 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.067041   753 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.037844   869 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:17:21.049461 32044 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.063324 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.066812   789 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.070118   845 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:21.085687 32658 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:21.109911 32658 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:21.015064   803 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.085251   833 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.091914   775 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:21.112341 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103900us
I20260709 12:17:21.110378 32658 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:21.096331   711 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.072862   806 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:21.114446 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 88584.1us
I20260709 12:17:21.119743 31964 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.103802   732 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.121652   818 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:21.103595   837 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:21.103436   879 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.094005   817 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:17:21.132808   858 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:21.116964 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:21.166358   818 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.169277   831 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:21.179776 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62886.5us
I20260709 12:17:21.188218   837 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.190223 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:21.191573 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:21.193642   879 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.202642 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:21.188700   862 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.207322   862 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:21.208417   890 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:21.208734   862 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.209167   890 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.188375   858 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.215194   730 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.215670   818 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.216432   815 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.219076   815 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.193949   845 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.220589   808 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.226526 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:21.227789 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 149564us
W20260709 12:17:21.206852   880 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:17:21.230757 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:21.231630 32657 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:21.230494 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:21.234655 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:21.236510   860 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.245085   818 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:17:21.253463 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.253939 32116 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:21.255003   808 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.258157 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:21.245085   815 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.239814   891 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.204586   833 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.251631   860 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.265926   770 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:21.276036   879 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:21.262797 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:21.271409   809 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.268668 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:21.267907   760 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.262871   818 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.265849   873 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.270720   858 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.274509   860 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.324067 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:21.267910   815 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.329165 31970 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.329617   907 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.285147   770 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.323050   879 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.269637   775 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.328554 32042 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.327574   873 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.336876   815 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.324126 31964 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.341884   907 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:21.369721   907 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.375196 31964 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:21.379969   890 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.385442   753 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:21.387873   753 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.388031 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.388754 31969 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:21.395941   711 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.401064   775 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.398075   711 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.403873   806 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.404016   869 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.404737   869 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.407492   869 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.409534   781 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:21.410178   781 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.416694   824 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.417303   824 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.419466 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 145) took 1543 ms. Trace:
I20260709 12:17:21.424924   869 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.331184   789 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.328692   760 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.350044   873 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:21.364722   894 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:17:21.365722   833 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.367223   888 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:21.368848   862 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.334568   818 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.427362   905 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.425690   869 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.451974   862 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.452805   905 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.244932 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:21.454293 31869 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 240) took 1335 ms. Trace:
I20260709 12:17:21.461750 31970 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:21.478713   726 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.479281   726 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.481493   726 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.460629   789 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.459472   760 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.485476 31870 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 3, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:21.499657 32651 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:21.501271   860 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.503193   824 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.506325 32046 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:21.419878 31867 rpcz_store.cc:276] 0709 12:17:19.876362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.877159 (+   797us) service_pool.cc:224] Handling call
0709 12:17:21.419390 (+1542231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:20.287192 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:20.389560 (+102368us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:20.389628 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:17:20.389669 (+    41us) tablet.cc:662] Decoding operations
0709 12:17:20.390148 (+   479us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:20.390272 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:20.390305 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:20.390667 (+   362us) tablet.cc:701] Row locks acquired
0709 12:17:20.390707 (+    40us) write_op.cc:260] PREPARE: finished
0709 12:17:20.390882 (+   175us) write_op.cc:270] Start()
0709 12:17:20.391046 (+   164us) write_op.cc:276] Timestamp: P: 1783599440390820 usec, L: 0
0709 12:17:20.391097 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 12:17:20.391880 (+   783us) log.cc:844] Serialized 530 byte log entry
0709 12:17:21.015497 (+623617us) op_driver.cc:464] REPLICATION: finished
0709 12:17:21.086824 (+ 71327us) write_op.cc:301] APPLY: starting
0709 12:17:21.086951 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:21.087205 (+   254us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:21.087429 (+   224us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:21.087769 (+   340us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:21.088160 (+   391us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:21.088249 (+    89us) write_op.cc:312] APPLY: finished
0709 12:17:21.088581 (+   332us) log.cc:844] Serialized 23 byte log entry
0709 12:17:21.088797 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:21.089050 (+   253us) write_op.cc:454] Released schema lock
0709 12:17:21.089121 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:21.090026 (+   905us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:21.090066 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":70986,"apply.run_cpu_time_us":2543,"apply.run_wall_time_us":3157,"num_ops":1,"prepare.queue_time_us":277635,"prepare.run_cpu_time_us":9158,"prepare.run_wall_time_us":350609,"raft.queue_time_us":216409,"raft.run_cpu_time_us":39142,"raft.run_wall_time_us":116025,"replication_time_us":624236,"spinlock_wait_cycles":22784,"thread_start_us":427250,"threads_started":4,"wal-append.queue_time_us":212639}]]}
I20260709 12:17:21.461752 31869 rpcz_store.cc:276] 0709 12:17:20.119253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.119533 (+   280us) service_pool.cc:224] Handling call
0709 12:17:21.454230 (+1334697us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:20.635850 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:20.698882 (+ 63032us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:20.698918 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:17:20.698935 (+    17us) tablet.cc:662] Decoding operations
0709 12:17:20.699301 (+   366us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:20.699347 (+    46us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:20.699375 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:20.699564 (+   189us) tablet.cc:701] Row locks acquired
0709 12:17:20.699585 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:17:20.699722 (+   137us) write_op.cc:270] Start()
0709 12:17:20.699824 (+   102us) write_op.cc:276] Timestamp: P: 1783599440699675 usec, L: 0
0709 12:17:20.699844 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:17:20.700466 (+   622us) log.cc:844] Serialized 535 byte log entry
0709 12:17:21.170019 (+469553us) op_driver.cc:464] REPLICATION: finished
0709 12:17:21.170450 (+   431us) write_op.cc:301] APPLY: starting
0709 12:17:21.170527 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:21.170640 (+   113us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:21.170735 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:21.170959 (+   224us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:21.171181 (+   222us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:21.171223 (+    42us) write_op.cc:312] APPLY: finished
0709 12:17:21.171403 (+   180us) log.cc:844] Serialized 23 byte log entry
0709 12:17:21.171546 (+   143us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:21.171621 (+    75us) write_op.cc:454] Released schema lock
0709 12:17:21.171661 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:21.172010 (+   349us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:21.172037 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":250005,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":1495,"apply.run_wall_time_us":1496,"num_ops":1,"prepare.queue_time_us":382850,"prepare.run_cpu_time_us":2645,"prepare.run_wall_time_us":95297,"replication_time_us":424250}]]}
I20260709 12:17:21.455487   905 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.512202   815 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.515990   873 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:21.525645 32042 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:21.527122   869 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.519887   907 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.521651   860 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.524655 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:21.525156   818 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.542640 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:21.543356 32117 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.536727   873 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.610584 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:21.611477 31968 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.537127   783 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:17:21.616137 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:21.615945 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:21.616853 31971 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.617380 32114 raft_consensus.cc:2368] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:21.579253   818 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.623760 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:21.624091 31969 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.624781 32113 raft_consensus.cc:2368] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:21.625576 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:21.626706 31964 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.634915 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:21.635771 31963 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.638823   818 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.614200 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:21.640125   890 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:21.640769   890 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.642880   890 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.644467 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:21.644785   907 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:21.640223 32116 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.648481 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:21.649226 32651 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:21.649382   873 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.649879   873 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.650863   818 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.599567   815 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:21.652139   873 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.651592 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:21.653424 32114 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:21.653671 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:21.662138 32117 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:21.668640 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:21.669536 32117 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:21.684001 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.684902 31970 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:21.692109 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:21.693152 32114 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:21.697863 31971 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:21.699466 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.703218 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:21.711433 31968 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:21.712407 32013 leader_election.cc:400] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:21.712813 32013 leader_election.cc:403] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:21.713794 32013 leader_election.cc:400] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:21.714123 32013 leader_election.cc:403] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:21.715883 31971 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:21.685678 32117 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:21.670071   871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.015s	sys 0.058s
I20260709 12:17:21.721971 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:21.723444 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:21.721102   871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.016s	sys 0.059s
I20260709 12:17:21.726655 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:21.727643   768 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.731643   768 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.731976   768 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.724941   890 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.727272   869 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.740880 31968 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.740588   811 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:21.727751   809 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:21.725944   818 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.727252 31964 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
W20260709 12:17:21.753295 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 486542us
I20260709 12:17:21.721239   781 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.765544 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.771039 31971 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:21.739719 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:21.776963 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.769301 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:21.780053 31970 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:21.753149   890 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:21.751106   811 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.776680   809 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:21.781772   901 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.804940   901 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.805423   901 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.805191   809 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.781937   892 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:21.769750   873 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.807220   873 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:21.745537 31968 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:21.742806   870 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:17:21.801266 31966 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:21.812469 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:21.803809 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:21.811379 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:21.811806 32114 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:21.802947   811 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.810408   873 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:21.922793 31971 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:21.877791 31968 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:21.943821 31968 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:21.888383   803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.006s	sys 0.020s
W20260709 12:17:21.928328   767 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.026s	sys 0.050s
W20260709 12:17:21.973131 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 161813us
I20260709 12:17:22.063799 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:21.949563   905 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:21.945960 32655 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.030s
W20260709 12:17:22.021586   803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.006s	sys 0.023s
I20260709 12:17:21.979673   873 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.953672 31963 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:21.991981 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.003s	sys 0.031s
I20260709 12:17:22.019893   860 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:22.055356   770 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.004s	sys 0.028s
I20260709 12:17:22.107738 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.102654 31968 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:22.065196 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:22.126729 31970 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
W20260709 12:17:22.126868 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61738.5us
W20260709 12:17:22.132490 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.009s	sys 0.009s
W20260709 12:17:22.132869 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.010s	sys 0.009s
I20260709 12:17:22.110397   905 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.106606   815 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:22.119077 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.003s	sys 0.032s
I20260709 12:17:22.085234 32651 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:17:22.103603 32655 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.007s	sys 0.035s
I20260709 12:17:22.134020 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.138482 31968 pending_rounds.cc:85] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 1
I20260709 12:17:22.138406 31967 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:22.065630   767 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.031s	sys 0.059s
I20260709 12:17:22.142436 31968 pending_rounds.cc:107] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:22.142912 31968 raft_consensus.cc:2889] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:22.126343   770 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.004s	sys 0.029s
W20260709 12:17:22.145526 31870 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.013s	sys 0.031s
I20260709 12:17:21.806483   892 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:22.137164   845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.020s	sys 0.030s
I20260709 12:17:22.122711   803 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.134732 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.153184 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:22.154826 31870 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.014s	sys 0.031s
W20260709 12:17:22.161206 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.161429 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.166374 31972 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.166877 31968 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:22.152875   831 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.171156   905 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:22.170912   831 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:22.151427 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 953253us
W20260709 12:17:22.172816   845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.020s	sys 0.031s
I20260709 12:17:22.173373   831 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.173575 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:22.173482   845 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.160662   837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.014s	sys 0.026s
W20260709 12:17:22.175369   837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.015s	sys 0.027s
I20260709 12:17:22.174970   845 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.176234 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.176313 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.177100   845 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.177196 31969 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:22.177624 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.178242 31964 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:22.178275 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.178995 31967 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:22.179101 31969 raft_consensus.cc:2376] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2. Re-sending same reply.
I20260709 12:17:22.180073 31968 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.160749   768 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:22.180949 32121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.008s	sys 0.045s
W20260709 12:17:22.181247 32121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.045s
I20260709 12:17:22.182399   768 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.184448   837 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.185024   837 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.187179   837 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.188020 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.188602 31969 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.190088   892 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.191927   808 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:22.192615   808 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:22.195497   871 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:22.196162   871 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.202491 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.203261 31970 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.104267   890 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.201323   770 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.228580 32657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.001s	sys 0.022s
W20260709 12:17:22.236531 32657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.002s	sys 0.022s
W20260709 12:17:22.229328 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.002s	sys 0.027s
W20260709 12:17:22.215059 32115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.034s	sys 0.034s
W20260709 12:17:22.229969 32112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.014s	sys 0.031s
I20260709 12:17:22.230070 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.262109 31967 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:22.233453 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.236626 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:22.220605   904 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:17:22.205610 32652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.016s	sys 0.040s
W20260709 12:17:22.257144 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.003s	sys 0.030s
W20260709 12:17:22.223852 32116 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.003s
I20260709 12:17:22.149163   873 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:22.263499 31971 raft_consensus.cc:2368] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:22.193786   808 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.220810   906 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:22.169659   803 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:22.273823 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 1.0121e+06us
W20260709 12:17:22.258121 32115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.034s	sys 0.034s
I20260709 12:17:22.187618   837 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:22.271677 32112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.014s	sys 0.032s
W20260709 12:17:22.273031   805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.197s	user 0.024s	sys 0.024s
W20260709 12:17:22.265584   879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.033s	sys 0.052s
I20260709 12:17:22.258217   781 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.272324 31970 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:22.277783 32652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.016s	sys 0.040s
I20260709 12:17:22.277616 31965 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:22.257651 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.702s (SENT) [suppressed 6 similar messages]
I20260709 12:17:22.278550   871 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:22.304040 31972 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:22.302968   879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.033s	sys 0.053s
I20260709 12:17:22.307107 31968 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:22.309154   902 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:17:22.276273 32116 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:22.298511   805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.025s	sys 0.025s
I20260709 12:17:22.296995   906 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:22.273240   770 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.307231 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:22.373471   893 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.323873 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1419) took 1401 ms. Trace:
I20260709 12:17:22.407606 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:22.298537   837 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:22.432550   908 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 12:17:22.427376   912 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:17:22.393172   905 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.462733   905 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.465638   905 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:22.466436   911 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:17:22.470603   837 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.404438 32115 rpcz_store.cc:276] 0709 12:17:20.919369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.919910 (+   541us) service_pool.cc:224] Handling call
0709 12:17:20.930681 (+ 10771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.930917 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:21.262784 (+331867us) log.cc:844] Serialized 14 byte log entry
0709 12:17:22.294178 (+1031394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:22.297986 (+  3808us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:22.298077 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.299437 (+  1360us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.320765 (+ 21328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51426,"prepare.queue_time_us":397407,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":109347,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"spinlock_wait_cycles":87936,"thread_start_us":1212144,"threads_started":2}
I20260709 12:17:22.427748   893 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.421545 32651 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:22.492759 32660 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:22.264019 31969 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:22.382170   869 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.000s
I20260709 12:17:22.538841 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 364) took 1346 ms. Trace:
I20260709 12:17:22.490612   781 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:22.543130   803 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:22.497035   811 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.001s
I20260709 12:17:22.534077 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1420) took 1531 ms. Trace:
I20260709 12:17:22.488327 31965 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:22.425397 31869 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 2 to 3, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 3 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:22.571106 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:22.551493   899 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:22.571892 31869 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 2 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:22.556190   869 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:22.556313   909 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:22.547143 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.549054   893 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.505050   906 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.552211 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.556125 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:22.556146 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:22.488072   818 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.703s	user 0.002s	sys 0.006s
I20260709 12:17:22.553423 32658 rpcz_store.cc:276] 0709 12:17:21.192268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.193336 (+  1068us) service_pool.cc:224] Handling call
0709 12:17:21.193666 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.282727 (+ 89061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.282777 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.283427 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:17:21.426848 (+143421us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b48000fd458. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:17:21.751618 (+324770us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:21.751702 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.751790 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.538577 (+786787us) raft_consensus.cc:1685] finished
0709 12:17:22.538595 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.538769 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147044,"raft.run_cpu_time_us":2987,"raft.run_wall_time_us":81414,"spinlock_wait_cycles":3712256,"thread_start_us":556118,"threads_started":2,"wal-append.queue_time_us":819594}
I20260709 12:17:22.409550   913 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:22.560740 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.000s	sys 0.049s
I20260709 12:17:22.530364   910 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.562654 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.519977 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:22.565097 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:22.496448   770 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:22.555925   888 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:22.580163 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.582425 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 277) took 1327 ms. Trace:
W20260709 12:17:22.573933   809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.015s	sys 0.010s
I20260709 12:17:22.625586 32046 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.606597 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.606850 31869 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:22.627049 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.000s	sys 0.055s
I20260709 12:17:22.605466 32120 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:22.627660 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:22.678637 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.679227 32043 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.679167 32045 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:22.583062 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:22.627772 32121 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.570732 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.686156 32045 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:22.593588 32042 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.685911   870 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.569326   811 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.566924   781 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.690785 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:22.675397 32657 rpcz_store.cc:276] 0709 12:17:21.255475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.256647 (+  1172us) service_pool.cc:224] Handling call
0709 12:17:21.256937 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.438334 (+181397us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b48000fd458. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:17:21.664901 (+226567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.664961 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.677576 (+ 12615us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.236980 (+559404us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:22.237021 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.237133 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.436042 (+198909us) raft_consensus.cc:1685] finished
0709 12:17:22.436060 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.582345 (+146285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91778,"raft.run_cpu_time_us":1215,"raft.run_wall_time_us":87867,"spinlock_wait_cycles":9659776,"thread_start_us":917211,"threads_started":2,"wal-append.queue_time_us":749814}
W20260709 12:17:22.677592   809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.016s	sys 0.011s
I20260709 12:17:22.578037   803 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.675774   873 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.710608   783 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.711393   781 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.711771   768 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.583083   920 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.584509   741 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.724864 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.725106   920 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.627859   909 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.725644 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:22.726922 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.727428   920 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:22.581326   899 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.727569 31969 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:22.727970   920 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.728394   920 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.728623 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.729391   899 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:22.724881   768 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.730089   899 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:22.730386   783 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.730603   899 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.731575   768 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.732146   768 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.732491   783 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:22.622417   837 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.732580   768 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.733796 31971 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.690514   893 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.688076 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.736542 32043 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:22.744213 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.036s	sys 0.039s
I20260709 12:17:22.725183   741 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.712951   803 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:22.765671 32655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.032s	sys 0.041s
I20260709 12:17:22.721644 32047 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:22.686527   805 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.724383   781 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:22.748799   892 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.730214   920 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:22.812374   860 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.685s	user 0.013s	sys 0.005s
I20260709 12:17:22.813220   783 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:22.770470 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.705469 31966 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
W20260709 12:17:22.826720   815 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.009s	sys 0.009s
I20260709 12:17:22.817934 31971 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:22.747646 32040 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
W20260709 12:17:22.792150 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.018s	sys 0.034s
W20260709 12:17:22.920025 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.018s	sys 0.034s
I20260709 12:17:22.714157   913 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.845212 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:22.761514 32117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.037s	sys 0.037s
W20260709 12:17:22.863713 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.036s	sys 0.049s
W20260709 12:17:22.925194 32117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.040s	sys 0.049s
W20260709 12:17:22.925407 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 754791us
I20260709 12:17:22.733135   837 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.675542   910 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:22.937280 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.727597   909 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:22.733245 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:22.624316   818 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:22.684541   888 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:22.880337 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 307201us
I20260709 12:17:22.717614   831 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.882066 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:22.689440 32044 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:22.716351   770 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.160s
I20260709 12:17:22.875373 32038 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.834415   869 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:22.716411   870 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:22.928500 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 114402us
I20260709 12:17:22.973289 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:22.914407 32655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.032s	sys 0.041s
I20260709 12:17:22.881337   899 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:22.937328 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:22.760942   893 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:22.922499   815 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.011s	sys 0.021s
W20260709 12:17:22.948290 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 371898us
I20260709 12:17:22.957232 32046 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:17:22.932530 32623 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.023s
I20260709 12:17:22.917939   860 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.962594 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:22.964074 32047 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:22.967023 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:22.957681   781 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.969985 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:22.918956   805 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:22.919050   892 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.921238   783 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.978621 32040 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:22.979192 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.944180 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:22.960101   934 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:22.970374   831 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:22.976447   818 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:23.187569 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:23.187724   818 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:23.188382 32045 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.000056   928 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:23.062019   919 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 12:17:23.191651   893 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:23.199512 32660 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:22.966492 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:23.200387 32118 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:23.186898   936 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 12:17:23.213546 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:23.093655   926 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
W20260709 12:17:23.059758   935 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.244s	user 0.001s	sys 0.003s
I20260709 12:17:23.244932 32040 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:23.019743   899 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:23.221940 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 245113us
I20260709 12:17:22.976516   942 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:23.032783   890 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.095728   932 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:23.258296 32043 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:23.274791 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:23.138902 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 278) took 1746 ms. Trace:
I20260709 12:17:23.216710 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:23.150823 32115 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:23.151752   768 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:23.205830   901 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.400s	user 0.008s	sys 0.005s
I20260709 12:17:23.268206 32118 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:22.987377   815 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:22.572315 32112 rpcz_store.cc:276] 0709 12:17:21.002425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.004928 (+  2503us) service_pool.cc:224] Handling call
0709 12:17:21.005356 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.047215 (+ 41859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.049210 (+  1995us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.440967 (+391757us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:21.441524 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.303217 (+861693us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:22.320895 (+ 17678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.325686 (+  4791us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.505333 (+179647us) raft_consensus.cc:1685] finished
0709 12:17:22.505373 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.530922 (+ 25549us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13040,"raft.queue_time_us":41746,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":19163,"spinlock_wait_cycles":6095232,"thread_start_us":854750,"threads_started":1,"wal-append.queue_time_us":1054403}
W20260709 12:17:23.018958 32623 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.008s	sys 0.024s
I20260709 12:17:23.197649   837 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.272410   910 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:22.976917   869 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.272739   951 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:23.266637 32041 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.212198 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:23.154458   937 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:23.370500   937 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:23.373181   937 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:23.374938   888 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:23.274622 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 206142us
I20260709 12:17:23.275660 32040 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:23.245479 32044 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.059681   929 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:23.208556   803 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.257542   934 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.309469 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:23.148571   927 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.152s
W20260709 12:17:23.364305 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 94299.6us
I20260709 12:17:23.369880 32114 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.303133   901 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.380043 32045 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:23.274367   899 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.381526 32047 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:23.283263   890 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:23.384194 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1432) took 1642 ms. Trace:
I20260709 12:17:23.427743 32117 rpcz_store.cc:276] 0709 12:17:21.742025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.742196 (+   171us) service_pool.cc:224] Handling call
0709 12:17:21.742569 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.742822 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.742859 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.923480 (+180621us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:21.967738 (+ 44258us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.925626 (+957888us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:22.925669 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.925741 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:23.377322 (+451581us) raft_consensus.cc:1685] finished
0709 12:17:23.377349 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.384106 (+  6757us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28089600,"thread_start_us":814314,"threads_started":1,"wal-append.queue_time_us":1288332}
I20260709 12:17:23.388192 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:23.430248 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:23.320246 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.06253e+06us
I20260709 12:17:23.431048 32047 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:23.366427   809 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:23.393971 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:23.403128 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 215601us
I20260709 12:17:23.405038 32118 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:23.379434 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:23.287075   932 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.435082 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:23.435797 32117 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:23.385759 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:23.426410 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:23.442269 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62928us
I20260709 12:17:23.383648   953 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:23.360562   741 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:23.403853 32042 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.004s
W20260709 12:17:23.437803 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43957.1us
W20260709 12:17:23.440188 32121 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.754s	user 0.006s	sys 0.004s
I20260709 12:17:23.442572 32045 raft_consensus.cc:2368] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
W20260709 12:17:23.535637 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.624723 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:23.494421 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 79028.8us
I20260709 12:17:23.464864 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:23.627400 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:23.628026 32121 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:23.627584 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:23.630416 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1438) took 1065 ms. Trace:
I20260709 12:17:23.632755 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:23.628291 32117 raft_consensus.cc:2368] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:23.638203 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:23.639657   869 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.640256   901 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:23.640961   901 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.641408   901 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:23.644320 32039 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.000s
I20260709 12:17:23.645380 32039 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:23.636574 32121 rpcz_store.cc:276] 0709 12:17:22.564744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.564868 (+   124us) service_pool.cc:224] Handling call
0709 12:17:23.630085 (+1065217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:23.614487   956 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:17:23.433036   809 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:23.617045 32038 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.001s
I20260709 12:17:23.615918   890 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:23.433346   888 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:23.532475   939 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:17:23.434580   928 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:23.593226   949 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:17:23.434569   831 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.433593   952 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:23.719745   954 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.443073   808 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:23.543586   913 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.533006   870 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:23.445683   937 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:23.463850   959 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:23.726527 32038 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:23.635910   781 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.632972 32042 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:23.474649   803 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.652190   805 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:23.725006   809 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:23.720062 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:23.286966   942 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:23.769354 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 136694us
I20260709 12:17:23.769526 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:23.814026 32118 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:23.653116   951 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 12:17:23.413168   929 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.464800   910 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.473877   815 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.723654   869 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:17:23.751557 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:23.722384 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 127424us
I20260709 12:17:23.759461   890 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:23.745839   964 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.767272   837 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:23.767448   831 raft_consensus.cc:2764] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:23.645359   901 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.784080   893 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.636270   953 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:23.750073   888 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:23.783936 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:23.899041 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:23.899734 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85334.3us
I20260709 12:17:23.803716 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1213) took 1197 ms. Trace:
I20260709 12:17:23.803293   803 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:23.773813   808 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.826360 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1203) took 1266 ms. Trace:
I20260709 12:17:23.795185   937 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.767513   952 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:23.917783   783 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.898394   910 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.783215   954 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.910045   741 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.910346   964 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.904577   781 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:17:23.795531   959 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.905308   928 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.936743 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:23.906355 32623 connection.cc:256] RPC call timeout handler was delayed by -1.0590431139999055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 12:17:23.975898 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:23.278364 32655 rpcz_store.cc:276] 0709 12:17:21.392050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.426595 (+ 34545us) service_pool.cc:224] Handling call
0709 12:17:21.427082 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:21.427334 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:22.155831 (+728497us) log.cc:844] Serialized 14 byte log entry
0709 12:17:22.978241 (+822410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:22.995646 (+ 17405us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:23.021634 (+ 25988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.048465 (+ 26831us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.134439 (+ 85974us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":775760,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":170,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":17900,"thread_start_us":1525092,"threads_started":2,"wal-append.queue_time_us":805587}
I20260709 12:17:24.508255 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 69) took 1109 ms. Trace:
W20260709 12:17:24.344996 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.326021 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:24.543675 32118 rpcz_store.cc:276] 0709 12:17:23.398695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.928995 (+530300us) service_pool.cc:224] Handling call
0709 12:17:24.508180 (+579185us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:24.526237 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 589565us
I20260709 12:17:24.376598 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.329896 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:24.546491 32121 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:24.546602 32045 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:24.548410 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:23.972050   961 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:24.546100 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:24.545890   910 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.353868 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1246) took 1636 ms. Trace:
I20260709 12:17:24.326606 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:24.366222 32039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1207) took 1814 ms (client timeout 2066 ms). Trace:
W20260709 12:17:24.565377 32039 rpcz_store.cc:269] 0709 12:17:22.551814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.551943 (+   129us) service_pool.cc:224] Handling call
0709 12:17:24.366061 (+1814118us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540013d3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:24.366165 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841758080}
I20260709 12:17:24.332029 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:24.345177 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.006s	sys 0.007s
I20260709 12:17:23.922670   950 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:23.921612   890 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.341212 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:24.566788 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.015s	sys 0.015s
W20260709 12:17:24.506919   770 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.006s	sys 0.008s
I20260709 12:17:23.924770   899 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.564311 32043 rpcz_store.cc:276] 0709 12:17:22.717592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.925506 (+207914us) service_pool.cc:224] Handling call
0709 12:17:22.948999 (+ 23493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.275162 (+326163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.353613 (+1078451us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400379e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:24.353806 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710462208}
W20260709 12:17:23.971311   962 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:17:24.086045   869 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:24.468791 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.506635   803 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.576121 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:24.576849 32046 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:23.973102   945 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:24.579838 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 941623us
I20260709 12:17:24.275738   932 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.480934 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.552449 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 73) took 1079 ms. Trace:
W20260709 12:17:24.133404   966 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 12:17:24.506949   964 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.521121   783 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.586920 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.106933 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1276) took 1209 ms. Trace:
I20260709 12:17:23.970398   933 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:23.964314 32656 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:23.928081   934 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:23.925267   893 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:23.972656   960 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:23.921245   837 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:24.571841 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.044s	sys 0.046s
W20260709 12:17:24.801775 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.044s	sys 0.046s
W20260709 12:17:24.802086 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 876665us
I20260709 12:17:24.599481 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:24.803438 32045 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:24.599717 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:24.804491 32047 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:24.600364 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:24.363353   818 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:24.374948 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1254) took 1396 ms. Trace:
I20260709 12:17:24.126081   968 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:23.927940 32038 rpcz_store.cc:276] 0709 12:17:22.606534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.621698 (+ 15164us) service_pool.cc:224] Handling call
0709 12:17:23.797226 (+1175528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:24.359593   811 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.801930   888 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:24.135730 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.809409   960 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.810415 32112 raft_consensus.cc:2410] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:24.810882 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1288) took 1542 ms. Trace:
I20260709 12:17:24.800261   933 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:24.811249 32112 rpcz_store.cc:276] 0709 12:17:23.269051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.973154 (+704103us) service_pool.cc:224] Handling call
0709 12:17:24.810823 (+837669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:24.812472 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.522115   741 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:24.613577   869 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:24.614578   965 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.802s	user 0.001s	sys 0.000s
I20260709 12:17:24.615247   997 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:24.613191 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:24.558133   946 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.491883   873 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:24.490571   913 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:24.334698   906 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:24.824807   906 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:24.825639 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1272) took 1944 ms. Trace:
I20260709 12:17:24.825714   873 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.826171 32120 rpcz_store.cc:276] 0709 12:17:22.880867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.881255 (+   388us) service_pool.cc:224] Handling call
0709 12:17:22.921880 (+ 40625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:23.143993 (+222113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.144037 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:23.191741 (+ 47704us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.402704 (+210963us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:23.757274 (+354570us) log.cc:844] Serialized 14 byte log entry
0709 12:17:23.757483 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.757572 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:24.825450 (+1067878us) raft_consensus.cc:1685] finished
0709 12:17:24.825468 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.825586 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8891,"raft.queue_time_us":488320,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":10035,"thread_start_us":759982,"threads_started":3,"wal-append.queue_time_us":539321}
I20260709 12:17:24.827301   906 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.471474 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.828116 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.630163   989 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.829330 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 283) took 1542 ms. Trace:
I20260709 12:17:24.631739 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:24.829840   913 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:24.338150 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.669536 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1435) took 2183 ms. Trace:
I20260709 12:17:24.670020   998 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.670806 32119 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:24.674253 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.593823   976 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:24.806910 32044 rpcz_store.cc:276] 0709 12:17:22.978971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.474557 (+495586us) service_pool.cc:224] Handling call
0709 12:17:23.724630 (+250073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.919659 (+195029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.921746 (+  2087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:23.925880 (+  4134us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:23.928290 (+  2410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.374670 (+446380us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000582d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:24.374733 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.374879 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418257280}
I20260709 12:17:24.788957   783 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.591893   967 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.696442   991 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.843307 32654 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:24.843194   991 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.696521   996 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.844184   996 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.844540 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.845314 32654 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:24.845592   967 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.687533   945 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:24.847046 32041 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.477s	user 0.006s	sys 0.006s
I20260709 12:17:24.847421 32041 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:24.847995 32041 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1208) took 2295 ms (client timeout 1898 ms). Trace:
I20260709 12:17:24.847746   945 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:24.848507 32041 rpcz_store.cc:269] 0709 12:17:22.553075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.565656 (+ 12581us) service_pool.cc:224] Handling call
0709 12:17:24.847915 (+2282259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:24.833927 32113 rpcz_store.cc:276] 0709 12:17:22.485749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.491087 (+  5338us) service_pool.cc:224] Handling call
0709 12:17:22.558927 (+ 67840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:22.923459 (+364532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:22.929422 (+  5963us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:23.049848 (+120426us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.403950 (+354102us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:23.407774 (+  3824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.425928 (+ 18154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:24.669233 (+1243305us) raft_consensus.cc:1685] finished
0709 12:17:24.669261 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.669442 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152363,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":408346,"threads_started":2,"wal-append.queue_time_us":556001}
I20260709 12:17:24.692508 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.587894   950 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.672771 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.702103   987 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.702463 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.682605   953 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:24.712100   959 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.713472   890 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.707537   929 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.715210   937 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.718866   984 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.859562 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.859462   984 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.720424   808 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.720940   952 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:24.723640   892 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.725131   781 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.863306   781 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.733038   942 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:24.729132   809 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.835s	user 0.007s	sys 0.002s
I20260709 12:17:24.863992   781 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.593438   927 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.782963 32121 rpcz_store.cc:276] 0709 12:17:23.553909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.714819 (+160910us) service_pool.cc:224] Handling call
0709 12:17:24.376423 (+661604us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:24.552208 (+175785us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14542720}
I20260709 12:17:24.881705   809 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.790125   993 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.883817   993 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.680771 32652 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:24.790684   910 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.791127 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 387) took 1071 ms. Trace:
I20260709 12:17:24.793648 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 286) took 1144 ms. Trace:
W20260709 12:17:24.788107 32114 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.359s	user 0.006s	sys 0.005s
I20260709 12:17:24.894208 32651 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:24.899025 32114 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:17:24.806545 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.890957 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:24.900502 32112 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.808048 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:24.587066   770 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.011s	sys 0.013s
I20260709 12:17:24.816538   741 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:24.818259 32039 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:24.819990   963 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:24.820652   869 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.828857   989 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.543705 32042 rpcz_store.cc:276] 0709 12:17:22.546690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.546849 (+   159us) service_pool.cc:224] Handling call
0709 12:17:23.810201 (+1263352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:24.787601 32115 rpcz_store.cc:276] 0709 12:17:22.897381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.914728 (+ 17347us) service_pool.cc:224] Handling call
0709 12:17:22.968154 (+ 53426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.285147 (+316993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.106600 (+821453us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540012b6d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:24.106847 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385720576}
W20260709 12:17:24.830243 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 198596us
I20260709 12:17:24.829716 32657 rpcz_store.cc:276] 0709 12:17:23.286828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.286990 (+   162us) service_pool.cc:224] Handling call
0709 12:17:23.287323 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:23.287671 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.287698 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:23.288196 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.401225 (+113029us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:23.401284 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.401375 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:24.829142 (+1427767us) raft_consensus.cc:1685] finished
0709 12:17:24.829161 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.829278 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":14224,"thread_start_us":14323,"threads_started":1,"wal-append.queue_time_us":270842}
I20260709 12:17:24.830727 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.822006   968 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.839329   998 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.840054   837 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:24.681507   979 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:17:24.832321   803 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:24.594264   920 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:24.841030 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1275) took 1944 ms. Trace:
I20260709 12:17:24.840351   976 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:24.697690   954 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.857928 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:24.883067 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.782402 32117 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:24.828006   906 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:24.890769 32116 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:24.890861 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 389) took 1160 ms. Trace:
I20260709 12:17:24.890957   910 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.891409 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.892321 32658 rpcz_store.cc:276] 0709 12:17:23.649665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.650164 (+   499us) service_pool.cc:224] Handling call
0709 12:17:23.727474 (+ 77310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:23.727913 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.727953 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:23.766706 (+ 38753us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.899403 (+132697us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:23.899441 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.899498 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:24.793472 (+893974us) raft_consensus.cc:1685] finished
0709 12:17:24.793489 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.793603 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":27752,"thread_start_us":130346,"threads_started":1,"wal-append.queue_time_us":1023013}
I20260709 12:17:24.897653 31869 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 226) took 2566 ms. Trace:
W20260709 12:17:24.980396 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 172461us
I20260709 12:17:24.980882 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.981588 32120 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:24.898268 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.821350   997 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:24.985198 32113 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:24.980146   888 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.989255   997 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.982229 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:24.989202   837 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.980768 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 70) took 1581 ms. Trace:
I20260709 12:17:24.993342 32117 rpcz_store.cc:276] 0709 12:17:23.399009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.480597 (+1081588us) service_pool.cc:224] Handling call
0709 12:17:24.980714 (+500117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:24.980916 32119 rpcz_store.cc:276] 0709 12:17:22.896992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.897121 (+   129us) service_pool.cc:224] Handling call
0709 12:17:24.670635 (+1773514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.833615 (+162980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.840840 (+  7225us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400149bd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:24.840979 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4022759808}
I20260709 12:17:24.902612   996 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:24.979868 32653 rpcz_store.cc:276] 0709 12:17:23.730931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.734284 (+  3353us) service_pool.cc:224] Handling call
0709 12:17:23.768956 (+ 34672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.824723 (+1055767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.824759 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:24.824808 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:24.824842 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.890630 (+ 65788us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003efb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:24.890679 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.890802 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422428544}
I20260709 12:17:24.984513   892 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.995357 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.980706 31869 rpcz_store.cc:276] 0709 12:17:22.322909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.328566 (+  5657us) service_pool.cc:224] Handling call
0709 12:17:24.889370 (+2560804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:22.923418 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:23.020179 (+ 96761us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:23.036277 (+ 16098us) write_op.cc:435] Acquired schema lock
0709 12:17:23.036312 (+    35us) tablet.cc:662] Decoding operations
0709 12:17:23.270283 (+233971us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:23.270566 (+   283us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:23.270614 (+    48us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:23.271035 (+   421us) tablet.cc:701] Row locks acquired
0709 12:17:23.271065 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:17:23.271248 (+   183us) write_op.cc:270] Start()
0709 12:17:23.271388 (+   140us) write_op.cc:276] Timestamp: P: 1783599443271191 usec, L: 0
0709 12:17:23.271431 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:17:23.367194 (+ 95763us) log.cc:844] Serialized 1285 byte log entry
0709 12:17:23.778434 (+411240us) op_driver.cc:464] REPLICATION: finished
0709 12:17:24.502724 (+724290us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cae28 00007f228fc05581 00007f228fc04d58 00007f228fc099f1 00007f228fc099a9 00007f228fc09939 00007f228fc09901 00007f228fc08b2d 0000000000388634 0000000000388479
0709 12:17:24.817058 (+314334us) write_op.cc:301] APPLY: starting
0709 12:17:24.817167 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:24.817353 (+   186us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:24.817384 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:24.858191 (+ 40807us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:24.858372 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:24.858414 (+    42us) write_op.cc:312] APPLY: finished
0709 12:17:24.858636 (+   222us) log.cc:844] Serialized 35 byte log entry
0709 12:17:24.858774 (+   138us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:24.858862 (+    88us) write_op.cc:454] Released schema lock
0709 12:17:24.858908 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:24.859271 (+   363us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:24.859300 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":1023084,"apply.run_cpu_time_us":2293,"apply.run_wall_time_us":42202,"num_ops":3,"prepare.queue_time_us":162957,"prepare.run_cpu_time_us":14687,"prepare.run_wall_time_us":1584934,"replication_time_us":492140,"spinlock_wait_cycles":7615360,"thread_start_us":140332,"threads_started":1}]]}
I20260709 12:17:24.987812 32118 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:24.992457 32121 raft_consensus.cc:2376] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3. Re-sending same reply.
I20260709 12:17:24.994176   913 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:24.881702   987 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:24.996169 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 72) took 1596 ms. Trace:
I20260709 12:17:24.988492 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:24.999128 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.994251 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.000288 32117 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:24.999405 32119 rpcz_store.cc:276] 0709 12:17:23.399646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.995096 (+1595450us) service_pool.cc:224] Handling call
0709 12:17:24.996115 (+  1019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.003786 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:24.986162 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:25.007861 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.005748 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:25.002965 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.949580   927 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:25.006470 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:24.933224 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:24.899566 32114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1277) took 1996 ms (client timeout 1572 ms). Trace:
I20260709 12:17:25.011303 32118 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:24.903502   809 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.013638 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:25.013299 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 380304us
I20260709 12:17:25.009619 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:24.903088   960 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.017951 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:25.022926 32121 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:25.007676   899 raft_consensus.cc:2764] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:25.002089   888 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:24.990288   906 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.020419 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1282) took 2115 ms. Trace:
I20260709 12:17:25.010880   910 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.032349   910 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:25.033433   906 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.033224   910 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.033530 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.034204 31969 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:24.892105 32656 rpcz_store.cc:276] 0709 12:17:23.719566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.722383 (+  2817us) service_pool.cc:224] Handling call
0709 12:17:24.791048 (+1068665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.031277 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:24.999212   920 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.030498 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1283) took 2125 ms. Trace:
I20260709 12:17:25.031239 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.031440 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.042179 31971 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:25.041893 32118 rpcz_store.cc:276] 0709 12:17:22.905294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.019338 (+2114044us) service_pool.cc:224] Handling call
0709 12:17:25.029855 (+ 10517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.030061 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.030100 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.030157 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:25.030203 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.030276 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.030431 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.035638 32117 rpcz_store.cc:276] 0709 12:17:22.904989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.019424 (+2114435us) service_pool.cc:224] Handling call
0709 12:17:25.019775 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.020008 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.020048 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.020106 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:25.020152 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.020223 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.020343 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.036134 32659 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:25.011978   954 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.037225 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1279) took 2133 ms. Trace:
I20260709 12:17:25.049782 32120 rpcz_store.cc:276] 0709 12:17:22.904075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.014674 (+2110599us) service_pool.cc:224] Handling call
0709 12:17:25.028461 (+ 13787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.036803 (+  8342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.036845 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.036917 (+    72us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:25.036962 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.037041 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.037170 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.017464   770 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.044661 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 65) took 1647 ms. Trace:
I20260709 12:17:25.053411 32117 rpcz_store.cc:276] 0709 12:17:23.397246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.043674 (+1646428us) service_pool.cc:224] Handling call
0709 12:17:25.044096 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.044294 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.044330 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.044383 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:25.044424 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.044486 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.044600 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.012984   927 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:25.052872 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.016s	sys 0.025s
I20260709 12:17:25.024140   942 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.038632 32113 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:25.025082   741 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.028009 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:25.043385   768 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.013s	sys 0.025s
W20260709 12:17:25.051796 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 475808us
I20260709 12:17:25.026407   968 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:25.154445 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.030s	sys 0.119s
I20260709 12:17:25.307806 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:25.327016 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.031s	sys 0.124s
I20260709 12:17:25.327184 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:25.077628 32047 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
W20260709 12:17:25.148294 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.227s	user 0.000s	sys 0.029s
W20260709 12:17:25.329411 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.000s	sys 0.036s
I20260709 12:17:25.309011 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 67) took 1911 ms. Trace:
W20260709 12:17:25.135059 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 78601.4us
I20260709 12:17:25.338989 32120 rpcz_store.cc:276] 0709 12:17:23.398050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.050834 (+1652784us) service_pool.cc:224] Handling call
0709 12:17:25.307866 (+257032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.308574 (+   708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.308618 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.308675 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:25.308718 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.308793 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.308940 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132}
I20260709 12:17:25.326169 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1285) took 2349 ms. Trace:
I20260709 12:17:25.345268 32118 rpcz_store.cc:276] 0709 12:17:22.977125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.043380 (+2066255us) service_pool.cc:224] Handling call
0709 12:17:25.325460 (+282080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.325702 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.325741 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.325800 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:25.325845 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.325930 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.326077 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:25.343104 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.022s	sys 0.025s
I20260709 12:17:25.262641 32121 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:25.348750 32121 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:25.013274 32114 rpcz_store.cc:269] 0709 12:17:22.903375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.146610 (+243235us) service_pool.cc:224] Handling call
0709 12:17:24.899503 (+1752893us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:25.127816 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 109840us
W20260709 12:17:25.365504 32084 connection.cc:256] RPC call timeout handler was delayed by -1.5229497420000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 12:17:25.365919 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1818497419999403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 12:17:25.366302 32084 connection.cc:256] RPC call timeout handler was delayed by -1.054049741999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
I20260709 12:17:25.147980 32038 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:25.372816 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.369619 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.049324 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:25.223335   815 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.013s	sys 0.014s
W20260709 12:17:25.328712   768 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.024s	sys 0.030s
I20260709 12:17:25.058036   950 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:25.186439   983 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.004s	sys 0.023s
I20260709 12:17:25.126219 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.614442 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.187781 32112 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:25.375092 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:25.383689 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0306497419999232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 12:17:25.619648 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 250104us
W20260709 12:17:25.581856 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.443337 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.629561 31971 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:25.128301   892 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.466674 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.431087 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:25.451952 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.431094 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:25.614704 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 287620us
I20260709 12:17:25.147287   770 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:25.126736   998 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:25.680980 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52743.5us
I20260709 12:17:25.681452 32047 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:25.685639 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.367779   767 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.681607 32046 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:25.681850 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.367470   968 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.684954 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.647951 32120 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:25.368096   960 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.688426 32039 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
W20260709 12:17:25.630651   815 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.020s	sys 0.021s
W20260709 12:17:25.101099   951 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.026s	sys 0.036s
W20260709 12:17:25.556684   975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.995s	user 0.015s	sys 0.022s
I20260709 12:17:25.453464   987 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.395805 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1443) took 1037 ms. Trace:
I20260709 12:17:25.706727 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.707093 32115 rpcz_store.cc:276] 0709 12:17:24.358035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.000983 (+642948us) service_pool.cc:224] Handling call
0709 12:17:25.395732 (+394749us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.707957 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.708561 31969 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:25.708739 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.395392   989 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.693347 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.703531   960 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.633865   953 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.710806 31965 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:25.711225   953 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:25.711830   953 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.712157 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.712894   960 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.690296 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1444) took 1332 ms. Trace:
I20260709 12:17:25.422189  1003 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.692232   892 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.703176   767 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.714536  1003 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.714345 32120 rpcz_store.cc:276] 0709 12:17:24.358272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.367573 (+1009301us) service_pool.cc:224] Handling call
0709 12:17:25.619373 (+251800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.690041 (+ 70668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.690241 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14080}
I20260709 12:17:25.467377   993 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.075927   964 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.716476   767 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.716655 31968 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.717316 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.717962 32040 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:25.056886   954 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.718823 31967 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.719794   954 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.720157   927 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:25.543478 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.011s	sys 0.022s
I20260709 12:17:25.404462   946 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.442565   803 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:25.547417   977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.010s	sys 0.023s
I20260709 12:17:25.706131 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:25.732503 32043 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.368206   741 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:25.706743   975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.015s	sys 0.027s
W20260709 12:17:25.617933   983 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.014s	sys 0.033s
W20260709 12:17:25.452011 32116 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1280) took 2547 ms (client timeout 1927 ms). Trace:
I20260709 12:17:25.722923 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1440) took 1365 ms. Trace:
I20260709 12:17:25.372790   942 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:25.747269   942 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.753341 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1284) took 2791 ms. Trace:
I20260709 12:17:25.753719 32119 rpcz_store.cc:276] 0709 12:17:22.962151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.021619 (+2059468us) service_pool.cc:224] Handling call
0709 12:17:25.752741 (+731122us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:25.752909 (+   168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.753065 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.753088 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.753124 (+    36us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:25.753149 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.753197 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.753292 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12054912}
W20260709 12:17:25.755940   781 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.873s	user 0.006s	sys 0.003s
I20260709 12:17:25.756335   781 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.757351 32042 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:25.395665 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.758136 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.758754 32039 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:25.759227 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.447381   920 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:25.759534 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:25.768699 32039 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:25.705013   951 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.799s	user 0.038s	sys 0.051s
I20260709 12:17:25.727289 31968 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:25.730785 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.770961 32047 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:25.205821 32044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:25.694473  1004 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.709486 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:25.698246   959 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.783666 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:25.730405 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.011s	sys 0.025s
I20260709 12:17:25.739894   927 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:25.743036   770 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.743732   976 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:25.745059 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.745545   964 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:25.126797   888 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.684182 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1449) took 1243 ms. Trace:
I20260709 12:17:25.806529 32112 rpcz_store.cc:276] 0709 12:17:24.437359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.890635 (+453276us) service_pool.cc:224] Handling call
0709 12:17:25.680963 (+790328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.716509   892 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:25.807570 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:25.766580 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 138832us
I20260709 12:17:25.731737   958 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.809561   958 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.697759   963 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.811898   958 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:25.737069   977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.013s	sys 0.024s
I20260709 12:17:25.782847 31972 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:25.786758   815 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.784092 32043 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:25.703912   968 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.738183  1000 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.794768 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:25.787819 32013 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.017s	sys 0.032s
I20260709 12:17:25.786725 31969 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.775084   975 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.738991   899 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.799197 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:25.783219   959 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.804165 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:25.784608 32044 raft_consensus.cc:606] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:25.787108  1004 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:25.892112  1000 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.799355   951 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.904151 32045 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:25.844323 32039 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:25.904039   951 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:17:25.844764 32626 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.015s	sys 0.022s
W20260709 12:17:25.905891 32626 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.015s	sys 0.023s
W20260709 12:17:25.906232 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 577897us
I20260709 12:17:25.800379  1005 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.802290 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:25.740044 32116 rpcz_store.cc:269] 0709 12:17:22.904384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.371377 (+466993us) service_pool.cc:224] Handling call
0709 12:17:24.471333 (+1099956us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aae9c 00007f22890aaa5b 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:25.451777 (+980444us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b54002fa154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:25.451937 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146217600}
I20260709 12:17:25.896454 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:25.796214   927 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.896168   975 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.895148   899 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.900636   959 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.902324   953 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.902766 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.918097 32120 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:25.843487   815 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.899461 32044 raft_consensus.cc:614] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:25.919860 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1446) took 1483 ms. Trace:
I20260709 12:17:25.842658 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:17:25.800948 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.20708e+06us
W20260709 12:17:25.907353 32626 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.502s (SENT)
I20260709 12:17:25.926990 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:25.911500 32119 raft_consensus.cc:2376] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3. Re-sending same reply.
W20260709 12:17:25.894343  1007 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.126s	user 0.001s	sys 0.001s
W20260709 12:17:25.914459 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 212977us
W20260709 12:17:25.929306 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 86730us
I20260709 12:17:25.925127 32118 rpcz_store.cc:276] 0709 12:17:24.436564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.367643 (+931079us) service_pool.cc:224] Handling call
0709 12:17:25.367905 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.368104 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:25.685606 (+317502us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.919178 (+233572us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.919251 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:25.919403 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.919550 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.919767 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31722,"prepare.queue_time_us":337929,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":8570,"thread_start_us":230264,"threads_started":1}
I20260709 12:17:25.930727 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:25.841914 32013 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.022s	sys 0.039s
W20260709 12:17:25.932700 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 790423us
I20260709 12:17:25.932727 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.909307 32112 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3. Re-sending same reply.
I20260709 12:17:25.933264 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.913302  1009 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:25.904178  1004 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:25.934894 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.936126 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.937369 32118 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:25.939319 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:25.939267 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:25.940304 32118 raft_consensus.cc:2368] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:25.940498 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.941710   987 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:25.942750   815 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:25.936090   955 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.943642 31969 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:25.943981   955 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:25.944444   955 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.945073   954 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:25.945464   954 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.945693   984 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.945799   954 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:25.928349 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:25.946882 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:25.947103 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.947414 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.935139   871 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.912434  1005 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.948840   909 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.945616 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.949404 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1459) took 1280 ms. Trace:
I20260709 12:17:25.960832 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.962459   741 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:25.962869 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.963380 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.964532 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.957799   815 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.965077 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.955698 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:25.960801 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.966820 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.967698 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:25.937656   901 raft_consensus.cc:2764] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:25.978487 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 291) took 1242 ms. Trace:
I20260709 12:17:25.965622 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.956390 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.953892   955 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:25.957890   984 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.978976 32659 rpcz_store.cc:276] 0709 12:17:24.736448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.857588 (+121140us) service_pool.cc:224] Handling call
0709 12:17:25.968499 (+1110911us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b5400403b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:25.978420 (+  9921us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122579200}
I20260709 12:17:25.968061   815 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.960417   781 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.960294   871 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:25.954905   951 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:25.981237 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53549.1us
I20260709 12:17:25.967429   783 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.976876   993 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.986804 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:25.981873  1018 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:17:25.982600   984 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:25.938555  1009 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:25.982131   952 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.993208 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.996026 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:25.996615 31967 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:25.997529 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:25.998207 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:25.998081 31967 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:25.992273   984 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.988520   993 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:25.992300   767 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.992285   968 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:26.015079  1021 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:17:26.004105 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.016259  1005 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.016532 31963 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:26.017822   955 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:26.009630 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:25.994122   952 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:26.009366 32116 raft_consensus.cc:2368] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.010540 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1457) took 1342 ms. Trace:
I20260709 12:17:26.013128  1030 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:26.013334   928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.284s	user 0.013s	sys 0.032s
I20260709 12:17:26.013422   767 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.012874 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.022580   954 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.023545   871 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:25.962121 32113 rpcz_store.cc:276] 0709 12:17:24.668695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.441816 (+773121us) service_pool.cc:224] Handling call
0709 12:17:25.684187 (+242371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:25.724973 (+ 40786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.726673 (+  1700us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:25.768420 (+ 41747us) log.cc:844] Serialized 26 byte log entry
0709 12:17:25.948613 (+180193us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:25.948669 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.949147 (+   478us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.949203 (+    56us) raft_consensus.cc:1685] finished
0709 12:17:25.949221 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.949350 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8576,"thread_start_us":147386,"threads_started":1,"wal-append.queue_time_us":157704}
I20260709 12:17:26.028512   899 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:26.012939   927 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.030299 32121 rpcz_store.cc:276] 0709 12:17:24.668022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.350828 (+682806us) service_pool.cc:224] Handling call
0709 12:17:25.351118 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.351339 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:25.804280 (+452941us) log.cc:844] Serialized 14 byte log entry
0709 12:17:26.009002 (+204722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.009051 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:26.010189 (+  1138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.010285 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.010465 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7839,"prepare.queue_time_us":335798,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":110,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":1924,"thread_start_us":628137,"threads_started":2}
I20260709 12:17:26.017881 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.040740 31967 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:26.010988   993 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:26.041491 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73857.2us
W20260709 12:17:26.032546   928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.304s	user 0.014s	sys 0.032s
I20260709 12:17:26.029425   952 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.048266 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.047339 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.049611 31967 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:26.050613 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.052874 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.053452   767 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.053622 31963 raft_consensus.cc:2368] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.054520 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.055047 31963 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.063264 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:26.052032 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.064177 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:26.064471 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.065196 31967 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.065867 31968 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.056710  1000 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.049080   899 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:17:26.066891   928 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:26.067291   928 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.067642   928 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.069271   984 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:26.070031 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 141750us
I20260709 12:17:26.066393 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:26.071516 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 85710us
I20260709 12:17:26.071625 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.072240 31968 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.073736 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.074540 32039 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:26.075374 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.076203 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.057124   951 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.078826 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.078131 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.079125 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.079516 31968 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.079702 31967 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:26.080431 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.075958 31964 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:26.073208 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.082392 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.083226 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.083645 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.083742 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.073314   999 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEARNER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.081820 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:26.075549  1000 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.085232 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.086838 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.084065 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.087273 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:26.088691   976 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.088775   920 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:26.089063   929 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:26.085865   892 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.089406   920 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:26.089902   892 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.090358   892 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.091616 31967 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:26.092312 32010 leader_election.cc:400] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.092648 32010 leader_election.cc:403] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.092936 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.093504 31967 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.094089 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.095798 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.096061   815 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.096479   815 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.096915  1005 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:26.097985   955 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.082330   975 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.090512   920 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.098716 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.095741   982 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:26.107784   982 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.107985 32013 leader_election.cc:400] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.108407   982 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.111861   984 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.112353 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.081527   958 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.114583   958 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:26.115239   958 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.082267   977 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:26.117560   977 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:26.118283   977 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.113885 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.106194   954 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.127839 31967 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:26.119133 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.096864   815 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.085992   964 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.087625   860 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.091786   768 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.089146   976 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:26.143091 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.144215 31968 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.146281 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.147364 31968 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:26.144263 31967 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.149665 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.120126   955 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:26.154642 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.150645 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:26.160441 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:26.135142   946 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.161733   963 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.142235   860 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:26.162158   963 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.162412   860 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.162566   963 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.164021 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 293) took 1123 ms. Trace:
I20260709 12:17:26.164959 31963 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:26.129886  1005 consensus_queue.cc:1243] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:26.167503 32657 raft_consensus.cc:2410] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:26.139599   803 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:26.168351 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70890.2us
I20260709 12:17:26.169610 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:26.150439   952 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:26.141309   993 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.099239  1000 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.177860 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.160529   955 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.151950   976 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.160638   893 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.143777   768 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.153352   964 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:26.164403 32656 rpcz_store.cc:276] 0709 12:17:25.040777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.163117 (+1122340us) service_pool.cc:224] Handling call
0709 12:17:26.163471 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.163688 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.163717 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.163759 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:26.163792 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.163847 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.163965 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.181761 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.193754 32659 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.170528   803 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.183012   869 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
W20260709 12:17:26.167362 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60789.3us
I20260709 12:17:26.200815 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.201741 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.202579  1036 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:26.203158  1036 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:26.204238  1036 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.205029   892 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.208616 32623 leader_election.cc:400] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:26.208910 32623 leader_election.cc:403] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:17:26.209542 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 95752.2us
I20260709 12:17:26.209568 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.209982 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:26.187836   993 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.198952  1037 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:26.191972 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60671.8us
I20260709 12:17:26.191886   768 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.212842 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.212911 32013 leader_election.cc:400] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.194855 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:26.196807   803 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.219100 32013 leader_election.cc:403] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.220247 32010 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.192819  1038 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.228955 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.232436 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.233655 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.212838  1033 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.206454  1009 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:26.236574 32121 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:26.218945   888 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:26.239202  1035 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:17:26.227388   991 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:25.740219 32117 rpcz_store.cc:276] 0709 12:17:24.357117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.054382 (+697265us) service_pool.cc:224] Handling call
0709 12:17:25.054700 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.054959 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:25.147468 (+ 92509us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.722615 (+575147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.722667 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:25.722704 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.722760 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.722859 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201471,"prepare.queue_time_us":94254,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":185458,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":13911,"thread_start_us":391365,"threads_started":1}
I20260709 12:17:26.212931   967 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.249111 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.232620   920 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.264081 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.263319 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.272253  1040 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.272955  1000 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.276945 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.261250   860 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.279762   860 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.263161   805 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.282534  1041 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.286863 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.290064 32659 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:26.241427  1038 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.293849 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.294615 32117 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:26.300076 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.297690 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:26.302614 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 93079.9us
I20260709 12:17:26.304206 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:26.305339 31938 connection.cc:256] RPC call timeout handler was delayed by -1.2301257389999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 12:17:26.306164 31938 connection.cc:256] RPC call timeout handler was delayed by -1.1239257390000148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:17:26.307418 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58420.9us
I20260709 12:17:26.308157 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.309432 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.312738 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.289440   805 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:26.263840   964 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.311690 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.313549 32117 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:17:26.409214 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99841.8us
W20260709 12:17:26.408291 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 198357us
I20260709 12:17:26.414722 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:26.333243 32623 connection.cc:256] RPC call timeout handler was delayed by -2.776857777000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 12:17:26.420077 32121 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:17:26.420097 32623 connection.cc:256] RPC call timeout handler was delayed by -2.6949577769998996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 12:17:26.399910   982 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:26.420826 32623 connection.cc:256] RPC call timeout handler was delayed by -2.5860577769999509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 12:17:26.413169 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101549us
W20260709 12:17:26.421283 32623 connection.cc:256] RPC call timeout handler was delayed by -2.5392577769998752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 12:17:26.421643 32623 connection.cc:256] RPC call timeout handler was delayed by -2.5032577770000444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
I20260709 12:17:26.422273 32623 leader_election.cc:400] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:26.422573 32623 leader_election.cc:403] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.422695   982 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:26.297276  1000 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.293892  1009 raft_consensus.cc:2764] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:26.431425 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.428444  1000 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.433629 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.433961  1000 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.434660  1000 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.297652  1042 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.312763  1049 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.398923  1038 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.400408   952 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:26.436915   805 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.435312   893 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:26.437772 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.335214   873 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.351227   892 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.437163 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.439090   959 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:26.440217 31938 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.564s (SENT)
W20260709 12:17:26.458695 31938 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.572s (SENT)
I20260709 12:17:26.424244   984 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.446070 32047 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:26.447301  1046 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:26.448750  1036 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.449635 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.299315  1040 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 12:17:26.457087 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.395613   955 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.442365 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.401142   977 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:26.459450 31938 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.633s (SENT)
I20260709 12:17:26.461244 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.403784   928 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.463266   959 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.463702  1042 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:26.476841 32039 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:26.470875 31870 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 241) took 5852 ms. Trace:
I20260709 12:17:26.481509 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.465433   958 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.482340 32039 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:26.483871 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.446630  1049 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.436691   963 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.496529 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.503353 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.505661 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.489394  1049 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.481071 31870 rpcz_store.cc:276] 0709 12:17:20.618624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.618827 (+   203us) service_pool.cc:224] Handling call
0709 12:17:26.470782 (+5851955us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:22.325110 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:22.439225 (+114115us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:22.439275 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:17:22.439302 (+    27us) tablet.cc:662] Decoding operations
0709 12:17:22.439712 (+   410us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:22.439788 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:22.439828 (+    40us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:22.440051 (+   223us) tablet.cc:701] Row locks acquired
0709 12:17:22.440080 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:17:22.440261 (+   181us) write_op.cc:270] Start()
0709 12:17:22.440415 (+   154us) write_op.cc:276] Timestamp: P: 1783599442440206 usec, L: 0
0709 12:17:22.440443 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:17:22.494123 (+ 53680us) log.cc:844] Serialized 530 byte log entry
0709 12:17:23.532543 (+1038420us) op_driver.cc:464] REPLICATION: finished
0709 12:17:24.616167 (+1083624us) write_op.cc:301] APPLY: starting
0709 12:17:24.616254 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:24.616360 (+   106us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:24.630191 (+ 13831us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:24.630472 (+   281us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:24.630647 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:24.630693 (+    46us) write_op.cc:312] APPLY: finished
0709 12:17:24.667970 (+ 37277us) log.cc:844] Serialized 23 byte log entry
0709 12:17:25.786532 (+1118562us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:25.795205 (+  8673us) write_op.cc:454] Released schema lock
0709 12:17:25.797109 (+  1904us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:25.896178 (+ 99069us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:25.899587 (+  3409us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":1083367,"apply.run_cpu_time_us":54297,"apply.run_wall_time_us":1211601,"num_ops":1,"prepare.queue_time_us":713643,"prepare.run_cpu_time_us":16764,"prepare.run_wall_time_us":579924,"raft.queue_time_us":751551,"raft.run_cpu_time_us":106177,"raft.run_wall_time_us":404530,"replication_time_us":1091896,"thread_start_us":2368372,"threads_started":5,"wal-append.queue_time_us":1418404,"wal-append.run_cpu_time_us":21555,"wal-append.run_wall_time_us":1350656}]]}
I20260709 12:17:26.459569   933 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.506917 32117 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:26.502486 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.522120 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.523710 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.524475 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.525038 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.525365 31964 raft_consensus.cc:2368] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:26.527165 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.527733 31963 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.528221   803 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.529047 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.529633 32112 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:26.530694 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.533164 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:26.535573  1045 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 12:17:26.536405 31963 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.536964   959 raft_consensus.cc:2764] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:26.537353 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.539474 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:26.540376 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.545984 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.547621 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.562639 32039 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:26.486285  1046 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.505764 32113 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:17:26.568673 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 146529us
I20260709 12:17:26.571686 32623 leader_election.cc:400] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:26.573137 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.537369 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.576180 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.576586 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.580054 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.580679 31963 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.530300   803 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.582984   946 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.566763 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.583559   946 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:26.583998   946 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.583050   955 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.561514   892 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.524516  1041 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.601444   963 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.604592  1047 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:26.604915 32040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1255) took 2288 ms (client timeout 1584 ms). Trace:
I20260709 12:17:26.605234  1047 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:26.605499 32040 rpcz_store.cc:269] 0709 12:17:24.316822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.326355 (+  9533us) service_pool.cc:224] Handling call
0709 12:17:26.604680 (+2278325us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:26.604847 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2650710272}
I20260709 12:17:26.607264   977 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.609544 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.611594 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.666070 31968 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:26.594437   815 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.613201   931 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:26.668139   931 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.669056   931 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.669986   815 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:26.667630 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.615350  1036 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:26.618723 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1146) took 1221 ms. Trace:
I20260709 12:17:26.617499   976 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.623816   946 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.627517  1003 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.630615 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.631551 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:26.631966  1051 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:17:26.632926 31938 leader_election.cc:336] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.927s (SENT)
I20260709 12:17:26.633531  1046 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.635694 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:26.637872 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:26.676011 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 99927.5us
I20260709 12:17:26.639060   928 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.640084   958 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.599524  1053 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.561949  1042 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:26.684726 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.645269   768 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.647624   909 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:26.648336 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 228084us
I20260709 12:17:26.650005 32118 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.710690 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.711551 32047 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:26.608721  1047 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.665136 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1266) took 1273 ms. Trace:
I20260709 12:17:26.671432 31972 rpcz_store.cc:276] 0709 12:17:25.396929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.397142 (+   213us) service_pool.cc:224] Handling call
0709 12:17:25.422115 (+ 24973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.813114 (+390999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.618490 (+805376us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400110b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:26.618665 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812739968}
I20260709 12:17:26.712939 32045 rpcz_store.cc:276] 0709 12:17:25.391676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.724311 (+332635us) service_pool.cc:224] Handling call
0709 12:17:26.664898 (+940587us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400217854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:26.665063 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275690368}
I20260709 12:17:26.713817 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.714273 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.714540 31972 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.715644 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.716831 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.717546 32045 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:17:26.676642 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 43751us
I20260709 12:17:26.671986 32117 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.681852 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.720005 32040 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:26.720861 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.682818 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.721637 31972 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:26.663316 31963 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:26.723254 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.723413 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.724021 31972 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.725373 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.696501 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.726627 32046 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:26.697976 31965 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 3. Current term is 4.
I20260709 12:17:26.728502 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.729336 31965 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.670267   931 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.698882 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.700176 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.684525 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.731336 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.721936 32043 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:26.731969 31968 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.721644 32040 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:26.675917   888 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.700693  1049 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.745867 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.755204 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.684307  1052 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.759042 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.759068 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.759991 32040 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:26.760958 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.733116   909 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.765131   967 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.766188 32047 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:26.767520 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:26.768054  1057 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:17:26.768194 31965 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.757588 31963 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:26.740408 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.772644 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.773360 32043 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:26.773639 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.774917 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.776044 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.776780 32043 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:26.766830 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.771075  1049 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.761472   976 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:26.774883  1058 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:17:26.763975   873 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.764213   953 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.785055 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.769518 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:26.786275 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 213216us
I20260709 12:17:26.786473 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.793632 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.794416 32043 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:26.793242 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:26.796252 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:26.796581 31968 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.787164 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.786690 31938 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.798283 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:26.798818 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:26.799250 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.798830 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.789037 31972 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:26.800164 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.801851 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.802199 31968 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.786648   967 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.807780 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.808606 32043 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:26.808604 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:26.809814 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.810391 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.810883 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.811640 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:26.811954 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.815586 32043 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:26.799515  1033 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.007s
I20260709 12:17:26.817497 32043 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.817716   873 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.817449   937 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:26.817543  1033 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
I20260709 12:17:26.819510 32043 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.820124 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.821419 32043 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:26.822551 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.808719 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.814880 31968 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:26.812876 32039 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:26.829530 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:26.816020 32040 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:26.806948  1036 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:26.816695 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.816885   860 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.794744  1037 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.802922  1052 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.825207 32038 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:26.805779  1046 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:26.801621   953 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.826542 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.857059 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.813831 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:26.829095 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.814765   967 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.824791  1060 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.838225 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:26.832206 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.840212 32040 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.841885 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:26.846784 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:26.851954 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.805961   899 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.854868 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.831293   933 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.858727 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:26.859479  1056 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:17:26.878515 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:26.855573  1052 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.879127 32039 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.855346 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 293) took 1115 ms. Trace:
I20260709 12:17:26.863229   967 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:26.863447 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54800.2us
I20260709 12:17:26.872905  1033 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.872632   955 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.829406 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.837144   768 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:26.872061 31968 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:26.876286 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:26.871129 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.817082  1059 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.897074   964 raft_consensus.cc:993] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:26.897543  1059 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:26.865897 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:26.897131 31963 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.874915 32043 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.899101  1033 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:26.899473  1033 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:26.854763   860 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.860672   893 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.861402 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.880746 32660 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:26.873968   933 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.837349 31965 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:26.872800   975 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:26.863662 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49911.6us
I20260709 12:17:26.881984  1052 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:26.906633 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45361.1us
I20260709 12:17:26.866829   946 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.860673  1037 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.855927   937 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:17:26.909304 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.856849  1046 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.910476 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:26.909574   937 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:26.890955 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:26.837175   984 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.892194 32042 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:26.893646  1054 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.893515 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.894177  1065 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:17:26.895973 31972 raft_consensus.cc:2368] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:26.910195   808 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.897739   964 raft_consensus.cc:1081] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:17:26.845906  1036 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.854287   888 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.880111 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.900331  1033 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.903488 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:26.860435   953 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.907370 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.890646 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:26.927006 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.906020   933 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.911048   767 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.911156   982 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:17:26.904690   860 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.911615   955 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:26.922097 32039 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:26.912657  1046 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.879845 32654 rpcz_store.cc:276] 0709 12:17:25.740217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.759672 (+ 19455us) service_pool.cc:224] Handling call
0709 12:17:25.979756 (+220084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.983665 (+  3909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.984783 (+  1118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.988661 (+  3878us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:25.991539 (+  2878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.995696 (+  4157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.844015 (+848319us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400403b50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:26.853901 (+  9886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427800448}
I20260709 12:17:26.919535 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:26.923996 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.925452 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:26.904116   893 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.912834   937 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.929088 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.915231 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.923122  1054 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:26.913995   984 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:26.912417 32044 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.878178  1060 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.951068   982 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:26.922998  1036 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.920194  1034 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:26.938975 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:26.929831   767 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.941864   893 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.961426 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:26.944136 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.942184 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.926259   808 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.947036  1005 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.940927  1000 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.947902 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:26.967410  1037 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:26.968334 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:26.968899 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1458) took 2300 ms. Trace:
I20260709 12:17:26.950127   955 consensus_queue.cc:1243] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:26.951550 31970 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:26.951538   982 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.928241   953 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.944937  1065 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:26.962989 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:26.963239 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.945179   975 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.936744   954 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.942829 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.964926  1005 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.952775   984 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.964882 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.956000  1033 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.945597 32652 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:26.966567 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.959667   767 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.970700 31965 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:26.964622   991 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.970746 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1147) took 1564 ms. Trace:
I20260709 12:17:26.974856  1065 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:26.969488 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:26.974175 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.979561 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:26.965638  1038 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.949380  1054 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:26.972504  1008 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.084s	user 0.004s	sys 0.001s
I20260709 12:17:27.038350  1030 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:26.982963 32041 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:27.041617  1030 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:27.042112  1030 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:26.983989 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:26.985051 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:26.988348 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:27.050359 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 66465.3us
I20260709 12:17:26.969321 32114 rpcz_store.cc:276] 0709 12:17:24.668362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.380951 (+712589us) service_pool.cc:224] Handling call
0709 12:17:25.949699 (+568748us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:25.949903 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.950200 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:25.950519 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 12:17:26.968211 (+1017692us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18002d8a40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a
0709 12:17:26.968504 (+   293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.968560 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:26.968623 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.968710 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.968840 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4463,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1309,"raft.queue_time_us":2348,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2741,"spinlock_wait_cycles":31004288,"thread_start_us":31799,"threads_started":1,"wal-append.queue_time_us":63024}
I20260709 12:17:26.967049  1034 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:26.980659   955 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:26.971560  1031 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.962985   815 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:17:26.959611  1062 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:17:26.983909  1005 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.998276  1060 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:26.998545 31963 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:26.972170 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.000741 31970 rpcz_store.cc:276] 0709 12:17:25.406041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.406282 (+   241us) service_pool.cc:224] Handling call
0709 12:17:26.951254 (+1544972us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:26.951399 (+   145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.970515 (+ 19116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.970696 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11716352}
I20260709 12:17:27.002022 32039 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.996829   937 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:26.981966 32042 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:27.057797 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.003672 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:27.043001   860 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.005268 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:27.007280  1000 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.060235 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:27.007143  1036 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.001070  1070 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:27.019980  1006 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.553s	user 0.004s	sys 0.008s
I20260709 12:17:27.023674 32651 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 1.
I20260709 12:17:27.052361  1071 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.025689   901 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.026532  1038 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:27.014092  1052 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.015055   975 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.054555 32038 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:26.955729   909 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.049440 32660 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.044628   808 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:27.064944   901 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:27.042594  1030 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:27.053694 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.024685 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.081067 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.049647   933 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.068157 31970 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:17:27.086056 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.079453  1030 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.088214  1030 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.089373 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.090386   888 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.090670 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1451) took 1071 ms. Trace:
I20260709 12:17:27.090816   888 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:27.091248   888 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.033953   767 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.042021  1065 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.077275 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1435) took 1083 ms. Trace:
I20260709 12:17:27.079173   901 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.066768   937 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.085520 32658 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:27.091034 31970 rpcz_store.cc:276] 0709 12:17:26.019453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.062229 (+1042776us) service_pool.cc:224] Handling call
0709 12:17:27.067952 (+  5723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.086549 (+ 18597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.090616 (+  4067us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.065071  1071 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.068153   953 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.072060 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.047262   982 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.097028   984 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.080555  1070 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:17:27.084285  1000 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.085858   975 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.086772   954 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.070963   955 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:17:27.092774 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.106237 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:27.094960   991 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.114481  1060 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.086143  1062 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:17:27.116185  1052 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.116878  1036 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.117610  1034 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.118331   982 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.118772 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:27.119517 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.146353 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:27.122175 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.102843  1065 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.150080 31970 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:27.151891 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1452) took 1131 ms. Trace:
I20260709 12:17:27.153211 31970 rpcz_store.cc:276] 0709 12:17:26.025761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.149564 (+1123803us) service_pool.cc:224] Handling call
0709 12:17:27.149878 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.151656 (+  1778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.151844 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.154742 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.157598 32114 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:17:27.159181 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.102154   933 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.144151 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:27.148070 32044 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.161259 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.162184 31972 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.162798 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1460) took 1134 ms. Trace:
I20260709 12:17:27.163101 31972 rpcz_store.cc:276] 0709 12:17:26.028213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.161734 (+1133521us) service_pool.cc:224] Handling call
0709 12:17:27.162014 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.162596 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.162751 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.148483 32113 raft_consensus.cc:2376] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4. Re-sending same reply.
I20260709 12:17:27.107369  1071 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:27.163699  1074 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:17:27.149595 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.164347 31972 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.164526 32038 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.164963 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1465) took 1117 ms. Trace:
I20260709 12:17:27.165293 31972 rpcz_store.cc:276] 0709 12:17:26.047194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.163886 (+1116692us) service_pool.cc:224] Handling call
0709 12:17:27.164183 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.164750 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.164907 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.139534   901 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.166589 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1444) took 1166 ms. Trace:
I20260709 12:17:27.127823   954 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.167089 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.138129  1000 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:27.141523 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48833.9us
W20260709 12:17:27.131692 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.090972  1038 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.108160   967 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.167322   954 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.311537   767 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.317024 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:27.324271   893 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.326786 32623 leader_election.cc:400] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3.
W20260709 12:17:27.416528 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89921.4us
I20260709 12:17:27.418089 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.387084 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.317939 32656 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
W20260709 12:17:27.290076 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.601s (SENT) [suppressed 26 similar messages]
W20260709 12:17:27.431334 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 141302us
W20260709 12:17:27.386216 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.431811 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:27.432844 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.434091 32623 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 3. Current term is 4.
I20260709 12:17:27.166949 31967 rpcz_store.cc:276] 0709 12:17:26.000668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.019402 (+1018734us) service_pool.cc:224] Handling call
0709 12:17:27.019836 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.020254 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.020294 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.020877 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.063921 (+ 43044us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.063968 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.064025 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.166412 (+102387us) raft_consensus.cc:1685] finished
0709 12:17:27.166430 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.166540 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1176,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":196,"thread_start_us":42625,"threads_started":1,"wal-append.queue_time_us":83304}
I20260709 12:17:27.185135 31972 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:27.280495 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.438521 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1156) took 1388 ms. Trace:
I20260709 12:17:27.438848 31972 rpcz_store.cc:276] 0709 12:17:26.050388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.165987 (+1115599us) service_pool.cc:224] Handling call
0709 12:17:27.166253 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.438260 (+272007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.438472 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.286845 32660 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
W20260709 12:17:27.443667 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56729.5us
I20260709 12:17:27.444655 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.488237 31967 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.489261 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1157) took 1438 ms. Trace:
I20260709 12:17:27.489779 31967 rpcz_store.cc:276] 0709 12:17:26.051291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.487622 (+1436331us) service_pool.cc:224] Handling call
0709 12:17:27.487983 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.488917 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.489194 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.143918  1033 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:27.490064 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:27.508749  1075 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.367s	user 0.000s	sys 0.000s
I20260709 12:17:27.510027 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1449) took 1503 ms. Trace:
I20260709 12:17:27.510429 32114 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.505829   860 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.510459 31965 rpcz_store.cc:276] 0709 12:17:26.006406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.026119 (+1019713us) service_pool.cc:224] Handling call
0709 12:17:27.026953 (+   834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.027357 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.027410 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.028309 (+   899us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.072605 (+ 44296us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.075878 (+  3273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.085025 (+  9147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.509784 (+424759us) raft_consensus.cc:1685] finished
0709 12:17:27.509814 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.509966 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":35263,"threads_started":1,"wal-append.queue_time_us":65293}
I20260709 12:17:27.513646 31965 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:27.103541 31969 rpcz_store.cc:276] 0709 12:17:25.989192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.991047 (+  1855us) service_pool.cc:224] Handling call
0709 12:17:27.003544 (+1012497us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:27.072480 (+ 68936us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9513728}
I20260709 12:17:27.515456 31969 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.516052 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 62) took 1039 ms. Trace:
I20260709 12:17:27.516389 31969 rpcz_store.cc:276] 0709 12:17:26.477037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.514927 (+1037890us) service_pool.cc:224] Handling call
0709 12:17:27.515253 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.515811 (+   558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.515993 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.517760 31969 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:27.519815  1046 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.002s
I20260709 12:17:27.514240 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1471) took 1090 ms. Trace:
I20260709 12:17:27.520519 31965 rpcz_store.cc:276] 0709 12:17:26.423518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.513023 (+1089505us) service_pool.cc:224] Handling call
0709 12:17:27.513379 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.514002 (+   623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.514190 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.527602 31965 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:27.523078 31968 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:27.530757 31965 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.533701 31965 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.440465  1036 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:27.548164 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58186.7us
I20260709 12:17:27.142963   975 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.525089   953 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.522426   893 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.528839 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 66) took 1048 ms. Trace:
W20260709 12:17:27.557605 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 123672us
I20260709 12:17:27.145131 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:27.567035 31967 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:27.581588 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.505231 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:27.587551  1071 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.506203   933 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.590523 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:27.590816 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:27.487720  1062 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.593823 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.169246   781 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.602062 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.601850 32121 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:27.565640  1036 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.606488 31972 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.166872  1037 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.384s
I20260709 12:17:27.504225   808 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.608803 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:27.611545 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.612951 32117 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.143204  1076 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.453s
I20260709 12:17:27.506930  1033 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:27.141772  1054 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.626612 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.613193   953 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.594965  1046 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.687896 32655 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.632747 32121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:27.690512 32121 raft_consensus.cc:606] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:27.690977 32121 raft_consensus.cc:614] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:27.693779  1046 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.627527  1030 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.638115   808 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.699440   808 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.700289   984 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.703735  1000 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.708002   871 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:27.708403   871 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:27.709314   937 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.601s	user 0.001s	sys 0.005s
I20260709 12:17:27.709229   871 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.714529   871 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:27.609246  1005 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.000s
I20260709 12:17:27.494077   888 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.612511  1062 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.658488  1054 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:27.629621 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1469) took 1562 ms. Trace:
I20260709 12:17:27.725421 31972 rpcz_store.cc:276] 0709 12:17:26.067213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.490161 (+1422948us) service_pool.cc:224] Handling call
0709 12:17:27.606246 (+116085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.629286 (+ 23040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.629548 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.663659 32116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:17:27.666266 32656 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:27.668252 32085 connection.cc:256] RPC call timeout handler was delayed by -1.2907622680002078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 12:17:27.674654 32112 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:27.675596  1036 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.677503 32659 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:17:27.731205 32659 pending_rounds.cc:85] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 2
I20260709 12:17:27.731097 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:27.731716 32659 pending_rounds.cc:107] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:27.732151 32659 raft_consensus.cc:2917] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:27.732671 32659 raft_consensus.cc:2889] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:27.678416  1033 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.680008 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.724812 31965 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:27.681733 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.683806   967 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:27.687032  1077 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:17:27.721725 32117 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:27.599902 31969 rpcz_store.cc:276] 0709 12:17:26.480643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.517238 (+1036595us) service_pool.cc:224] Handling call
0709 12:17:27.517573 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.518129 (+   556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.528768 (+ 10639us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:27.721086   901 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.555s	user 0.004s	sys 0.004s
I20260709 12:17:27.726763 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:27.736064 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 291485us
W20260709 12:17:27.737936 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57998.2us
W20260709 12:17:27.680675 32038 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.001s
I20260709 12:17:27.746266 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:27.726266  1062 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.776859 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.778738 32121 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.780373 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.782007 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.766278 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1454) took 1739 ms. Trace:
I20260709 12:17:27.787479 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:27.730791 32116 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:27.730986  1036 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:27.734576 31963 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.037s	sys 0.049s
I20260709 12:17:27.748453  1065 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.789697 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:27.791517 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.749871 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.794559 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.772929 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:27.807740 32117 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:27.807942 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58175.7us
I20260709 12:17:27.808820 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:27.809260 32085 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:27.799299   967 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.725020  1005 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:27.795979  1033 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.748057   953 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.730567 32115 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:27.806212 31972 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:27.784752 31970 rpcz_store.cc:276] 0709 12:17:26.026451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.154647 (+1128196us) service_pool.cc:224] Handling call
0709 12:17:27.154951 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.156800 (+  1849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.156830 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.157255 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.387966 (+230711us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.388023 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.388120 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.766024 (+377904us) raft_consensus.cc:1685] finished
0709 12:17:27.766053 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.766210 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4883,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":230345,"threads_started":1,"wal-append.queue_time_us":607359}
I20260709 12:17:27.800012 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:27.811915 32117 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.804204 32116 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.803670   984 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.833657   998 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:27.805730 31963 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.037s	sys 0.052s
I20260709 12:17:27.795635   808 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.775825 32038 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:27.834959 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1239) took 1782 ms. Trace:
I20260709 12:17:27.835408 32041 rpcz_store.cc:276] 0709 12:17:26.052195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.054013 (+  1818us) service_pool.cc:224] Handling call
0709 12:17:26.665101 (+611088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.040883 (+375782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.042103 (+  1220us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400217854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:27.834716 (+792613us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b5400217850. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:27.834872 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127962240}
I20260709 12:17:27.836560 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:27.837837 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1470) took 1417 ms. Trace:
I20260709 12:17:27.841442   951 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.842089 32118 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.842401   951 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.842864   951 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:27.843784   951 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.844877  1071 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.845010   951 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.820422 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.846177 32113 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.846459 32112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:27.847111 32112 raft_consensus.cc:606] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:27.810407   982 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.845024  1034 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:27.817968   945 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.850957  1034 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:27.852651 32653 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.805845  1036 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.853427 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 298) took 1011 ms. Trace:
I20260709 12:17:27.807283  1065 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:27.858582  1065 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.858774   999 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:27.859210   999 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:27.860150   999 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.826350 31970 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.862646 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:27.863377 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1276) took 1179 ms. Trace:
I20260709 12:17:27.863430 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:27.863916 32045 rpcz_store.cc:276] 0709 12:17:26.683512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.718086 (+ 34574us) service_pool.cc:224] Handling call
0709 12:17:27.862493 (+1144407us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:27.863323 (+   830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11829120}
I20260709 12:17:27.864682 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1487) took 1016 ms. Trace:
I20260709 12:17:27.865026 31970 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.864096  1036 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.867887  1036 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.867972  1005 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.838902   871 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:27.868767   871 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.869185   871 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.871337 31964 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:27.821180  1082 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:27.873155 31964 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.873561  1082 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:27.874607  1082 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:27.815738 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 189232us
I20260709 12:17:27.823921   953 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.826068   909 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.811007   901 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.814706   910 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.788746   954 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.903836   954 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.853868 32653 rpcz_store.cc:276] 0709 12:17:26.842212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.854004 (+ 11792us) service_pool.cc:224] Handling call
0709 12:17:27.852205 (+998201us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:27.852439 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.853116 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.853331 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8029440}
I20260709 12:17:27.905413 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.905900 31964 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.907227 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:27.908499 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:27.909093 31964 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.910089 31972 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.892149   953 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.912314 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1175) took 1007 ms. Trace:
I20260709 12:17:27.912686 31972 rpcz_store.cc:276] 0709 12:17:26.905109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.820789 (+915680us) service_pool.cc:224] Handling call
0709 12:17:27.821095 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.910560 (+ 89465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.912073 (+  1513us) spinlock_profiling.cc:243] Waited 88.8 ms on lock 0x7b54000b5c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:27.912250 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204182272}
I20260709 12:17:27.914057 31972 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.826866  1037 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.827615   942 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.951606  1031 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:27.952160  1031 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.828300   967 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:27.956969   967 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.957340   967 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:27.958082   967 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.958596   964 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.829020   933 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:27.959821   933 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:27.959403   964 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.960628   933 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.961790   933 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.829689 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.852138  1034 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.962458  1060 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.865602   893 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:27.962884 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 133303us
I20260709 12:17:27.963686   893 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.963724 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.964402   893 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:27.964745   893 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:27.964963 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:27.965528   893 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.965852   954 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.965793 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.825410   888 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:27.838230 31967 rpcz_store.cc:276] 0709 12:17:26.420943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.491006 (+1070063us) service_pool.cc:224] Handling call
0709 12:17:27.491285 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.601131 (+109846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.837593 (+236462us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b54004556d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:27.837777 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173778816}
I20260709 12:17:27.970221 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 296) took 1128 ms. Trace:
I20260709 12:17:27.970615 32658 rpcz_store.cc:276] 0709 12:17:26.841524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.841638 (+   114us) service_pool.cc:224] Handling call
0709 12:17:27.970040 (+1128402us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54005eabd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:27.970169 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506830080}
I20260709 12:17:27.973608 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.974231 32045 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.975597  1062 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:27.977063 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.978623  1062 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.978981 32651 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.979087 32041 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.979561 32115 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.980997 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:27.865125 32114 rpcz_store.cc:276] 0709 12:17:26.848229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.161747 (+313518us) service_pool.cc:224] Handling call
0709 12:17:27.162014 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.510878 (+348864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.864450 (+353572us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b540012bbd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:27.864624 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615299456}
I20260709 12:17:27.901736 32112 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:27.982604 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1279) took 1282 ms. Trace:
I20260709 12:17:27.914675 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:27.983989 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1438) took 1993 ms (client timeout 2186 ms). Trace:
I20260709 12:17:27.984107   815 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:27.826727  1011 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 2.082s	user 0.014s	sys 0.005s
I20260709 12:17:27.982961 32047 rpcz_store.cc:276] 0709 12:17:26.700466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.712388 (+ 11922us) service_pool.cc:224] Handling call
0709 12:17:27.982422 (+1270034us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b540032ad54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:27.982551 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19721984}
I20260709 12:17:27.984997   815 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:27.982020 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.985410   815 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:27.926606 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.941530 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:27.987258 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45813.4us
I20260709 12:17:27.987396 32047 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.988816   920 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.989085 32046 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:17:27.989529 32046 pending_rounds.cc:85] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 2
I20260709 12:17:27.989789 32046 pending_rounds.cc:107] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:27.990116 32046 raft_consensus.cc:2917] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:27.990571 32046 raft_consensus.cc:2889] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:27.990887 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1172) took 1175 ms. Trace:
I20260709 12:17:27.990998  1000 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.991755 32041 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.992187  1004 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:27.992635  1004 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:27.993494  1004 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:27.994907   964 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:27.954783   955 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:27.995675 31971 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.995815 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.996536 32040 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.923101   975 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.997175   975 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.997614   975 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:27.967852 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 386376us
I20260709 12:17:27.966722   954 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:27.999923 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.968406   999 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.967186 32043 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.968437   803 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.981143 32652 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.000976   803 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.995100 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1472) took 1568 ms. Trace:
I20260709 12:17:27.983166 32038 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.001657 31968 rpcz_store.cc:276] 0709 12:17:26.426351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.914992 (+488641us) service_pool.cc:224] Handling call
0709 12:17:27.994920 (+1079928us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54000b6654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:27.995048 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332185344}
I20260709 12:17:28.006466 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:27.921321   953 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.012064 31870 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 3, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:27.959219   967 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:28.017796 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.968106 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.018558 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:28.018937 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50913.4us
I20260709 12:17:28.019826 32045 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:27.823357 32117 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.962977  1060 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:27.891043 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 91088us
I20260709 12:17:28.010459 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.011052 32120 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:27.997658 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1173) took 1171 ms. Trace:
I20260709 12:17:28.000509  1034 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.020071 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.024144 32040 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:28.029161 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1493) took 1179 ms. Trace:
I20260709 12:17:28.029563 32120 rpcz_store.cc:276] 0709 12:17:26.849928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.060999 (+211071us) service_pool.cc:224] Handling call
0709 12:17:27.905268 (+844269us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:28.029090 (+123822us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15472384}
I20260709 12:17:28.028903 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:28.016718 31870 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:28.001492   803 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.027186 31971 rpcz_store.cc:276] 0709 12:17:26.826706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.680929 (+854223us) service_pool.cc:224] Handling call
0709 12:17:27.995486 (+314557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.996077 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.997482 (+  1405us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54000b6650. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:27.997607 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722713472}
I20260709 12:17:28.026217 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:28.037768 32046 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:27.985301 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:28.039736 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:28.045569 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 151753us
I20260709 12:17:28.047016 31870 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:27.991350 31965 rpcz_store.cc:276] 0709 12:17:26.815225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.587567 (+772342us) service_pool.cc:224] Handling call
0709 12:17:27.587947 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.749389 (+161442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.990637 (+241248us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b54001e9bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:27.990827 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119622016}
W20260709 12:17:28.048888 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.014225  1033 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:27.922264  1065 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.049949 31870 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 2, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:28.056620 31971 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.060798 32651 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:17:28.017614 32115 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:28.045456  1062 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.036514   909 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:28.028870   954 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.045662  1052 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:28.095996 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:28.064298 31870 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:28.089411 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:28.090785 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:27.984418 31966 rpcz_store.cc:269] 0709 12:17:25.990217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.991579 (+  1362us) service_pool.cc:224] Handling call
0709 12:17:27.914562 (+1922983us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:27.983903 (+ 69341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10677504}
I20260709 12:17:28.051694   781 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.032894 31964 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:28.084340 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:28.095089 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.052479   998 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:28.068378 32651 pending_rounds.cc:85] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 4
I20260709 12:17:28.057593   767 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.051848  1037 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:28.104954 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.115442 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:27.986232   815 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.065466 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.069118   893 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.045835  1003 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:17:28.046080   976 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:17:28.057538   952 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:28.057644  1071 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:17:28.101315   909 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:28.061228   873 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:17:28.061007   869 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:17:28.132272 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.123461 32651 pending_rounds.cc:107] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:28.158267 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.136605  1038 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:28.137678   937 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:28.138553   899 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.136596 32119 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.106431   901 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.066617  1033 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.118971  1062 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:28.108649  1036 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:17:28.107252   910 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:17:28.123231  1052 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:28.121335   888 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.128142  1030 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.157935 32651 raft_consensus.cc:2889] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:28.155836 32041 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.133780   954 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.130348  1048 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:17:28.163893  1054 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.132411   860 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:17:28.168658 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:28.173198  1086 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.130894  1037 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:28.220252 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 187965us
W20260709 12:17:28.227784 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 188139us
W20260709 12:17:28.229118 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93682.8us
I20260709 12:17:28.261543 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.173341   901 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.200673  1003 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:28.272835 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:28.299382 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 170004us
I20260709 12:17:28.264117 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.224640   767 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:28.311519 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50114.9us
I20260709 12:17:28.228591  1004 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.311838 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1177) took 1365 ms. Trace:
W20260709 12:17:28.314711 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 87754.1us
I20260709 12:17:28.346551 31970 rpcz_store.cc:276] 0709 12:17:26.946414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.865842 (+919428us) service_pool.cc:224] Handling call
0709 12:17:27.866118 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.877396 (+ 11278us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:28.144496 (+267100us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.311517 (+167021us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:28.311568 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:28.311613 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.311674 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.311785 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48243,"prepare.queue_time_us":300931,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":49439,"thread_start_us":308633,"threads_started":2}
I20260709 12:17:28.346580 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:28.340605 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.349162 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.270805  1104 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:28.337998 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65262us
I20260709 12:17:28.312757   998 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:28.343981 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 79954.3us
I20260709 12:17:28.335503  1084 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.315182  1096 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.279285  1003 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:28.342470  1054 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.365414 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:28.369246 32657 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:28.352489   952 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:28.370149   873 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.370782   976 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:28.371096   873 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:28.372254   873 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.374099   982 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.355993  1052 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.376410 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.353688  1065 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:28.354393  1104 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:28.384683 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.385485   954 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.385326   910 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.385619   955 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:28.386729   910 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.387125   860 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.387704   910 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.388161   910 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:28.381256 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 118786us
W20260709 12:17:28.368247 32116 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.537s	user 0.005s	sys 0.005s
I20260709 12:17:28.367679   953 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:28.403112  1102 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.403752  1102 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:28.405092  1048 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.405539  1048 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:28.406356  1048 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.379910   952 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:28.407542  1048 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.407908   952 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.409535 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.410524  1005 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:28.411727 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:28.412951 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:28.302249  1062 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:28.432687 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44139.7us
I20260709 12:17:28.433895 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:28.389928   945 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.440358 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:28.441442 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.441452 31972 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:28.391709  1060 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:28.448998 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:28.452577 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:28.445374 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 86461.1us
I20260709 12:17:28.453722 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.445756 31968 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:28.448889 32042 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.449273 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.458043  1095 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:17:28.405108  1102 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.458930 31966 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:28.459487  1102 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.397718   767 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.446666 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:28.470672 31972 raft_consensus.cc:2376] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4. Re-sending same reply.
I20260709 12:17:28.382109  1054 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.457535 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:28.457907 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.460340 31970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71e0cf115549454c894335e70c43843a"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:17:28.479265 31970 raft_consensus.cc:606] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:28.451110 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.467803 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.469079 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:28.471603 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.455456 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.375583  1096 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.478122 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.511978 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.513551 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.460062 31964 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:28.514400 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 105007us
I20260709 12:17:28.480196 32047 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.518486 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.514222 32046 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.516084 31968 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 2: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:28.451149   808 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.473508   920 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.434412  1105 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.521252 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:28.525230 31972 raft_consensus.cc:2376] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3. Re-sending same reply.
W20260709 12:17:28.522975 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.523603 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.419567 32116 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.525820 31968 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:28.528205 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.526676 31972 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:28.460139 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.529701 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.530019 31972 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:28.530278 32039 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.458964  1095 consensus_queue.cc:1243] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:28.471992  1060 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:28.539917 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.540452 32046 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:28.514855  1036 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:28.518718  1033 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:28.550028 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.548791 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.552251 32046 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:28.479826  1104 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.546416  1110 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:28.558806 32118 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:28.542737  1094 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 12:17:28.573598  1054 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:28.533211  1107 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:28.567523 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.558001 31966 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.569546   976 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.472290   998 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.570655  1110 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:28.572155   871 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.000s
I20260709 12:17:28.532323   808 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:28.563705 31972 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:28.586030 32043 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:28.531747  1000 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 12:17:28.577731  1089 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
W20260709 12:17:28.575963   975 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.000s
I20260709 12:17:28.581131 32041 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:28.527508   920 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.534904   803 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.668416 32039 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:28.574496 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:28.644549 31964 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:28.646215   933 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.004s
I20260709 12:17:28.697220  1049 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.704456  1049 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.706015  1096 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:28.706451  1087 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 12:17:28.678023   975 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.707702 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 103) took 1549 ms. Trace:
I20260709 12:17:28.585647  1037 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:28.652442 32113 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.806s	user 0.008s	sys 0.002s
I20260709 12:17:28.708101 32044 rpcz_store.cc:276] 0709 12:17:27.158809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.163087 (+  4278us) service_pool.cc:224] Handling call
0709 12:17:27.163441 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.197192 (+ 33751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.208530 (+ 11338us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.673714 (+465184us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b2c0005c228. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:27.674116 (+   402us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.928748 (+254632us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b1800391180. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22900a358f 00007f22890e2571 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd
0709 12:17:28.048082 (+119334us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:28.057944 (+  9862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.065507 (+  7563us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.707485 (+641978us) raft_consensus.cc:1685] finished
0709 12:17:28.707504 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.707642 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22951,"raft.queue_time_us":32822,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":36292,"spinlock_wait_cycles":38132864,"thread_start_us":350089,"threads_started":1,"wal-append.queue_time_us":564537}
I20260709 12:17:28.709637 32113 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:28.710978 32112 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:17:28.711199  1091 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
I20260709 12:17:28.589195  1107 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.712446 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1174) took 1885 ms. Trace:
I20260709 12:17:28.697019 32040 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.559090  1111 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
W20260709 12:17:28.714219   893 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.000s
I20260709 12:17:28.715039  1107 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:28.561482  1036 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.701910   803 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:28.696771   808 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:28.713011 32116 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:28.653870   953 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.736088 32657 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:28.677084  1097 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:28.671847   871 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.730708 32039 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.729346  1030 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.748046 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.721318  1060 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:28.749574 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:47764 (request call id 1498) took 1795 ms. Trace:
I20260709 12:17:28.743750  1097 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:28.750510  1034 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.723s	user 0.003s	sys 0.008s
I20260709 12:17:28.746744   976 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:28.748108  1115 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.777158   803 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.794391   803 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:28.794824   803 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:28.795584   803 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.796677   803 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:28.797797   999 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.797s	user 0.008s	sys 0.003s
I20260709 12:17:28.777828  1000 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:28.798683  1000 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.799726   909 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.800693  1000 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:28.800885   909 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.776777  1060 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.802380 31966 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:28.777490   871 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:28.813009  1034 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:28.811316  1036 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:28.812856  1090 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.510s	user 0.000s	sys 0.006s
I20260709 12:17:28.748191  1030 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:28.757427 32116 rpcz_store.cc:276] 0709 12:17:26.943177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.663461 (+720284us) service_pool.cc:224] Handling call
0709 12:17:28.738225 (+1074764us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:28.816061   781 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.697s	user 0.012s	sys 0.000s
I20260709 12:17:28.756537 32044 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:28.785990  1115 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:28.841382 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.825739  1107 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.807621   999 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:17:28.860219  1105 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:28.815893  1102 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
W20260709 12:17:28.877152 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.858457   781 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.854219 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:28.861141  1097 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:28.896777  1115 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.824237  1113 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
W20260709 12:17:28.918852 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 449815us
I20260709 12:17:28.919675 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:28.930336 32121 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.863356  1107 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:28.940532 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 416946us
I20260709 12:17:28.942106 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.943704 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:28.944087 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:28.920975  1031 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.968s	user 0.004s	sys 0.009s
I20260709 12:17:28.945896 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:28.864305   976 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.919518   975 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:28.924357   781 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:28.951475  1031 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:28.956185 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 388746us
I20260709 12:17:28.953949 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:28.934227  1115 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.946606  1111 consensus_queue.cc:1243] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:28.959776 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:28.956616  1110 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:28.965433 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:28.981700 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:28.987437 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.713048 31969 rpcz_store.cc:276] 0709 12:17:26.827331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.805776 (+978445us) service_pool.cc:224] Handling call
0709 12:17:27.824339 (+ 18563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.891964 (+ 67625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.892007 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.916284 (+ 24277us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.264889 (+348605us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:28.357645 (+ 92756us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.367917 (+ 10272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.380260 (+ 12343us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.712106 (+331846us) raft_consensus.cc:1685] finished
0709 12:17:28.712124 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.712357 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53510,"prepare.queue_time_us":73925,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":73724,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":34401,"thread_start_us":302812,"threads_started":1,"wal-append.queue_time_us":309473}
I20260709 12:17:28.993251 32121 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:28.957468   781 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.941172   871 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:17:28.997258 32010 leader_election.cc:400] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:28.996806 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:28.998241 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:28.950214  1118 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.964284  1033 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:28.992640  1048 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.004s
I20260709 12:17:28.984972  1034 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.981220  1122 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.993764 32116 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:28.999166   781 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.023305  1134 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.024014  1128 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.961952  1132 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:17:29.014106  1097 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.001502  1130 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.019229   999 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:29.067713 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.071040  1114 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.042034  1052 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:29.076944 31972 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:29.038693  1111 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:17:29.020076  1034 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:28.991318  1105 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.065971   871 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.020251  1033 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.066305  1097 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:29.067539  1134 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:29.075420   999 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.092720  1121 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 12:17:29.094678 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.094586  1034 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.095918 32652 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:29.082334  1130 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.056701  1123 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:17:29.067569   964 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:17:29.108107 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:29.108963 31972 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:29.089416 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:29.094238   999 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.102058 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:29.084565  1052 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.066133  1110 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:29.124505  1125 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:17:29.112571  1054 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.116287 32116 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:29.100625   871 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.127686  1114 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:29.128717 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:29.129455 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:29.131239 32114 pending_rounds.cc:85] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 1
I20260709 12:17:29.131966  1097 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.115027  1107 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.133191  1134 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.134173 32046 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:29.134368 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:29.135684 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68043us
I20260709 12:17:29.064179  1135 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:17:29.136384  1033 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:29.189993  1138 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:29.192041 31965 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:29.195446 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:29.195960 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.197105 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:29.197566  1135 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.183207 32114 pending_rounds.cc:107] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:29.198973  1135 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.199069 32114 raft_consensus.cc:2889] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.199466  1135 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:29.197101 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.123241  1111 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.202701 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:29.182049 32658 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:29.203728 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:29.185698 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.205077 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:29.085471  1120 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:17:29.148108  1031 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.200351  1135 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.217831 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.214164 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:29.211598 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77329.2us
I20260709 12:17:29.219085 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:29.217907 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:29.186002  1114 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.225436 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:29.224978 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.224532 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:29.201915   803 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.220253 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.233165 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:29.234300 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.188098   999 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:29.235606 32117 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:29.182116  1054 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.217188  1037 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.210503  1111 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:29.223547  1129 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.220692  1118 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.240975 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.223352  1122 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.235926  1142 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.229202  1128 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.228945  1107 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.255913 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.236285  1127 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.236100  1135 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.237308  1120 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:29.238624 32038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:29.271870 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:29.250497   869 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.290176 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:29.296034 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.300303 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.247067   999 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.264708  1118 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:29.285508  1132 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.288532  1123 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.265596 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.288761  1120 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:29.251899  1037 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:29.302057 32654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.144s	user 0.035s	sys 0.047s
I20260709 12:17:29.302284  1126 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:29.302253 32038 raft_consensus.cc:606] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:29.298790  1127 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:29.335479  1127 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.307508  1121 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.331925 32116 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:29.333202   942 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.317641  1142 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:29.343282 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.350488 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55128.6us
I20260709 12:17:29.316830  1049 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:29.337209  1085 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.210s	user 0.012s	sys 0.006s
I20260709 12:17:29.329795  1046 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.331010 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:29.328847 32654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.175s	user 0.036s	sys 0.048s
I20260709 12:17:29.343643 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1494) took 2493 ms. Trace:
I20260709 12:17:29.326078  1120 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.341275   937 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.358997 32119 rpcz_store.cc:276] 0709 12:17:26.850207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.656225 (+806018us) service_pool.cc:224] Handling call
0709 12:17:27.656522 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.164623 (+508101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.343387 (+1178764us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540010efd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:29.343560 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103525248}
I20260709 12:17:29.348462  1005 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.361575 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:29.360704 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:29.344671  1126 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:29.360239 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:29.371918 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:29.372632 31968 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:29.351245  1031 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:29.378186  1031 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.348955  1121 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.355347  1046 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:29.377563 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 306) took 1869 ms. Trace:
I20260709 12:17:29.378506 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:29.340879  1132 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:17:29.355736   871 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.376001 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:29.353236   942 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.375932  1126 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.381801 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.362818 32115 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:29.383322  1031 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.385427  1121 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:29.386859  1149 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:17:29.387635 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 305) took 1433 ms. Trace:
I20260709 12:17:29.387290   781 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.388607 32660 rpcz_store.cc:276] 0709 12:17:27.508714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.554441 (+ 45727us) service_pool.cc:224] Handling call
0709 12:17:27.555165 (+   724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.555641 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.555678 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.570752 (+ 15074us) log.cc:844] Serialized 548 byte log entry
0709 12:17:27.810342 (+239590us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:27.810381 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.810445 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:29.222436 (+1411991us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:29.377198 (+154762us) raft_consensus.cc:1685] finished
0709 12:17:29.377229 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.377492 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11272576,"thread_start_us":239267,"threads_started":1,"wal-append.queue_time_us":498538}
I20260709 12:17:29.316745  1111 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.396183 32119 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:29.360273  1128 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:17:29.360034  1049 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:29.411783 32046 raft_consensus.cc:2368] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3.
I20260709 12:17:29.372737   937 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:29.437755 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56001.5us
I20260709 12:17:29.395777   808 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:29.440162   808 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.395267  1130 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.443727 32658 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.373729  1005 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.374071  1118 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.455394 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.455969 32119 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:29.456979 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:29.444000  1105 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.459805 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:29.442432   808 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.446786  1163 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.462684 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.448673 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:29.475447 32658 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:17:29.475991 32658 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 3
I20260709 12:17:29.476405 32658 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 12:17:29.476786 32658 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:17:29.477341 32658 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.483106   942 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.484825   781 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.486785   937 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.488068 31969 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:29.489460 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:29.478899 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:29.491021  1142 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.490231 31969 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.491881  1005 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.456424  1054 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.464301  1121 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.492980  1121 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.493381  1121 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.497555 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:29.497929   815 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.462989   999 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.500264 32658 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:29.501463 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:29.510614 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:29.511229 32651 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:29.512423 32011 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:29.473639  1130 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.478854 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:29.507483  1123 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:17:29.515404  1049 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.515708  1037 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.451575  1046 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.410241 32654 rpcz_store.cc:276] 0709 12:17:27.954346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.966671 (+ 12325us) service_pool.cc:224] Handling call
0709 12:17:27.967018 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.034709 (+ 67691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.048321 (+ 13612us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.118272 (+ 69951us) log.cc:844] Serialized 26 byte log entry
0709 12:17:29.183295 (+1065023us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f22890c50ac 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b
0709 12:17:29.373229 (+189934us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:29.380079 (+  6850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.387319 (+  7240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:29.387370 (+    51us) raft_consensus.cc:1685] finished
0709 12:17:29.387385 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.387577 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61009,"raft.queue_time_us":60701,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":18401,"spinlock_wait_cycles":140742912,"thread_start_us":1224313,"threads_started":1,"wal-append.queue_time_us":1203728}
I20260709 12:17:29.518738  1107 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:29.521533  1122 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.522149  1049 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.522437   901 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.530922  1049 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.530812 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:29.479159  1163 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.550825  1084 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:29.545938  1115 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.564s	user 0.005s	sys 0.008s
I20260709 12:17:29.531137  1107 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.543929  1122 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.556245  1084 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.563069  1084 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.563709  1084 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.564196  1084 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:29.554452   770 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:29.572221  1003 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.573899  1003 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.567339  1120 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.561172  1129 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.585359 31969 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:29.589046 31969 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:29.592270  1101 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.592852  1101 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:29.569193   770 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:29.594048   990 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.568132  1114 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:29.601773   990 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.605701   990 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.607050 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:29.607769  1080 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.403s	user 0.005s	sys 0.006s
I20260709 12:17:29.609050 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:29.609894  1120 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.610404 32042 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:29.580833  1084 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.609992 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1506) took 1665 ms. Trace:
I20260709 12:17:29.615665 32118 rpcz_store.cc:276] 0709 12:17:27.944658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.945399 (+   741us) service_pool.cc:224] Handling call
0709 12:17:27.945703 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.559344 (+613641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.609668 (+1050324us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400149bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:29.609904 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409456896}
W20260709 12:17:29.633351 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.621939 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:17:29.660939 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 269748us
I20260709 12:17:29.662256  1166 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.662984  1166 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.665685  1166 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.681563 31972 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.3-2.4]
W20260709 12:17:29.700596  1071 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.524s	user 0.001s	sys 0.013s
I20260709 12:17:29.611483   770 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.621989  1163 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:29.739763  1071 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.566s	user 0.002s	sys 0.014s
I20260709 12:17:29.709925  1054 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.751658 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1274) took 1703 ms. Trace:
I20260709 12:17:29.752215 32040 rpcz_store.cc:276] 0709 12:17:28.048716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.057630 (+  8914us) service_pool.cc:224] Handling call
0709 12:17:28.057917 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.713282 (+655365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.751433 (+1038151us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54000334d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:29.751604 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469220224}
I20260709 12:17:29.753795 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:29.739513   901 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.758255 32040 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:29.759790 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:29.593895  1101 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.802381 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1450) took 3791 ms. Trace:
I20260709 12:17:29.756716  1054 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.789557   901 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:29.874375 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1507) took 1928 ms. Trace:
I20260709 12:17:29.815321  1083 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:17:29.876008  1142 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:29.897958  1101 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.817104 31963 rpcz_store.cc:276] 0709 12:17:26.011744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.057452 (+1045708us) service_pool.cc:224] Handling call
0709 12:17:27.059055 (+  1603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.068105 (+  9050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.071224 (+  3119us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.127795 (+ 56571us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.834467 (+706672us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.834521 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.834607 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:29.606977 (+1772370us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:29.750938 (+143961us) raft_consensus.cc:1685] finished
0709 12:17:29.754912 (+  3974us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.802260 (+ 47348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47836544,"thread_start_us":671992,"threads_started":1,"wal-append.queue_time_us":986337}
I20260709 12:17:29.934054 32654 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.816437  1114 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.816195  1107 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.935246  1120 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.926437  1049 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.937650  1142 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:29.960094  1175 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:17:29.945747  1122 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:29.960331  1054 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.960453   901 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:29.962661   920 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.962786  1142 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:29.946933  1120 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.968976  1121 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:29.973948  1049 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:30.005604 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:29.942181 31971 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.009656 32040 raft_consensus.cc:2368] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:29.929683 32112 rpcz_store.cc:276] 0709 12:17:27.944978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.983108 (+ 38130us) service_pool.cc:224] Handling call
0709 12:17:28.710703 (+727595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.711400 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.802225 (+1090825us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54001482d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:29.872883 (+ 70658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672440064}
I20260709 12:17:30.015934 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1515) took 1091 ms. Trace:
I20260709 12:17:30.017858 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:29.975492  1121 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.043586 32114 rpcz_store.cc:276] 0709 12:17:28.924054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.928166 (+  4112us) service_pool.cc:224] Handling call
0709 12:17:29.130668 (+202502us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:29.130933 (+   265us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:29.200069 (+ 69136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.200117 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:29.200829 (+   712us) log.cc:844] Serialized 52 byte log entry
0709 12:17:29.613396 (+412567us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b18001466e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:17:29.748832 (+135436us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:29.750679 (+  1847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.755088 (+  4409us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.015220 (+260132us) raft_consensus.cc:1685] finished
0709 12:17:30.015263 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.015643 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5177,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9374,"spinlock_wait_cycles":30229120,"thread_start_us":126537,"threads_started":1,"wal-append.queue_time_us":745463}
I20260709 12:17:30.023268 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:30.048161 32658 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.048355 32113 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.049664 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1510) took 1127 ms. Trace:
I20260709 12:17:30.026139 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1192) took 1571 ms. Trace:
I20260709 12:17:30.027114 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.035637 32653 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:17:30.052639 32120 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.050874 31970 rpcz_store.cc:276] 0709 12:17:28.450651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.549205 (+ 98554us) service_pool.cc:224] Handling call
0709 12:17:28.549504 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.560979 (+ 11475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.561011 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:29.751508 (+1190497us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:29.757268 (+  5760us) log.cc:844] Serialized 26 byte log entry
0709 12:17:29.940220 (+182952us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:29.940275 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.940383 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.021433 (+ 81050us) raft_consensus.cc:1685] finished
0709 12:17:30.021482 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.021954 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154039,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":111,"spinlock_wait_cycles":3335424,"thread_start_us":61418,"threads_started":1,"wal-append.queue_time_us":152256}
I20260709 12:17:30.054345   920 raft_consensus.cc:993] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:30.054550 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.055069   920 raft_consensus.cc:1081] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:30.055544 31970 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:30.057199   860 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.059283 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:30.059836 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:30.060251 31964 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.059587   860 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.051466 32653 pending_rounds.cc:85] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 4
I20260709 12:17:30.064960 32047 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:30.063465 32653 pending_rounds.cc:107] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:30.075234 32653 raft_consensus.cc:2889] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:30.070648 32652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:17:30.079304   951 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.079592 32652 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:30.079854   951 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:30.080147 32652 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.088320   951 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.089274   860 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:30.084112 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.086833 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.085039 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.093641 32117 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:30.094981  1060 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:30.095487  1060 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.095949   991 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:30.097122  1060 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.082474  1000 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.100427  1000 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.082237   767 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.105302 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:30.106352   976 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:30.107343  1049 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:30.107960  1049 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.108444  1049 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.107302 31963 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:30.111236 32656 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.112638 31963 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:30.102173  1000 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.114387  1052 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.114840  1000 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.115159   984 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:30.115674   984 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:30.117030   984 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.090530 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:30.120132 32653 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:30.120373 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.122453  1095 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:30.050189 32113 rpcz_store.cc:276] 0709 12:17:28.922288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.938636 (+ 16348us) service_pool.cc:224] Handling call
0709 12:17:30.047170 (+1108534us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:30.047672 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.049212 (+  1540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.049574 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10178816}
I20260709 12:17:30.123088  1095 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.124269  1120 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:30.124779 32047 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:30.069643   910 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.125365  1095 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.062527  1126 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.130589   910 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.136813 32120 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:30.137331 32658 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:30.129630   964 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:30.105273   767 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.142585  1099 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.143240  1099 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.145831  1099 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.145951 32047 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:30.138418   984 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.155159 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:30.156941  1030 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.157810  1113 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.143653   767 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:30.158836 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.159334 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.158880  1060 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.163926 32660 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.167882 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1284) took 1220 ms. Trace:
I20260709 12:17:30.168146 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:30.151226   951 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.169641  1098 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.171725 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.172423 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1287) took 1213 ms. Trace:
I20260709 12:17:30.173342  1166 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:30.184725 31969 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:30.187862 32653 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:30.173975  1038 raft_consensus.cc:993] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:17:30.192795  1038 raft_consensus.cc:1081] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
W20260709 12:17:30.193553 32651 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.008s
I20260709 12:17:30.194172 32651 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:30.174391  1124 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.196712  1124 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:30.199257 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.199302  1082 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:30.200723 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.201463 32047 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:17:30.176014 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.202697 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:30.203022 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.176517 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1315) took 1031 ms. Trace:
I20260709 12:17:30.204095 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.204069 32046 rpcz_store.cc:276] 0709 12:17:29.145236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.204816 (+ 59580us) service_pool.cc:224] Handling call
0709 12:17:30.176086 (+971270us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54001937d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:30.176295 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473345792}
I20260709 12:17:30.206930  1166 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.202693  1124 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.207645  1166 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.177197 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.210930 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.211481  1110 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.212378 31972 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:30.212159  1110 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.171766 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:30.183996 32039 rpcz_store.cc:276] 0709 12:17:28.947102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.947562 (+   460us) service_pool.cc:224] Handling call
0709 12:17:30.166253 (+1218691us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:30.166733 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.167204 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.167244 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.167320 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:30.167374 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.167459 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.167608 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8581888}
W20260709 12:17:30.178013 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73095.1us
I20260709 12:17:30.189711  1030 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:30.189393  1113 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:30.190287 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.190364 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:30.190915  1098 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.203577 32654 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:30.216706  1030 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:30.214841 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.219833 32046 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:17:30.220273 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48805.5us
I20260709 12:17:30.220711 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.221202 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.221508 32116 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:30.221911 32121 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:30.223349 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:30.224453 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:30.224586 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:30.225049 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.226039 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1356) took 1006 ms. Trace:
W20260709 12:17:30.227072 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45424us
I20260709 12:17:30.224402  1102 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.229200 32044 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.229725 32044 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.229847  1098 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.230522 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.232867 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.233291 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:30.234508 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:30.234838 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.232976  1121 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:30.235976 32044 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.236734  1121 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:30.236841 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 86729us
I20260709 12:17:30.237265  1121 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.237702 32044 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.237982  1099 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.227391 32121 rpcz_store.cc:276] 0709 12:17:29.219614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.220793 (+1001179us) service_pool.cc:224] Handling call
0709 12:17:30.225989 (+  5196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:30.238711 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:30.240024  1083 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:30.240437  1083 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:30.240550 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.240867  1083 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.241580  1187 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.242202   933 raft_consensus.cc:2764] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:17:30.237731  1121 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.244513  1086 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.242249  1187 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.246554 32657 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47552
I20260709 12:17:30.247232 32657 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:17:30.248034  1140 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.130s	user 0.002s	sys 0.002s
I20260709 12:17:30.233393   910 raft_consensus.cc:1064] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:30.234704  1030 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.247700 32657 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.225680  1110 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.262512 32042 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:30.255195   910 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:30.263067 31965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 76) took 2178 ms (client timeout 1846 ms). Trace:
W20260709 12:17:30.263517 31965 rpcz_store.cc:269] 0709 12:17:28.084217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.091986 (+  7769us) service_pool.cc:224] Handling call
0709 12:17:30.262668 (+2170682us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54001ce654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:30.262850 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120725248}
I20260709 12:17:30.264060 32042 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.255774 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:30.217967  1166 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.272504 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.255697 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:30.279002 32046 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.279174 31966 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:30.280246  1187 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.278995 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1279) took 1823 ms. Trace:
I20260709 12:17:30.282670 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:30.281776 32045 rpcz_store.cc:276] 0709 12:17:28.451383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.452272 (+   889us) service_pool.cc:224] Handling call
0709 12:17:28.470786 (+ 18514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.917711 (+446925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.917757 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.918306 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.996846 (+ 78540us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:28.996883 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.996958 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.247192 (+1250234us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:30.251741 (+  4549us) raft_consensus.cc:1685] finished
0709 12:17:30.251769 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.274761 (+ 22992us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60698,"raft.queue_time_us":76027,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":125944,"spinlock_wait_cycles":26879232,"thread_start_us":487920,"threads_started":2,"wal-append.queue_time_us":145429}
I20260709 12:17:30.283511 32656 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:30.283603 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.284909 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.285192  1142 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.187566 32038 rpcz_store.cc:276] 0709 12:17:28.958868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.334167 (+375299us) service_pool.cc:224] Handling call
0709 12:17:29.334590 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:29.349673 (+ 15083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.353729 (+  4056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:29.355444 (+  1715us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:29.358599 (+  3155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.171726 (+813127us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b5400450954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:30.171823 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.172142 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6503808}
I20260709 12:17:30.285940  1142 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.291936 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.293388 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.265996  1113 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:30.284610 32045 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:30.303000 32044 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.306488 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1285) took 1349 ms. Trace:
I20260709 12:17:30.306859 32044 rpcz_store.cc:276] 0709 12:17:28.954093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.072433 (+118340us) service_pool.cc:224] Handling call
0709 12:17:30.228807 (+1156374us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b5400484f20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d2e18 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6
0709 12:17:30.303528 (+ 74721us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10102528}
I20260709 12:17:30.295814  1142 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:30.315055 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.316107 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 81724.5us
I20260709 12:17:30.310593 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:30.330633 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:30.345249 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.317363 31938 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:30.285192  1054 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:30.373378 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 76908.1us
W20260709 12:17:30.375751 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.316633  1101 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:30.376937 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 80507.1us
I20260709 12:17:30.329137   767 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.316452 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:30.379962 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.331350  1110 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:30.381264 32045 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:30.374111  1054 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.315632   976 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:17:30.377929 31963 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.387724 32041 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:30.406085 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.380728  1101 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.405119 32657 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.407738 32041 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:30.408190 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.428011 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.428447 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.429116 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:30.411953  1104 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.412467  1113 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:30.414063  1188 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:17:30.323788  1115 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:17:30.381271  1118 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:30.403828 32043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1310) took 1947 ms (client timeout 2549 ms). Trace:
I20260709 12:17:30.410404  1134 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:17:30.409578 32652 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.380517   767 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.315382   920 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 12:17:30.390367 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.395324  1054 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.446666 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1506) took 1001 ms. Trace:
I20260709 12:17:30.392329 31964 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:30.443869 32043 rpcz_store.cc:269] 0709 12:17:28.454685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.527925 (+ 73240us) service_pool.cc:224] Handling call
0709 12:17:30.380244 (+1852319us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b54001a6dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:30.401763 (+ 21519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130161920}
I20260709 12:17:30.414080  1034 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:17:30.405968  1142 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.463420  1166 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.464605 32657 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.402442   964 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:17:30.498983 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.499080   964 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.499715 32116 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.500113 32116 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:30.499876   964 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.500953 32116 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.502631   803 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.503090   803 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.503758 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 113) took 1123 ms. Trace:
I20260709 12:17:30.504796 32653 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:30.504293   803 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.506439 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.507658 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.508818 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.509430 32043 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.511099  1114 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.436367  1101 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.406361  1060 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:17:30.443830  1118 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.454744 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:30.464433 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 147206us
I20260709 12:17:30.463753  1095 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:30.492254 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.494565  1142 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.495723 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 312) took 1021 ms. Trace:
I20260709 12:17:30.479880  1099 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.504150 32040 rpcz_store.cc:276] 0709 12:17:29.380564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.759491 (+378927us) service_pool.cc:224] Handling call
0709 12:17:30.503525 (+744034us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54001a4fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:30.503703 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570682624}
I20260709 12:17:30.466737   767 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.541767 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:30.558773 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 114900us
I20260709 12:17:30.506135   803 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.459040  1030 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:17:30.514418 32116 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.488586  1166 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.586584  1166 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.587143 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.588368 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.589751 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:30.540330  1192 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:17:30.591001 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.591673 32040 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
W20260709 12:17:30.591768 32010 leader_election.cc:336] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.138s (SENT)
I20260709 12:17:30.592275 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:30.592536 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:30.593629 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.593811 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.594558 31965 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.595597 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.596068  1034 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.596805  1113 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.596822  1034 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.597478  1113 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.598901  1095 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.599673  1113 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.599787  1034 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.600010  1095 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.601980 32652 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.603899 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.506148 32651 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:30.566493 32660 rpcz_store.cc:276] 0709 12:17:29.474616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.474830 (+   214us) service_pool.cc:224] Handling call
0709 12:17:29.490792 (+ 15962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.184960 (+694168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.495449 (+310489us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54003a07d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:30.495663 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546603904}
I20260709 12:17:30.561059  1118 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.484843  1121 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.581502 32043 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:30.582602   964 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.583868  1049 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.585348 31963 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.594189 32040 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.615100 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.609786 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:30.617878 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.604712 31965 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.623696 32045 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:30.625257 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.479247 31969 rpcz_store.cc:276] 0709 12:17:29.444916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.590223 (+145307us) service_pool.cc:224] Handling call
0709 12:17:29.615860 (+ 25637us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.188083 (+572223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.446260 (+258177us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54001e82d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:30.446568 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286337792}
I20260709 12:17:30.544112  1038 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:17:30.614259 32116 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:30.621392  1115 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:30.635712 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.636785 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.619318 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.623332 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.634039 32117 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:30.642823 32120 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 12:17:30.644297 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.647228 32114 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.650501 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.653272 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:30.655874 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.661202 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:30.668634 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.660091  1190 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:17:30.672353 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.672938   910 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.673105 32121 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.674665 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.680822 32120 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:30.676671 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.677196 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 91811.2us
W20260709 12:17:30.682094 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 96524.8us
I20260709 12:17:30.683323 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:30.672968  1195 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:17:30.660465 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.660874 31963 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:30.666011  1038 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 1.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:30.714854   931 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:30.721036   931 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:30.721428 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.722147 31968 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:30.723315 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.724663  1142 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.715739   975 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:30.725239 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:30.725553   975 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.725970   975 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.660732  1113 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:17:30.682080  1030 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:30.675525   933 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.718938 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.727375 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:29.757441 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:30.735183 31867 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 2 to 3, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:30.736533  1121 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.738471 32115 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:30.740079  1038 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.741060 32121 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:30.754760 31935 leader_election.cc:400] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3.
I20260709 12:17:30.726238 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:30.743921 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.767944 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:30.720052  1086 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.728229 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:30.772874 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54772.7us
I20260709 12:17:30.728763 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.778035 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.779706 32623 leader_election.cc:400] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:30.782074 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.782827   933 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.785388   933 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.789786 31969 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:30.791110 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.792143  1121 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.792718  1121 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.793363  1086 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.794587  1118 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.795827  1086 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.795851 31968 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:30.796239 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:30.778801 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.796784  1118 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.798355 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.798542   975 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.799911 32010 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.800681   975 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.801875   975 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:30.802186 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:30.803126  1134 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:30.803305  1102 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:30.803695  1134 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.805022  1134 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.806734 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.803762  1102 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.807188   964 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:30.807766 31963 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:30.722162   931 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.809254 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.807634  1102 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:30.812280  1083 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
I20260709 12:17:30.812716  1083 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.767412 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.765128  1082 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.817785  1082 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.820616  1082 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.822996  1054 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.824150 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.828478 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.829835 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:30.830627  1036 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.831394  1036 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.831744  1036 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.832108  1036 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.760838  1030 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:30.834712  1187 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:30.835162  1030 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.754302  1038 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.837679  1038 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.838871  1099 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:30.825898 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.840539  1036 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.841735 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.843294 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:30.844079 31963 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.776769   910 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.849233   910 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.852246  1110 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.852887  1110 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:30.857669  1194 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.324s	user 0.001s	sys 0.000s
I20260709 12:17:30.858932 32119 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:30.861135  1115 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.861768  1115 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.864112  1115 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.864482 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:30.865628 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:30.871997  1111 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.872866   770 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.866180 32043 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.875031 32043 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.911810 32040 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:30.914690 31972 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.020s	sys 0.042s
I20260709 12:17:30.792580 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:30.915768 31972 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.021s	sys 0.042s
I20260709 12:17:30.807673   964 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.917487 32120 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:30.918491  1054 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:30.795778  1121 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.919449   933 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.924118 32043 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.913388 32114 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:30.939452 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:30.938453 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 212284us
I20260709 12:17:30.925637   964 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:30.930644 31969 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:30.921622  1166 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.937336 31964 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:30.941680  1121 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.952826 31963 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:30.956686  1121 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:30.958189 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:30.954571   767 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.947536  1083 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:30.959929 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.960561 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.955622  1101 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:30.961012  1104 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.520s	user 0.002s	sys 0.006s
I20260709 12:17:30.961378  1166 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:30.962407   869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.588s	user 0.038s	sys 0.087s
I20260709 12:17:30.962695  1101 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:30.962788   869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.646s	user 0.038s	sys 0.087s
I20260709 12:17:30.963260   869 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:30.963387  1101 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:30.963874 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.964459 31967 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:30.964335   869 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:30.965128 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.959118  1121 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:30.957284 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:30.953920   998 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.962702 32043 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.974522 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:30.986218 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.975553 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:31.003046 32121 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:30.986871 32118 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:30.982710 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:31.005064 32120 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.971951 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:30.984083 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:30.976393 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1485) took 2536 ms. Trace:
I20260709 12:17:31.010354 32657 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:17:30.967706 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1493) took 2522 ms (client timeout 2138 ms). Trace:
I20260709 12:17:31.004169 32118 pending_rounds.cc:85] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 1
I20260709 12:17:31.013528 32118 pending_rounds.cc:107] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 3.2
W20260709 12:17:31.013244 31967 rpcz_store.cc:269] 0709 12:17:28.445010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.467432 (+ 22422us) service_pool.cc:224] Handling call
0709 12:17:30.967491 (+2500059us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:30.967627 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5631108864}
I20260709 12:17:31.013931 32118 raft_consensus.cc:2889] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:31.008042 31971 rpcz_store.cc:276] 0709 12:17:28.440350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.440830 (+   480us) service_pool.cc:224] Handling call
0709 12:17:28.441145 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.044669 (+1603524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.976031 (+931362us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54004303d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:30.976231 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3451599488}
I20260709 12:17:31.021610 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:31.022462 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:30.984884 32116 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:30.982509 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.966296 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:30.969252  1107 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.978108  1034 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:30.975198 32652 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:30.969624 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.052331 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:31.052942 32114 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.991842   910 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:30.986950 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.961010   767 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:30.972280 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:30.967247 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.033565 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:30.981132   984 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:30.959146  1114 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:30.969858 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:30.970021 31963 raft_consensus.cc:2376] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2. Re-sending same reply.
I20260709 12:17:30.980551  1099 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.048465  1120 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:31.050716 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:31.054992 31967 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:31.054517  1034 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:31.059468 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.063407 32113 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.063789 32121 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.980883   998 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.098961 32115 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:31.100329   998 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.098112 32113 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:31.102739 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.103577 32118 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:31.104640 32118 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:31.053871  1038 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:31.075598 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53287.1us
I20260709 12:17:31.077399 32120 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:30.968585  1101 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.112421 32040 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:31.083204   984 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.083698  1121 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:31.085073 32041 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.076356   975 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:31.064107  1107 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.087597  1049 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.088263 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:31.091223  1193 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.628s	user 0.001s	sys 0.002s
I20260709 12:17:31.091867 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:31.149955 32045 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.150120 32114 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:31.153278 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:31.093163 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 83) took 2022 ms. Trace:
I20260709 12:17:31.154040 31966 rpcz_store.cc:276] 0709 12:17:29.070469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.072596 (+  2127us) service_pool.cc:224] Handling call
0709 12:17:30.278535 (+1205939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.279642 (+  1107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.092935 (+813293us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001ce650. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:31.093099 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705251072}
I20260709 12:17:31.155990 31870 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 227) took 3740 ms. Trace:
I20260709 12:17:31.076062 31964 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:31.067620  1114 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.157732 32046 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.158042 31964 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:31.158620  1003 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.159111  1003 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.159720 31964 raft_consensus.cc:1217] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.4]   Dedup: 2.3->[2.4-2.4]
I20260709 12:17:31.160431 31964 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: [2.3-2.4]
I20260709 12:17:31.161800 31964 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:31.162526   975 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:31.162890 31867 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.164839   975 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.164593 31867 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.166625 32041 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.167059  1118 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:31.168061  1107 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.168156 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.097468   910 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:31.169456   910 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.169855   910 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.096517  1110 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.153848 32047 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:31.112150 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:31.079555   767 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.075832  1120 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:31.127540 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:31.179347 32040 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.179934 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:31.128839 32119 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:31.129999 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.063776  1102 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.139142   964 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.146044  1101 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.146842 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:31.147845 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:31.149416 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 130643us
I20260709 12:17:31.097355  1046 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:31.166795 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.204532 32117 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:31.175274 31938 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.112963   803 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.182461 32120 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:31.177309 31867 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:17:31.178977 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66905.3us
I20260709 12:17:31.196934 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:31.211407 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:31.214568 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.197372 32121 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:31.198559 32119 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:31.208666  1101 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:31.218034   999 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.218777 32038 pending_rounds.cc:85] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 1
I20260709 12:17:31.219123 32038 pending_rounds.cc:107] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:31.219373 32038 raft_consensus.cc:2889] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:31.219626   803 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.220444 32120 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:31.220801 32120 pending_rounds.cc:85] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 1
I20260709 12:17:31.221035 32120 pending_rounds.cc:107] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:31.221248 32120 raft_consensus.cc:2889] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:31.199996 32045 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:31.222460  1115 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.222939  1115 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.081117  1099 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.115157   770 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:17:31.173758 32043 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:31.178730 32113 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:31.208078 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60299.1us
I20260709 12:17:31.217331  1000 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.232887  1000 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.233039  1120 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.234055  1120 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.196800  1142 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:31.235092  1142 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:31.218181   951 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:31.236162   951 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:31.236040  1142 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.237097  1003 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.237051   951 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.223781 32013 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:31.224045 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48853.3us
I20260709 12:17:31.223484  1102 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.226118 32656 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:31.184670  1049 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.228271 31970 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.199575 32040 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:31.246466 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:31.253963 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.224114   803 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:31.256717  1049 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.156342 31870 rpcz_store.cc:276] 0709 12:17:27.415282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.415446 (+   164us) service_pool.cc:224] Handling call
0709 12:17:27.682568 (+267122us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890ab0b2 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:17:27.702357 (+ 19789us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aae9c 00007f22890ab0f6 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:17:27.831161 (+128804us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1d7f3a 00007f2291f8de9c 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:17:31.155762 (+3324601us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:28.512495 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:28.673476 (+160981us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:28.673555 (+    79us) write_op.cc:435] Acquired schema lock
0709 12:17:28.673613 (+    58us) tablet.cc:662] Decoding operations
0709 12:17:28.737009 (+ 63396us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:28.742680 (+  5671us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:28.745100 (+  2420us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:17:28.817634 (+ 72534us) tablet.cc:701] Row locks acquired
0709 12:17:28.817666 (+    32us) write_op.cc:260] PREPARE: finished
0709 12:17:28.817842 (+   176us) write_op.cc:270] Start()
0709 12:17:28.818010 (+   168us) write_op.cc:276] Timestamp: P: 1783599448817789 usec, L: 0
0709 12:17:28.818037 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:17:28.866392 (+ 48355us) log.cc:844] Serialized 2054 byte log entry
0709 12:17:29.334684 (+468292us) op_driver.cc:464] REPLICATION: finished
0709 12:17:29.367650 (+ 32966us) write_op.cc:301] APPLY: starting
0709 12:17:29.367741 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:29.367916 (+   175us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:29.368070 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:29.368845 (+   775us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:29.369122 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:17:29.369194 (+    72us) write_op.cc:312] APPLY: finished
0709 12:17:29.369522 (+   328us) log.cc:844] Serialized 47 byte log entry
0709 12:17:29.380837 (+ 11315us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:29.392691 (+ 11854us) write_op.cc:454] Released schema lock
0709 12:17:29.396705 (+  4014us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:29.446876 (+ 50171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:29.450296 (+  3420us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":95324544,"child_traces":[["op",{"apply.queue_time_us":32707,"apply.run_cpu_time_us":2884,"apply.run_wall_time_us":67519,"num_ops":5,"prepare.queue_time_us":311467,"prepare.run_cpu_time_us":39010,"prepare.run_wall_time_us":579445,"raft.queue_time_us":216599,"raft.run_cpu_time_us":12309,"raft.run_wall_time_us":57198,"replication_time_us":516487,"thread_start_us":446483,"threads_started":4,"wal-append.queue_time_us":179493,"wal-append.run_cpu_time_us":52014,"wal-append.run_wall_time_us":1302483}]]}
I20260709 12:17:31.257721 32112 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:31.246614  1107 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:31.259225  1049 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:31.261752 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:31.262116  1038 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.262238   803 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:31.262778  1038 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:31.263010 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:31.263437  1038 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:31.264300  1111 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.225396   975 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:17:31.264550 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:31.273126  1101 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.274353 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:31.274506  1107 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.275537  1107 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.276839  1114 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.278468  1111 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.278966  1099 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.279951 31936 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:31.224148   767 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.273118 32041 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:17:31.281551  1082 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.281397   767 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.281047 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:31.281533 32041 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 3
I20260709 12:17:31.282644 32121 raft_consensus.cc:2410] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:31.281134  1099 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:17:31.283591   767 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.282588 32041 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 12:17:31.283808  1082 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:31.284229 32041 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:17:31.283334  1104 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.283752 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.312299 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:31.399212  1110 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:31.264653   984 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:31.345561 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:31.457479 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.285025  1003 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.457669 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1297) took 1233 ms. Trace:
I20260709 12:17:31.313249 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.458036 32046 rpcz_store.cc:276] 0709 12:17:30.223781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.279849 (+ 56068us) service_pool.cc:224] Handling call
0709 12:17:30.280189 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.158095 (+877906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.457452 (+299357us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b540037a354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:31.457618 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017411456}
I20260709 12:17:31.250641   998 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:17:31.318138   951 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.366925   910 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.322958  1101 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:31.322852 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48582.5us
I20260709 12:17:31.265725  1177 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.200s
I20260709 12:17:31.319087 32041 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:31.284613  1082 consensus_queue.cc:1243] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:31.319569  1046 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.231191   920 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 12:17:31.389689 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.449743 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:31.450380 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:31.451108 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:31.470755 32660 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.429840  1114 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.456139 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1303) took 1164 ms. Trace:
I20260709 12:17:31.383257  1083 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.279034  1111 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.463830 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:31.470882 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.471529 32044 rpcz_store.cc:276] 0709 12:17:30.291819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.309115 (+ 17296us) service_pool.cc:224] Handling call
0709 12:17:31.455903 (+1146788us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54000339d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:31.456067 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595240704}
W20260709 12:17:31.472579 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 187387us
I20260709 12:17:31.473384 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.463469   976 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.484160 32118 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:31.487294 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.465567  1107 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.494540 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.495692 32121 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:31.465137  1220 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:31.496894 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.488090 32118 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:17:31.493975  1119 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:17:31.536408 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1281) took 3026 ms. Trace:
I20260709 12:17:31.537120  1101 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:31.536638  1083 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.536937 32041 rpcz_store.cc:276] 0709 12:17:28.475846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.706138 (+230292us) service_pool.cc:224] Handling call
0709 12:17:28.706591 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.387487 (+1680896us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54003915a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d11c0 00007f228c1d2231 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c
0709 12:17:31.084620 (+697133us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b54003915a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14d06f 00007f228c1c5f60 00007f228c1c77de 00007f228c1d1b49 00007f228c1d2416 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62
0709 12:17:31.485625 (+401005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.501617 (+ 15992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629024384}
I20260709 12:17:31.538558 32118 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:31.538664 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.537604  1101 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.544914 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:17:31.536867  1230 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:17:31.481228  1046 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.556470 32121 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:31.560319   910 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.557765   920 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.493099   976 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.562564 32117 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:31.563956   976 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:31.564044  1226 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:17:31.547053  1101 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.572455 32119 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:31.578042 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:31.580137 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.575349   976 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.581300   976 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.581743  1210 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:31.582129  1210 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.582495  1210 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.583693   976 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.562224   920 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.587288  1104 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:31.589149  1212 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.505s	user 0.005s	sys 0.007s
I20260709 12:17:31.592275  1034 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:31.593422  1046 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.595568 32121 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:31.603523 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.604851 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.596989 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:31.599752 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.600364 32660 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:31.594182  1218 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:31.595798 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.602176  1142 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:31.629763 32013 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:31.636065 32121 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:31.737064 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.737994 32118 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:31.636891 32651 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:31.636601  1142 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.638459 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.631886  1218 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.646258  1034 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.599759   920 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:31.692793 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 114862us
I20260709 12:17:31.702438   964 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.634748   767 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:31.634459  1119 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:31.735412 31965 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.637696   888 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:31.738749 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54436.8us
W20260709 12:17:31.741586 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 102278us
I20260709 12:17:31.741647 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1329) took 1258 ms. Trace:
I20260709 12:17:31.742710 32013 leader_election.cc:403] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:31.740487  1142 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.743104 32112 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:31.742195 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.760408 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:31.743883 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:17:31.762940 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 133353us
I20260709 12:17:31.743561  1083 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:31.765533 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:31.756754 32047 rpcz_store.cc:276] 0709 12:17:30.483050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.212461 (+729411us) service_pool.cc:224] Handling call
0709 12:17:31.212846 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:31.217846 (+  5000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:31.217879 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:31.218355 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:17:31.428927 (+210572us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:31.428965 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.429039 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.741425 (+312386us) raft_consensus.cc:1685] finished
0709 12:17:31.741457 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.741589 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1110,"raft.queue_time_us":5607,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":210199,"threads_started":1,"wal-append.queue_time_us":214329}
I20260709 12:17:31.791916 32118 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:31.798554 32657 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:17:31.747859   998 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.748595  1177 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.758993  1036 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.810590  1177 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.811090 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:31.753891  1225 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 12:17:31.745012  1119 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.813036   888 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:31.813391  1119 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.752816   984 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:31.814652 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:31.814864   984 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.815266 32660 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.815748   984 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.817145   984 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.789350  1220 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:31.794391   869 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:31.818821  1220 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:31.819417  1220 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.819974 32659 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:31.821168 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 77387.5us
I20260709 12:17:31.759920  1210 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.824352  1210 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.824954  1000 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.795948   964 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:31.745860 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1215) took 1332 ms. Trace:
I20260709 12:17:31.808174 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:31.744573   767 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.794099   976 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3.
I20260709 12:17:31.827136 32655 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:31.834713 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:31.747952  1101 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.740763 32116 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:31.829888   869 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:31.830128 32115 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:31.797353  1120 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:31.832464 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:31.815284   951 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:31.833698 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.840337  1120 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.848202   910 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.850500 31964 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:31.851105 31971 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:31.848793   984 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:31.850584  1120 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.847011 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.833705 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:31.860996 32660 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:31.851786   984 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.838912  1142 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:31.831077  1003 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:17:31.829026 31965 rpcz_store.cc:276] 0709 12:17:30.413223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.607242 (+1194019us) service_pool.cc:224] Handling call
0709 12:17:31.734965 (+127723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.745331 (+ 10366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.745776 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:31.741431  1218 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.840073  1036 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:17:31.877009 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:31.873528 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:31.837618   998 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.867714   975 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:31.879473 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.868258  1142 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.877523 31972 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:31.883898 32652 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:31.884493 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:31.885290 31972 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.854060   869 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.865003  1083 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:17:31.890185 31967 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:31.877767 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:31.899852 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 105062us
I20260709 12:17:31.866163   999 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:17:31.875623  1210 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.905594 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.906507 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:31.910351   910 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.910776  1104 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.911118  1038 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.891301 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:31.912045 31966 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:31.912786 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.912670   910 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.914143   910 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:31.890594  1142 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.897678  1107 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:31.916762   920 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.903681  1238 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:17:31.918356 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:31.913712 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.925773 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.929311 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:31.929899 31967 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.914559   910 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.933255   910 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.935380   998 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.876513   767 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.936349  1083 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:31.937028  1083 consensus_queue.cc:1243] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:31.936707  1220 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.938608  1238 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:31.938994  1238 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:31.939867  1238 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.941016  1238 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.941418  1101 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.945122 31972 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:31.919601  1107 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.914790 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:31.949046 31967 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:31.924232  1177 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:31.942517  1218 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.928364  1113 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:31.914443  1210 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:31.962965 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71172.7us
I20260709 12:17:31.968756  1177 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.969522  1177 consensus_queue.cc:1243] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:31.970000  1177 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:31.970304  1177 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.971491 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:31.972244 32119 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:31.973466 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:31.958427  1107 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.977058 31967 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:31.927870   920 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:31.986526  1219 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:31.990638 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 63337.4us
I20260709 12:17:31.975996   888 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:31.974231   975 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:31.998205 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:31.986953  1219 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:32.035327 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.059702 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 68010.5us
I20260709 12:17:32.062031  1251 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:32.062708  1251 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:31.988631  1220 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:31.960894   984 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.002374   888 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.991427 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:32.060674  1219 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.989857  1177 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:32.090468 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 164747us
I20260709 12:17:32.092962 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:32.093993 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:32.085211   888 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.096031  1038 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:32.099895  1101 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:32.100069  1038 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:32.100620  1101 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:32.101194  1038 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.101890  1101 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:32.104903 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.108521  1038 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:32.122383  1247 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:17:32.122269   975 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.124230  1251 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.124723  1251 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.095278  1113 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:32.126746 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.127584 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:32.125140  1251 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.094967  1220 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:17:32.146921 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.114151  1256 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 12:17:32.148058 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.148541  1256 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:32.149355  1251 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.142479   964 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:32.150189  1111 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.150071   964 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.153503 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:32.153708  1119 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.154132 32119 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:32.155606 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:32.156890  1251 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.157280  1251 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.157788  1251 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.160171 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:32.169247 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.161410 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:32.157156 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.171638 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:32.135680  1245 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:32.142114  1038 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:32.180864 32045 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.186862 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.181311 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.180099 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:32.178646  1111 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.179279 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.189457 32040 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:32.189900 32046 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.184751 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.187958 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:32.136021   910 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.180817   999 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.137784  1120 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.199028 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:32.204963 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:32.209471 32119 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.183092  1046 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:32.205214 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.206743 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:32.205164  1217 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.978s	user 0.011s	sys 0.000s
W20260709 12:17:32.219245 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57927.9us
I20260709 12:17:32.218470 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:32.196507  1111 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.221781  1049 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:32.195551  1263 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:32.223397 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:32.205422  1210 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.224809 32013 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:32.225854 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.227634 31968 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.222247  1049 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.222733   920 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:32.221581  1111 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.238212 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:32.261351 32117 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:32.225660 32042 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:32.257812 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59554us
I20260709 12:17:32.213411   999 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.224305  1263 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:32.187040  1113 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:32.239857  1250 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:17:32.232416  1210 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.236651  1049 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.235800 32041 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:32.278990 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:32.299809  1259 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.196708  1245 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.334985 32119 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:32.391264 32117 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:32.336022 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70552.3us
W20260709 12:17:32.373459 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 237428us
I20260709 12:17:32.374239 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:32.374814 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 195639us
I20260709 12:17:32.397183 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.375506  1249 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.258225   931 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 12:17:32.405912 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:32.383219  1227 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.879s	user 0.004s	sys 0.006s
I20260709 12:17:32.400964 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.264060   803 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 12:17:32.413081 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1206) took 2017 ms. Trace:
I20260709 12:17:32.418354 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.420946 31970 rpcz_store.cc:276] 0709 12:17:30.395556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.034253 (+638697us) service_pool.cc:224] Handling call
0709 12:17:31.060480 (+ 26227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.248371 (+187891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.412790 (+1164419us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400313154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:32.413015 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384905600}
I20260709 12:17:32.436923 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:32.437187  1261 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.437587 32042 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.437879  1261 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.440680  1261 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:32.447054  1264 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 12:17:32.447536 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:32.448220 32655 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:32.383872  1275 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.406754   920 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:32.240070  1038 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.451545 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1331) took 1966 ms. Trace:
W20260709 12:17:32.451659  1267 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:17:32.456327 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.458717  1259 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:32.462615 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:32.465409  1266 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.158s	user 0.002s	sys 0.000s
I20260709 12:17:32.417565  1274 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:32.467212  1262 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:17:32.467454 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.469511 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:32.484017 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 246251us
I20260709 12:17:32.517336 32040 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:32.518162 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.518050 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.383677  1210 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.527885 32045 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:32.500656 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.466454 32652 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.512866   931 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:32.518445 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:32.513959 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 95713us
I20260709 12:17:32.475989  1275 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:32.486826 32112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.019s
I20260709 12:17:32.536309 32117 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:32.542212 32040 raft_consensus.cc:2368] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4.
I20260709 12:17:32.542160 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.543284 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.483248  1104 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.529538 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.526949 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.550187 31972 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:32.498024   933 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:32.490670  1114 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:32.498067  1259 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:32.503821  1083 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:32.507953 31966 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:32.523780  1218 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.545s	user 0.004s	sys 0.007s
W20260709 12:17:32.563971  1238 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.623s	user 0.013s	sys 0.000s
I20260709 12:17:32.508984  1110 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.546358 32112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.007s	sys 0.019s
I20260709 12:17:32.546187  1275 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.567540 32045 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:32.519490   910 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.574831 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.559871  1120 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:32.434768  1249 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:32.595008 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 196153us
I20260709 12:17:32.602505 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.565094 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.607203 31972 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:32.576534 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:32.580062 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.609474 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.636212 31970 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:32.640125 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 190334us
I20260709 12:17:32.648118 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.648461 32045 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:32.572930  1046 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:17:32.609347  1219 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.002s
I20260709 12:17:32.657912 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:32.640791   888 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.001s
I20260709 12:17:32.569059  1114 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.484098  1263 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.606953 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1372) took 1619 ms. Trace:
I20260709 12:17:32.640396   933 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.636480  1115 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:32.558197  1274 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.679052  1259 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.688318 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:32.728586 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 159571us
W20260709 12:17:32.726214 32046 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.000s
I20260709 12:17:32.600633 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 328) took 2123 ms. Trace:
I20260709 12:17:32.692340  1289 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.693055  1291 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.694422  1288 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.660874  1219 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.699911  1279 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:32.700999 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.670540 32113 rpcz_store.cc:276] 0709 12:17:30.987974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.103203 (+115229us) service_pool.cc:224] Handling call
0709 12:17:31.103524 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.236201 (+132677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.570672 (+1334471us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b540012c0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:32.602663 (+ 31991us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77414272}
I20260709 12:17:32.704291  1287 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:32.709475  1113 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.711586 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.763891 31967 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.717190  1126 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.668624 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.719082 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:32.779738 32044 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.720881  1249 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.721068   920 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.721341  1263 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.634855  1251 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.612602  1110 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:32.482285 32038 rpcz_store.cc:276] 0709 12:17:30.484865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.218200 (+733335us) service_pool.cc:224] Handling call
0709 12:17:31.218586 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:31.220228 (+  1642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:31.220263 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:31.220738 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:17:31.338766 (+118028us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:31.350822 (+ 12056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.364043 (+ 13221us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.170025 (+805982us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:32.448467 (+278442us) raft_consensus.cc:1685] finished
0709 12:17:32.448495 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.451473 (+  2978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5541632,"thread_start_us":89682,"threads_started":1,"wal-append.queue_time_us":192893}
I20260709 12:17:32.723138   931 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.726083 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.725986 32045 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:32.690809  1046 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.646839  1218 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:17:32.729386 32046 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:32.802361 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.803017 32046 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:32.804153 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.805370 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.729709 31970 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:32.807117 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.807775 31970 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.809720 31966 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:32.739069 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:32.811045 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72053.1us
I20260709 12:17:32.811100 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:32.812304   976 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.812541 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.813082   976 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.813655   976 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.814527   803 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:32.597973 31964 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.021s	sys 0.035s
W20260709 12:17:32.762928 31968 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 12:17:32.725595   999 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.776722 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 146) took 8385 ms. Trace:
I20260709 12:17:32.815662 31968 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:32.671061   888 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.816196   976 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.816520 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.817688 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.818060   888 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.748100 32041 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.820364 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.815886 31867 rpcz_store.cc:276] 0709 12:17:24.377227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.377389 (+   162us) service_pool.cc:224] Handling call
0709 12:17:32.762635 (+8385246us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:31.552321 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:31.569150 (+ 16829us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:31.569206 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:17:31.569227 (+    21us) tablet.cc:662] Decoding operations
0709 12:17:31.736773 (+167546us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:31.736951 (+   178us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:31.737017 (+    66us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:17:31.738079 (+  1062us) tablet.cc:701] Row locks acquired
0709 12:17:31.738196 (+   117us) write_op.cc:260] PREPARE: finished
0709 12:17:31.738540 (+   344us) write_op.cc:270] Start()
0709 12:17:31.738764 (+   224us) write_op.cc:276] Timestamp: P: 1783599451738419 usec, L: 0
0709 12:17:31.738874 (+   110us) op_driver.cc:348] REPLICATION: starting
0709 12:17:31.740209 (+  1335us) log.cc:844] Serialized 1723 byte log entry
0709 12:17:32.114274 (+374065us) op_driver.cc:464] REPLICATION: finished
0709 12:17:32.705146 (+590872us) write_op.cc:301] APPLY: starting
0709 12:17:32.705270 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:32.705496 (+   226us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:32.705665 (+   169us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:32.706520 (+   855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:32.706854 (+   334us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:17:32.706914 (+    60us) write_op.cc:312] APPLY: finished
0709 12:17:32.707296 (+   382us) log.cc:844] Serialized 41 byte log entry
0709 12:17:32.707552 (+   256us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:32.707677 (+   125us) write_op.cc:454] Released schema lock
0709 12:17:32.707747 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:32.708168 (+   421us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:32.708198 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1104955,"reported_tablets":8,"spinlock_wait_cycles":128384,"child_traces":[["op",{"apply.queue_time_us":578572,"apply.run_cpu_time_us":3019,"apply.run_wall_time_us":3020,"num_ops":4,"prepare.queue_time_us":258586,"prepare.run_cpu_time_us":71808,"prepare.run_wall_time_us":349967,"raft.queue_time_us":151816,"raft.run_cpu_time_us":41906,"raft.run_wall_time_us":485628,"replication_time_us":353257,"thread_start_us":732251,"threads_started":4,"wal-append.queue_time_us":149759}]]}
I20260709 12:17:32.822556 31968 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:32.823527 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.824115 31968 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:32.824851 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.793865 31967 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:32.790961   920 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:32.825634 31964 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.021s	sys 0.036s
I20260709 12:17:32.825685 32046 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:32.796921 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.826773 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.827075 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.827688 31971 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:32.828318 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.828449 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.828565   976 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:32.829027   976 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:32.829398   976 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.830004 31964 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:32.735522 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 338409us
I20260709 12:17:32.835798 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:32.836416 31964 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.836850   999 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:32.837249   999 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:32.837208 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.837882   999 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.838263  1288 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.838374 31964 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:32.839730 32041 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:32.840147 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.788874 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.824952 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:32.797322 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 252534us
I20260709 12:17:32.842521 31968 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:32.826097 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.832758   920 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:32.844092 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55299.9us
I20260709 12:17:32.844251 31967 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:32.845119 31970 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:32.850574 31935 leader_election.cc:400] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4.
I20260709 12:17:32.851553 31935 leader_election.cc:403] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:32.853189 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:32.861419 31967 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:32.861366 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.863260 31967 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:32.787060 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:32.854664 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43681.9us
I20260709 12:17:32.826748 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.867267 32046 raft_consensus.cc:2410] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:32.870082 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.874191 31968 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.875958 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:32.876650 31967 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:32.878477 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.879719 32044 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:32.807396 32042 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:32.741341 32656 rpcz_store.cc:276] 0709 12:17:30.451594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.453611 (+  2017us) service_pool.cc:224] Handling call
0709 12:17:30.754134 (+300523us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aaf8c 00007f22890aaef6 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:30.767372 (+ 13238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.255311 (+487939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.559117 (+1303806us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:32.573192 (+ 14075us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073286784}
I20260709 12:17:32.871532   933 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4.
I20260709 12:17:32.794494 31963 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:32.886226 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:32.887823 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47759.8us
I20260709 12:17:32.802848  1259 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:32.879848 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.624s (SENT) [suppressed 7 similar messages]
W20260709 12:17:32.894665 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 107672us
I20260709 12:17:32.850839  1104 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.909782 32046 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.783761  1113 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.869257  1210 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.869511  1288 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.920976 32046 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:32.905149  1095 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.903100  1083 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.870038  1249 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.887410   888 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:32.785735  1126 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.924293 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.847704   931 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:32.779774  1287 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.944056 32046 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:32.944459 32046 pending_rounds.cc:85] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 2
I20260709 12:17:32.944684 32046 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:32.944556 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.945115 32046 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:32.785997  1279 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.951668 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.919376  1285 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.948724   920 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:32.949730   931 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.922766  1104 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.950140  1248 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:32.897200   910 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:17:32.948464   888 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:32.763888  1289 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.971753  1114 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.973690  1126 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.973845 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.976356   976 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.952724 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:32.953449 31967 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:32.926003  1095 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.977471  1095 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.977638 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.977914 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:32.978631 31967 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:32.980394 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:32.982558  1298 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:32.983080  1298 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.983534  1298 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.984586  1118 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:32.953154  1049 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.000s
I20260709 12:17:32.985947  1118 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:32.986615  1049 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:32.987989  1036 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.988408  1036 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.988742  1036 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.989289  1049 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.990329 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:32.991511 32041 raft_consensus.cc:2410] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:32.991555  1287 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.992926  1287 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:32.993331  1287 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:32.993690  1287 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.993963 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:32.994794 32652 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:17:32.995033  1301 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:17:32.996935 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:32.997277  1279 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.998600 32042 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:32.998961  1110 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:32.999629  1036 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.970824   888 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:33.000380  1142 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.072s	user 0.008s	sys 0.004s
I20260709 12:17:33.000749  1142 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:32.983331  1300 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:32.992750 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:33.001948  1107 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.999s	user 0.008s	sys 0.005s
I20260709 12:17:33.002414 32041 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:32.999975  1251 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.002787  1142 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.003965  1291 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.006678 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1235) took 1081 ms. Trace:
I20260709 12:17:32.967927  1218 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:33.011435  1111 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.002s
I20260709 12:17:32.964598 32046 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.937058  1283 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.015673 31968 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:32.954348  1288 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:33.029601  1274 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:32.985797  1275 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.009444   976 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.032635 31965 rpcz_store.cc:276] 0709 12:17:31.925597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.058848 (+133251us) service_pool.cc:224] Handling call
0709 12:17:32.089505 (+ 30657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.373238 (+283733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.373288 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.484036 (+110748us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.791957 (+307921us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:32.798920 (+  6963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.842623 (+ 43703us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.006388 (+163765us) raft_consensus.cc:1685] finished
0709 12:17:33.006428 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.006606 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51835,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"thread_start_us":469668,"threads_started":2,"wal-append.queue_time_us":375534}
I20260709 12:17:33.069371 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.070448  1288 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:33.059733  1111 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.072315  1275 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.073824  1210 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.074618 32040 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:33.074779  1210 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.051654  1218 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.069402 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:33.082110 31968 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.022825  1219 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.032390  1104 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.067898   999 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.093905 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:33.033542  1220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.022s	sys 0.014s
I20260709 12:17:33.053306  1285 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.023017  1107 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.086515  1279 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:33.118721 31968 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
W20260709 12:17:33.119915 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:33.145546 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 265715us
I20260709 12:17:33.146603 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.147346 31968 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:33.091866  1280 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:33.101807  1104 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:33.120091  1220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.023s	sys 0.015s
I20260709 12:17:33.176294 32046 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:33.090704 31965 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
W20260709 12:17:33.163980 32120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.009s	sys 0.027s
I20260709 12:17:33.092368  1218 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.144711  1115 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.184302 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:33.106519  1111 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.148654   964 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.185710 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.183950   770 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.072335  1288 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.190052  1295 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.201800 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:33.183102  1104 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.176374   976 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.198489 31968 raft_consensus.cc:2368] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4.
I20260709 12:17:33.170550  1294 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.170455  1279 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.183343  1210 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.188247 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:33.368698 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1329) took 1191 ms. Trace:
I20260709 12:17:33.574167 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:33.214421 32046 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:33.433076 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:33.652782 32044 rpcz_store.cc:276] 0709 12:17:32.177029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.881878 (+704849us) service_pool.cc:224] Handling call
0709 12:17:32.907167 (+ 25289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.941633 (+ 34466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.941667 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.942608 (+   941us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.182703 (+240095us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:33.199835 (+ 17132us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.368134 (+168299us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b18002be040. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:17:33.368318 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.368388 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.368436 (+    48us) raft_consensus.cc:1685] finished
0709 12:17:33.368536 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.368646 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26926,"prepare.queue_time_us":16197,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":207048,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":13706,"spinlock_wait_cycles":45367040,"thread_start_us":143448,"threads_started":1,"wal-append.queue_time_us":234637}
W20260709 12:17:33.656864 31972 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1536) took 1547 ms (client timeout 1939 ms). Trace:
I20260709 12:17:33.435503 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:33.659077 32120 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:33.321097 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:33.498759 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:33.471625 32658 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:33.259148 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:33.317934 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:33.686952 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 365947us
W20260709 12:17:33.356509 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:33.575706 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57108.7us
I20260709 12:17:33.460704 32659 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:33.687569 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:33.687119 32046 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.296512 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:33.373764  1299 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.424s	user 0.001s	sys 0.001s
I20260709 12:17:33.690069 32039 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:33.367348 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1559) took 1134 ms. Trace:
I20260709 12:17:33.744650 32044 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:33.743834 32118 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.458364 32117 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:33.470633 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:33.754384 32045 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.754431 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 128) took 1095 ms. Trace:
I20260709 12:17:33.322891 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:33.657418 31972 rpcz_store.cc:269] 0709 12:17:32.109190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.645660 (+536470us) service_pool.cc:224] Handling call
0709 12:17:33.432898 (+787238us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:33.656805 (+223907us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11163008}
I20260709 12:17:33.363648 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.415293 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.626397  1206 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.541909  1082 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:33.814481  1325 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:17:33.814680  1206 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.815008  1082 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.817135  1206 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.626819  1275 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:33.368197  1316 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:17:33.817867  1275 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.818933 31963 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:17:33.820030  1275 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.444310  1220 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:33.226373   976 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.820816  1220 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.814304   770 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:33.822635   770 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:33.356920  1312 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:17:33.835449 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.822947  1220 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.825685 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 333) took 1927 ms. Trace:
I20260709 12:17:33.207470  1104 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.306638  1177 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.854595  1239 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:33.823352  1261 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.855365  1239 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.855831  1239 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.856700 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1548) took 1114 ms. Trace:
I20260709 12:17:33.857849  1239 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.408627  1208 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.790601 32040 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:17:33.341531  1317 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:33.652333  1274 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:33.503203  1303 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.302s	user 0.002s	sys 0.000s
I20260709 12:17:33.824317 32046 rpcz_store.cc:276] 0709 12:17:32.659247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.686582 (+1027335us) service_pool.cc:224] Handling call
0709 12:17:33.686939 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.754147 (+ 67208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.754373 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:33.832824 32039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1313) took 1661 ms (client timeout 1627 ms). Trace:
W20260709 12:17:33.256968  1306 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:17:33.366451  1310 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:17:33.467686  1142 raft_consensus.cc:476] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Not starting pre-election -- already a leader
I20260709 12:17:33.457772 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.457329 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:33.362146  1304 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:17:33.834517 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:33.858870  1238 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.859043   998 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:33.904482 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70074.1us
I20260709 12:17:33.905376 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:33.292521 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:33.215552  1113 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.840224 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:33.928608 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 88462.5us
I20260709 12:17:33.830060   869 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.840890   770 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:33.841300  1095 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.863s	user 0.010s	sys 0.010s
I20260709 12:17:33.932353 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1312) took 1761 ms. Trace:
I20260709 12:17:33.911718   998 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.841415   976 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.933398   869 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:33.361385  1119 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.225780  1272 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.841854  1107 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.944100   869 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:33.842080  1259 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.934s	user 0.014s	sys 0.001s
I20260709 12:17:33.192279  1115 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.813825 32651 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:33.842245 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.408739  1276 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:33.573057   767 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.369086  1102 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.887897  1208 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.843104 31869 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 2, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:33.844224   964 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.453864  1245 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:33.845036  1298 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.861s	user 0.010s	sys 0.009s
I20260709 12:17:33.188527  1280 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.836669   920 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:33.966442 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 111) took 2006 ms. Trace:
I20260709 12:17:33.959355  1286 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.966580   920 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.967267  1286 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.846406 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:33.847088 32652 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.852s	user 0.014s	sys 0.004s
I20260709 12:17:33.968968 32652 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:33.969259 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:33.969347 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:33.425431   975 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.422s
I20260709 12:17:33.969771 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:33.849689 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 275589us
I20260709 12:17:33.970943 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:33.971287 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:33.971861 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:33.847759 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.972795 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:33.973140 31968 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:33.456760 32113 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.959482   964 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.974130 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:33.974938 31968 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:33.346820  1302 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:33.975414  1302 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.975775 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:33.956584  1276 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.479364 32115 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:33.441766 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:33.854713  1249 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.918s	user 0.009s	sys 0.004s
I20260709 12:17:33.826481 32119 rpcz_store.cc:276] 0709 12:17:32.233250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.405668 (+172418us) service_pool.cc:224] Handling call
0709 12:17:32.405980 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.424373 (+ 18393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.424418 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.477227 (+ 52809us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.912075 (+434848us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:32.912123 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.912211 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.367135 (+454924us) raft_consensus.cc:1685] finished
0709 12:17:33.367155 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.367287 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6375,"raft.queue_time_us":43000,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":16256,"thread_start_us":383068,"threads_started":1,"wal-append.queue_time_us":587306}
I20260709 12:17:33.858768 32660 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.977540  1249 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.826141 31970 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:33.978024 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:33.857124 31967 rpcz_store.cc:276] 0709 12:17:32.742076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.979189 (+237113us) service_pool.cc:224] Handling call
0709 12:17:32.979506 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:33.035447 (+ 55941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.035488 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.060594 (+ 25106us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.378986 (+318392us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b1800435a00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:17:33.574800 (+195814us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:33.739618 (+164818us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.847002 (+107384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.856374 (+  9372us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b5400312c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:33.856431 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.856489 (+    58us) raft_consensus.cc:1685] finished
0709 12:17:33.856505 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.856642 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96260,"prepare.queue_time_us":165749,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":154632,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":9534,"spinlock_wait_cycles":107892864,"thread_start_us":195598,"threads_started":1,"wal-append.queue_time_us":491472}
I20260709 12:17:33.979097 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:33.979249  1249 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:33.859370 32045 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:33.507769  1256 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:33.980063 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1550) took 1237 ms. Trace:
I20260709 12:17:33.980342  1256 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:33.686628   999 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:33.491155 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:33.981209  1275 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:33.957015  1102 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.981786 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 338) took 1201 ms. Trace:
I20260709 12:17:33.982213  1275 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:33.979885 32119 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:33.983642  1275 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:33.984054 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:33.984220  1275 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:17:33.873064 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 297172us
I20260709 12:17:33.984699  1275 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.955505  1259 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.985263  1259 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.979142 31970 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:33.985899  1120 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:33.986539  1120 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:33.986768 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:33.969985 32112 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.987435 32045 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.987488  1120 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.988765   999 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:33.988951 32658 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:33.989367   999 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.991855   976 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:33.992244   976 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:33.993192   976 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.993961 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1556) took 1791 ms. Trace:
I20260709 12:17:33.994413 32121 rpcz_store.cc:276] 0709 12:17:32.203052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.203218 (+   166us) service_pool.cc:224] Handling call
0709 12:17:33.321637 (+1118419us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.321854 (+   217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:33.322260 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.322285 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.325003 (+  2718us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.833971 (+508968us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:33.834009 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.834069 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.993594 (+159525us) raft_consensus.cc:1685] finished
0709 12:17:33.993627 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.993879 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10392,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":6126,"spinlock_wait_cycles":7059840,"thread_start_us":457395,"threads_started":1,"wal-append.queue_time_us":600473}
I20260709 12:17:33.995524 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:33.996363 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.989732 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1549) took 1247 ms. Trace:
I20260709 12:17:33.347184  1036 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:33.889048 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:33.889070 32039 rpcz_store.cc:269] 0709 12:17:32.170899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.180933 (+ 10034us) service_pool.cc:224] Handling call
0709 12:17:33.259001 (+1078068us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:33.832720 (+573719us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17316096}
W20260709 12:17:33.859279 32038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.002s	sys 0.010s
I20260709 12:17:34.008672  1082 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.501708  1295 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.365s
I20260709 12:17:33.897579 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:33.933199 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:33.845747  1313 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.744537  1291 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.015239  1313 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.015476  1291 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:33.825155 32043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1326) took 1651 ms (client timeout 1905 ms). Trace:
I20260709 12:17:34.017604  1313 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.008914  1245 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 12:17:34.017740 32043 rpcz_store.cc:269] 0709 12:17:32.174131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.212023 (+ 37892us) service_pool.cc:224] Handling call
0709 12:17:33.322716 (+1110693us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:33.825052 (+502336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8713600}
I20260709 12:17:33.744251  1294 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.892076  1271 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.921058  1034 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:33.921743 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.923744  1142 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:33.924890  1287 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.931s	user 0.004s	sys 0.012s
I20260709 12:17:33.897970  1038 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:33.929976 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:33.911311  1270 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:33.925432  1288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.022s	sys 0.034s
I20260709 12:17:33.933632 32047 rpcz_store.cc:276] 0709 12:17:32.170621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.181914 (+ 11293us) service_pool.cc:224] Handling call
0709 12:17:32.188486 (+  6572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.376237 (+187751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.376287 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.418265 (+ 41978us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.725057 (+306792us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:32.725095 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.725163 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.372986 (+647823us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:33.929369 (+556383us) raft_consensus.cc:1685] finished
0709 12:17:33.929389 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.932295 (+  2906us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30255,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":9092480,"thread_start_us":395882,"threads_started":2,"wal-append.queue_time_us":380497}
I20260709 12:17:33.249493   951 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.925504  1268 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.935307  1000 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.936757 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1557) took 1722 ms. Trace:
I20260709 12:17:33.896883  1274 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.945637 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.033983  1268 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.034991 32047 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.035727 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1366) took 1021 ms. Trace:
I20260709 12:17:33.948187 32117 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:34.036015 32047 rpcz_store.cc:276] 0709 12:17:33.014497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.034506 (+1020009us) service_pool.cc:224] Handling call
0709 12:17:34.034825 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.035357 (+   532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.035672 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.028503  1034 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:34.036816 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.037262  1279 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.024793  1271 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.037667  1034 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.038127 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.039291 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.039750  1271 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.040318 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:34.040844 32117 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.843379  1289 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:33.913434  1238 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.913823  1210 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:17:33.953769 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 455070us
W20260709 12:17:34.030457  1288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.022s	sys 0.034s
I20260709 12:17:33.950919  1272 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.049186 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.036588  1268 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.055882  1286 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.056249  1327 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:34.056674  1327 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.057040  1327 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.057507  1272 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.028280 32114 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:33.933583  1095 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.960583  1298 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.032351  1038 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.060937  1095 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.061146  1298 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:34.061316  1038 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:33.962734 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.030870  1287 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:33.971735   869 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:33.972376 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:33.974735 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.976351 31968 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.040946  1261 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.977394 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1554) took 1779 ms. Trace:
I20260709 12:17:34.069308  1274 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.070746   999 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.071050  1302 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.071668   964 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.071868  1280 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.068858 32115 rpcz_store.cc:276] 0709 12:17:32.198118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.208042 (+  9924us) service_pool.cc:224] Handling call
0709 12:17:33.478634 (+1270592us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.478865 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.977117 (+498252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.977336 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4121472}
I20260709 12:17:34.073743  1102 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.074331 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.075986 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.076716 32115 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:34.077760 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.470110 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.079617 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.079895 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.080804 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.081421 32655 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:33.982457 31969 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:33.982268 32653 rpcz_store.cc:276] 0709 12:17:32.780569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.782806 (+  2237us) service_pool.cc:224] Handling call
0709 12:17:33.490996 (+708190us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:33.981694 (+490698us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27161984}
I20260709 12:17:33.849002 32659 rpcz_store.cc:276] 0709 12:17:31.828132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.828299 (+   167us) service_pool.cc:224] Handling call
0709 12:17:32.448898 (+620599us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.460193 (+1011295us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aae9c 00007f22890aaa5b 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.460348 (+   155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.742765 (+282417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.755200 (+ 12435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85551360}
I20260709 12:17:33.989710 31869 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 2, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 2 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:33.990491  1206 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:33.991318  1239 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:34.051056  1289 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:34.098748  1289 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:34.099742  1289 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:33.429740  1305 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:33.997751 31970 rpcz_store.cc:276] 0709 12:17:32.742225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.978699 (+1236474us) service_pool.cc:224] Handling call
0709 12:17:33.978989 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.986281 (+  7292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.989680 (+  3399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.886950 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:34.101429  1305 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:34.101997 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:34.102284 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.102587 31970 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.102398  1305 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.103878  1305 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.291688 32656 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:33.432101 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:33.252513 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:33.980607 31967 rpcz_store.cc:276] 0709 12:17:32.742450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.979254 (+1236804us) service_pool.cc:224] Handling call
0709 12:17:33.979530 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.979698 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.979723 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:33.979766 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:33.979794 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.979855 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.979986 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.002513 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:34.108804 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 106421us
W20260709 12:17:34.007452 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.008002 32010 leader_election.cc:400] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4.
I20260709 12:17:34.009016 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:34.112706 32010 leader_election.cc:403] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:33.293395  1251 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.587s
I20260709 12:17:34.009922 32046 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:34.116837 32046 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:34.117235 32046 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:34.008769 32038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.003s	sys 0.010s
I20260709 12:17:33.980959 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.013847   998 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.014578  1276 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.881119  1220 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.996132  1119 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:17:33.987390  1259 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.022035 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:34.019639  1115 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.009912   976 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.022305 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.028466 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.029031 32660 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:34.193128 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 263236us
I20260709 12:17:34.193562 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.194756 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.195596 32660 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:33.979933  1249 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:34.033027 32118 rpcz_store.cc:276] 0709 12:17:32.214862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.215219 (+   357us) service_pool.cc:224] Handling call
0709 12:17:33.439734 (+1224515us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.439969 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.825466 (+385497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.936469 (+111003us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400139054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:33.936687 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689127552}
I20260709 12:17:33.937318 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.196870 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.197436 32118 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:34.197959  1249 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.055015 32045 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:34.198258 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:34.198936 32118 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.032140  1142 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:34.200135  1142 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.030233  1270 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.059132 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:33.969578 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 339) took 1004 ms. Trace:
I20260709 12:17:34.036677  1294 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:33.761581 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:33.966920 32116 rpcz_store.cc:276] 0709 12:17:31.959665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.961140 (+  1475us) service_pool.cc:224] Handling call
0709 12:17:33.304432 (+1343292us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.304686 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:33.305138 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.305172 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.305681 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.753766 (+448085us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:33.828143 (+ 74377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.837737 (+  9594us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.963716 (+125979us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b18003474a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:17:33.966178 (+  2462us) raft_consensus.cc:1685] finished
0709 12:17:33.966199 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.966381 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2572,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":128196864,"thread_start_us":360522,"threads_started":1,"wal-append.queue_time_us":381921}
I20260709 12:17:33.981359  1256 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.082798 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:34.088639 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:34.212390 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 129654us
I20260709 12:17:34.209483 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.214102 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:34.091007 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.091171 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.091543 31869 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.089713  1239 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:34.094740 32115 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:34.134523  1280 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:34.096410 31968 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:34.096611 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.066277   869 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:34.097368  1327 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.103956 31972 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:34.207324 32114 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.111442 31970 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:34.112984 32039 raft_consensus.cc:606] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:34.119326 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 111523us
I20260709 12:17:34.241627 32039 raft_consensus.cc:614] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:34.084045  1261 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:34.088399  1208 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.121173   964 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.107748 32656 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:34.118140 32046 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:34.131273 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 225990us
I20260709 12:17:34.134979 32121 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.253077 32046 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.254427 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.254959   999 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:34.255592  1343 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:17:34.255968  1274 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:34.138720 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1530) took 2031 ms. Trace:
I20260709 12:17:34.123544  1210 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.130717  1313 raft_consensus.cc:1027] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: attempt to promote peer ec7f26ea46944ea498ad253b1b1819b0: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:17:34.207773 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.257180  1210 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.257063 31869 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.258169 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.258967 32660 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:34.259827  1210 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.258914 31869 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 2 to 3, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.207832 32120 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:34.218448 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1325) took 2044 ms. Trace:
I20260709 12:17:34.190861 31965 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:34.126202 32654 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:34.095386  1287 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.198352 32041 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:34.137724  1083 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.121851  1206 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:17:34.221642 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1248) took 1259 ms. Trace:
I20260709 12:17:34.217585 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:34.202421  1270 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.206001   920 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.203831  1245 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4.
I20260709 12:17:34.280107 32118 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:34.237401 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 130) took 1063 ms. Trace:
I20260709 12:17:34.263334 31966 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:34.240923 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1246) took 1317 ms. Trace:
I20260709 12:17:33.984712 32119 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.130625  1220 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:34.243773 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 221538us
I20260709 12:17:34.217476  1102 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.247421 32045 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:34.219353  1276 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.215330  1034 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:17:34.255395 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.130656  1251 consensus_queue.cc:1243] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:34.261126 31869 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:33.465626  1248 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:34.228003   869 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:34.252882   964 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.204484 32652 rpcz_store.cc:276] 0709 12:17:32.965422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.968187 (+  2765us) service_pool.cc:224] Handling call
0709 12:17:33.969515 (+1001328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.264207  1334 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.263368 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.224207  1272 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:34.264430 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 175886us
I20260709 12:17:34.266767 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:34.229110  1238 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.256484  1274 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:34.269709 32658 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:34.270255 31967 rpcz_store.cc:276] 0709 12:17:32.962072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.118706 (+1156634us) service_pool.cc:224] Handling call
0709 12:17:34.119040 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.201064 (+ 82024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.201127 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.201182 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:34.201224 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.221327 (+ 20103us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b540034dd54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:34.221404 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.221570 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188144000}
I20260709 12:17:34.236991  1259 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.256474 31971 rpcz_store.cc:276] 0709 12:17:32.107252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.829218 (+721966us) service_pool.cc:224] Handling call
0709 12:17:32.829500 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.829676 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:32.889017 (+ 59341us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.500315 (+611298us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b1800244f00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:17:33.936319 (+436004us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.128243 (+191924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.128316 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:34.128373 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.137321 (+  8948us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.138640 (+  1319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16202,"prepare.queue_time_us":58339,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":85,"raft.queue_time_us":85456,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":15856896,"thread_start_us":627244,"threads_started":2,"wal-append.queue_time_us":1125183}
I20260709 12:17:34.224691  1329 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.237152 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.237332 31968 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:34.270671  1333 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.349689  1329 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:34.349934 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1250) took 1034 ms. Trace:
I20260709 12:17:34.350286  1329 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.350499 31968 rpcz_store.cc:276] 0709 12:17:33.315431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.230646 (+915215us) service_pool.cc:224] Handling call
0709 12:17:34.231015 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.349657 (+118642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.349882 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.351330 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.351562  1220 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:34.352649 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 115584us
I20260709 12:17:34.281116 32047 rpcz_store.cc:276] 0709 12:17:33.173987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.036769 (+862782us) service_pool.cc:224] Handling call
0709 12:17:34.037028 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.037216 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:34.128954 (+ 91738us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.237065 (+108111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.237115 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:34.237158 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.237224 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.237345 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26463,"prepare.queue_time_us":91912,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":29,"raft.queue_time_us":30273,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":9855,"thread_start_us":172638,"threads_started":2,"wal-append.queue_time_us":93765}
I20260709 12:17:34.353256 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.354216 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.354692 32660 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:34.354435 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.349860 32659 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:34.355214 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:34.357108 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 101780us
I20260709 12:17:34.357625 32660 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:34.358649 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.298528 32039 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:34.363107 32121 raft_consensus.cc:2410] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:34.327824 32117 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:34.364248 31972 rpcz_store.cc:276] 0709 12:17:32.923407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.103367 (+1179960us) service_pool.cc:224] Handling call
0709 12:17:34.103674 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.240693 (+137019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.240873 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.284261 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:34.366129 32120 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.356591 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:34.266925  1256 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.370844 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.267127   975 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.230083  1280 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:34.425417 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55959.4us
I20260709 12:17:34.381505 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.428288 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.428732 31967 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:34.380690 31869 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:34.433506 32119 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
W20260709 12:17:34.433912 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 79768.2us
I20260709 12:17:34.434644 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.305615 32112 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.264003  1298 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:17:34.299371 32116 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:34.437880 31971 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:34.440531 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69789.7us
I20260709 12:17:34.444357 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:34.438930 31869 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.440243 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1361) took 1445 ms. Trace:
I20260709 12:17:34.449738 32044 rpcz_store.cc:276] 0709 12:17:32.994855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.827659 (+832804us) service_pool.cc:224] Handling call
0709 12:17:33.865108 (+ 37449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.865387 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:34.202127 (+336740us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.439830 (+237703us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.439900 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.439956 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.440041 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.440178 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60925,"prepare.queue_time_us":263471,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":62675,"thread_start_us":471592,"threads_started":2}
I20260709 12:17:34.322460 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1559) took 1110 ms. Trace:
I20260709 12:17:34.367942  1102 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:34.367141   920 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.436962   869 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.452322 31971 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:34.452688  1327 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.437860  1238 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:34.322281 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:34.454017 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 207766us
I20260709 12:17:34.460327 32039 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:34.310964  1335 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.344442 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:34.349977  1333 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:34.466856 32047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58386
I20260709 12:17:34.468531 31972 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:34.368516 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1551) took 1625 ms. Trace:
I20260709 12:17:34.362917 31966 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:34.382632  1287 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.380995  1208 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.474110 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.518242 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.354054  1272 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:34.425511   951 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:34.380769  1336 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.381824  1261 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.483007 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.519941 32039 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:34.461006  1120 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.424985  1248 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.452769  1331 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:17:34.320159 32041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d2c5dc46254473902eca585b82958f"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:34.466192  1276 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:34.443418  1295 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:34.454593   975 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.486733 31869 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 3 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.425452  1119 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:17:34.380865  1259 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.508050 32047 raft_consensus.cc:606] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 12:17:34.507267 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:34.511266 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.480006  1102 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.512318 32120 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:34.459604  1330 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:34.551491 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.441049  1249 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:17:34.325977 31964 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:34.480054  1302 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.479103   767 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:34.480713   770 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.441092  1286 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:17:34.437220  1337 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:17:34.297087  1239 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.367379   964 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.479333  1000 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:17:34.380693  1334 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.773526  1000 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:34.774186 32116 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:34.754498   869 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:34.778488  1347 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:17:34.758531 32114 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:34.781195 32114 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:34.767377   770 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.736683 32047 raft_consensus.cc:614] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:34.532021 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 106) took 1029 ms. Trace:
I20260709 12:17:34.788046 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:34.788323 31968 rpcz_store.cc:276] 0709 12:17:33.502673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.351412 (+848739us) service_pool.cc:224] Handling call
0709 12:17:34.351736 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.351920 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:34.352212 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.531652 (+179440us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.531715 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:34.531762 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.531829 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.531963 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":559,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":11753,"raft.queue_time_us":2849,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":179118,"threads_started":1,"wal-append.queue_time_us":420502}
I20260709 12:17:34.264261 32038 rpcz_store.cc:276] 0709 12:17:32.173912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.870692 (+696780us) service_pool.cc:224] Handling call
0709 12:17:32.938797 (+ 68105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:33.013370 (+ 74573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.022638 (+  9268us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.146491 (+123853us) log.cc:844] Serialized 26 byte log entry
0709 12:17:34.126272 (+979781us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.131303 (+  5031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.191118 (+ 59815us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:34.203137 (+ 12019us) raft_consensus.cc:1685] finished
0709 12:17:34.204872 (+  1735us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.216061 (+ 11189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15134,"raft.queue_time_us":53076,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":15538,"thread_start_us":913732,"threads_started":1,"wal-append.queue_time_us":723433}
I20260709 12:17:34.791109 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:34.791680 32038 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.515377  1354 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:34.794649  1354 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:34.795557  1354 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:34.797032  1305 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.000s
I20260709 12:17:34.543599  1300 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:34.798955  1300 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:34.799590  1259 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.799764  1300 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.595734  1274 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.800792  1300 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.513178  1294 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.596263 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1221) took 2741 ms. Trace:
I20260709 12:17:34.802585  1294 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.802537 31969 rpcz_store.cc:276] 0709 12:17:31.854489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.867652 (+ 13163us) service_pool.cc:224] Handling call
0709 12:17:33.503472 (+1635820us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:33.503742 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.086870 (+583128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.511521 (+424651us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400068e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:34.551163 (+ 39642us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103216384}
I20260709 12:17:34.803486 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1404) took 1043 ms. Trace:
I20260709 12:17:34.804064 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.526038  1208 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.804836  1294 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.804971  1208 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.741586  1276 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:17:34.742097 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:34.813877 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.598703 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.744007 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:34.512565  1335 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.525511 32044 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:34.763825  1331 consensus_queue.cc:1243] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:34.759805  1261 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.765672 32041 raft_consensus.cc:606] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.818383 32011 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:34.821475 32011 leader_election.cc:403] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:34.747579  1295 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.748067   975 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.748844 31869 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.466396 31965 rpcz_store.cc:276] 0709 12:17:33.166627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.203854 (+ 37227us) service_pool.cc:224] Handling call
0709 12:17:34.276614 (+1072760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.751005  1336 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.827859  1336 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.752379 32120 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:34.828325 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.828761 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1412) took 1010 ms. Trace:
I20260709 12:17:34.828995 31965 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:34.829115 32120 rpcz_store.cc:276] 0709 12:17:33.818534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.751600 (+933066us) service_pool.cc:224] Handling call
0709 12:17:34.751962 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.828523 (+ 76561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.828707 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.504235   920 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:34.757625 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:34.830216   920 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:34.830770 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73236.2us
I20260709 12:17:34.831786  1335 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:34.831979 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.832258   920 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:34.832322  1335 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:34.832821  1335 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:34.826709  1104 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.965s	user 0.013s	sys 0.002s
W20260709 12:17:34.766151 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 312456us
I20260709 12:17:34.833766 31969 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:34.834189 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.515015 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1363) took 1518 ms. Trace:
W20260709 12:17:34.834957 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 475400us
I20260709 12:17:34.760094 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.513775  1333 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.493479  1275 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.526239   951 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:34.790159 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:34.839542 31968 raft_consensus.cc:2376] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6. Re-sending same reply.
I20260709 12:17:34.738749 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.606526  1291 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:34.586795 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 152206us
I20260709 12:17:34.803929 32112 rpcz_store.cc:276] 0709 12:17:33.760060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.503949 (+743889us) service_pool.cc:224] Handling call
0709 12:17:34.596226 (+ 92277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.734119 (+137893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.738265 (+  4146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.751708 (+ 13443us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:34.760037 (+  8329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.767599 (+  7562us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.803420 (+ 35821us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:34.759590  1350 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:17:34.807085 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 209119us
I20260709 12:17:34.808502 32045 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:34.781890 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1362) took 1786 ms. Trace:
I20260709 12:17:34.816640 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.845873 32653 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:34.846283 32044 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:34.504855  1345 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.239s
I20260709 12:17:34.819192 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.822477 32041 raft_consensus.cc:614] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:17:34.855225 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 95207.4us
I20260709 12:17:34.855955 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.765277  1330 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:34.845686 32043 rpcz_store.cc:276] 0709 12:17:32.995499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.018845 (+1023346us) service_pool.cc:224] Handling call
0709 12:17:34.020003 (+  1158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.052138 (+ 32135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.052167 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:34.066725 (+ 14558us) log.cc:844] Serialized 26 byte log entry
0709 12:17:34.193783 (+127058us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.194079 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.227101 (+ 33022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.781316 (+554215us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b5400056ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:34.781405 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:34.781477 (+    72us) raft_consensus.cc:1685] finished
0709 12:17:34.781592 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.781807 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32764,"prepare.queue_time_us":929,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":36835,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":16785,"spinlock_wait_cycles":72959232,"thread_start_us":125206,"threads_started":1,"wal-append.queue_time_us":125534}
I20260709 12:17:34.823742 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:34.806846   770 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:34.833176  1335 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.517776 32115 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.770354   767 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:34.830025 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:34.849052 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.511636 31963 rpcz_store.cc:276] 0709 12:17:32.742636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.034936 (+1292300us) service_pool.cc:224] Handling call
0709 12:17:34.035232 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.035427 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:34.035769 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.261922 (+226153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.265152 (+  3230us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.271872 (+  6720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.278062 (+  6190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.355549 (+ 77487us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82905,"prepare.queue_time_us":10378,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":71095,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6984,"thread_start_us":156955,"threads_started":1,"wal-append.queue_time_us":158688}
I20260709 12:17:34.767855  1302 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.852387 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.875572 32044 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:34.876420 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.852392 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:34.862957 32047 raft_consensus.cc:2368] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:34.865056 32040 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:34.865796  1270 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.562s	user 0.004s	sys 0.005s
I20260709 12:17:34.878098 32039 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:34.869203 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:34.843122   951 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.878567 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 105) took 1378 ms. Trace:
I20260709 12:17:34.850633   933 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.833710  1104 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:34.872747   976 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.000s
I20260709 12:17:34.874056 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:34.874509 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 51015.4us
I20260709 12:17:34.878942 32043 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:34.879377 32039 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:34.880134 32044 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:34.878925 31970 rpcz_store.cc:276] 0709 12:17:33.502346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.241751 (+739405us) service_pool.cc:224] Handling call
0709 12:17:34.242075 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.306991 (+ 64916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.307037 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:34.307653 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 12:17:34.507914 (+200261us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:34.605796 (+ 97882us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.606135 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.854562 (+248427us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b54001ce654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:34.854612 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:34.872275 (+ 17663us) raft_consensus.cc:1685] finished
0709 12:17:34.878389 (+  6114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.878512 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30623,"prepare.queue_time_us":130827,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":65538,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"spinlock_wait_cycles":140406400,"thread_start_us":199838,"threads_started":1,"wal-append.queue_time_us":325685}
I20260709 12:17:34.836222  1261 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.865146  1289 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.846997  1359 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:17:34.843382  1220 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:17:34.865442   920 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:34.879413  1294 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:34.872972  1345 consensus_queue.cc:1243] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:17:34.896514 32659 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.000s
I20260709 12:17:34.896602 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.872622 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.918880 32659 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:34.919081 31970 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:34.900578 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.876227  1115 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.892012 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.864852  1034 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:17:34.878186  1280 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:34.862131  1356 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:17:34.905055  1338 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.923013 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:34.905375  1289 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.920035 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.934540 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.881695  1038 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:17:34.939848 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:34.883697  1270 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.916348   976 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.887535   933 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.903213  1206 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:17:34.960529 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:35.000046 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 133408us
I20260709 12:17:34.955360 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:35.007246 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:34.935827   920 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:34.933954  1294 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:35.016731  1115 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.931205 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:35.023411 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68156.9us
I20260709 12:17:34.945148  1280 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.932510  1083 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.941409  1338 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.945696  1249 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.945935  1289 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.925518 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:35.017204 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56799.9us
I20260709 12:17:34.926177  1336 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:34.995193  1298 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:35.050196 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:35.053100 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.073715 31972 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:35.002122  1274 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:35.047446 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.061748 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:34.999567   964 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:34.998999  1345 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.921428  1276 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.007054   770 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:17:35.028716  1115 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.077015 32041 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:34.995803  1358 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:17:35.080718 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:35.057492  1338 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.052372  1083 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.052491  1248 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:34.256673  1095 consensus_queue.cc:1175] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ec7f26ea46944ea498ad253b1b1819b0" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:35.061234 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:35.077509 32113 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:35.054724   998 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:35.095219 31971 raft_consensus.cc:2368] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:35.059333 32653 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:35.014150   976 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.085181  1249 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 12:17:35.118216 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 102062us
I20260709 12:17:35.097875 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.100311 32044 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:34.835043 32042 rpcz_store.cc:276] 0709 12:17:32.996249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.028861 (+1032612us) service_pool.cc:224] Handling call
0709 12:17:34.041793 (+ 12932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.202792 (+160999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.205394 (+  2602us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:34.263044 (+ 57650us) log.cc:844] Serialized 26 byte log entry
0709 12:17:34.451518 (+188474us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.468170 (+ 16652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.483176 (+ 15006us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:34.489027 (+  5851us) raft_consensus.cc:1685] finished
0709 12:17:34.493028 (+  4001us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.513524 (+ 20496us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13563,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":238252,"threads_started":2,"wal-append.queue_time_us":152079}
I20260709 12:17:35.082506  1034 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:35.093849 31963 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:35.103696  1083 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.048556  1372 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:35.085305  1275 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.021143   767 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:35.109509 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48353.8us
I20260709 12:17:35.161700 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:35.106828 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:35.099471  1115 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:35.094508 31964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1560) took 1927 ms (client timeout 2100 ms). Trace:
I20260709 12:17:35.080134  1298 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.103709  1295 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.144640 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:35.188993 32120 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:35.136577  1373 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:35.184298 31964 rpcz_store.cc:269] 0709 12:17:33.166861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.441409 (+274548us) service_pool.cc:224] Handling call
0709 12:17:35.082702 (+1641293us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400455bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:35.089292 (+  6590us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233275776}
I20260709 12:17:35.196324 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:35.196346 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.196965 32042 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.198441 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 112) took 1112 ms. Trace:
W20260709 12:17:35.198859  1366 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:17:35.198825 31964 rpcz_store.cc:276] 0709 12:17:34.085637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.197475 (+1111838us) service_pool.cc:224] Handling call
0709 12:17:35.197778 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.198075 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.198104 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:35.198150 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:35.198179 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.198240 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:35.198385 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:35.179495   999 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.200750   964 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.184290  1356 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:17:35.152838  1374 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:35.205893  1374 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.138156 31972 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:35.105652  1302 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.084707  1274 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.107566  1359 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.108191  1238 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:35.014623  1330 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:35.165484  1370 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:17:35.165396 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59145.6us
I20260709 12:17:35.219183  1038 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:35.219806  1034 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:35.165916  1210 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.221413 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1405) took 1404 ms. Trace:
I20260709 12:17:35.168269  1208 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:35.170526  1372 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:35.170877  1363 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:17:35.221849 32117 rpcz_store.cc:276] 0709 12:17:33.816602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.510912 (+694310us) service_pool.cc:224] Handling call
0709 12:17:34.792601 (+281689us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:34.792846 (+   245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.793279 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.793317 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:34.793917 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:17:35.218749 (+424832us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:35.218795 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.218862 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:35.221084 (+  2222us) raft_consensus.cc:1685] finished
0709 12:17:35.221106 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:35.221334 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7115,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"spinlock_wait_cycles":51417856,"thread_start_us":424385,"threads_started":1,"wal-append.queue_time_us":424765}
I20260709 12:17:35.236083 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:35.171092  1365 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260709 12:17:35.171492  1371 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 12:17:35.171835  1367 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:35.172405 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:35.173679 32046 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:17:35.243494 32046 pending_rounds.cc:85] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 5
I20260709 12:17:35.243873 32046 pending_rounds.cc:107] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:17:35.244172 32046 raft_consensus.cc:2889] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:35.175832   933 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.134904   998 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.184685 32039 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:35.186156  1036 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:35.148910  1261 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.189819 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:35.146674 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:35.175871 31966 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:35.191313  1329 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.003s
I20260709 12:17:35.191614  1336 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.194229  1298 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.194695 32659 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:35.214079 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:35.223481 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:35.309396 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85994.9us
I20260709 12:17:35.311396 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:35.223567 32045 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:35.184118 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:35.189254   976 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.117405  1294 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.246758 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 133) took 1996 ms. Trace:
I20260709 12:17:35.314796 32046 rpcz_store.cc:276] 0709 12:17:33.251594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.116422 (+864828us) service_pool.cc:224] Handling call
0709 12:17:34.116713 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.173340 (+1056627us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54000cb5a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d1d87 00007f228c1d2470 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c
0709 12:17:35.246454 (+ 73114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.246694 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920796544}
I20260709 12:17:35.316249 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:35.316823 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102851us
I20260709 12:17:35.317516 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:35.317770 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.246191 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.247263 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:35.224694  1238 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:35.291664 32121 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:35.244678  1302 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.244334 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.322466 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.323730 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.324586 32039 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:35.313226 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.294922 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:35.320194 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73053.3us
I20260709 12:17:35.333061 32045 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:35.333389 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:35.333648 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:35.334419 32118 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:35.320194 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:35.335148 32117 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.242162  1210 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:17:35.256703  1036 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:35.337070 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:35.265841 31963 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.240552   770 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
W20260709 12:17:35.341521 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 132482us
I20260709 12:17:35.352628  1374 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.314515  1249 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.345176 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.357496 32115 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:35.358103 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 116) took 1599 ms. Trace:
I20260709 12:17:35.358446 32115 rpcz_store.cc:276] 0709 12:17:33.758268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.280537 (+522269us) service_pool.cc:224] Handling call
0709 12:17:35.358016 (+1077479us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:35.354388  1336 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.359045  1286 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:35.359615  1336 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.359679  1286 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:35.360101  1208 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:35.325512  1276 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:35.347230 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.343077 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.319657  1373 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:35.363716 32038 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.004s
I20260709 12:17:35.321199 31966 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.366788  1373 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.370545 32117 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:35.320163   964 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.366083   920 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:35.340386  1377 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:35.360118  1336 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:35.367930 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.314378  1329 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.343587 32010 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
W20260709 12:17:35.491856 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 151982us
I20260709 12:17:35.340860  1302 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.319998   869 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.354331  1268 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.368884 32038 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:35.345134  1248 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:17:35.320412   951 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.356222 32039 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:35.676465 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.677624 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.492717 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:35.678550 32658 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:35.619554 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.679625 31970 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:17:35.375486  1300 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.000s
I20260709 12:17:35.359077 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.374150  1034 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.377918   964 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.680459 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:35.439893  1377 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:35.681392 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 188732us
I20260709 12:17:35.753896 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:35.478595   976 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.624167  1305 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.699646 32660 raft_consensus.cc:1217] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-2.5]   Dedup: 2.5->[]
I20260709 12:17:35.777665 32660 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:17:35.719484 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.625722   869 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:35.677893  1102 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.914s	user 0.006s	sys 0.005s
I20260709 12:17:35.688234 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.624523  1268 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:35.484836   920 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.645201  1357 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:35.680389  1256 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.213s	user 0.011s	sys 0.000s
I20260709 12:17:35.789638 31970 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:35.647221  1361 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:35.682042 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 138) took 1168 ms. Trace:
I20260709 12:17:35.700311  1345 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:35.838857   951 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.695473 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 350) took 1581 ms. Trace:
W20260709 12:17:35.839165 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.394663  1286 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.824208  1102 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.879859  1272 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.775717  1377 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.878827 32655 rpcz_store.cc:276] 0709 12:17:34.114059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.317685 (+203626us) service_pool.cc:224] Handling call
0709 12:17:34.318225 (+   540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:34.478602 (+160377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.482235 (+  3633us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:34.763524 (+281289us) log.cc:844] Serialized 52 byte log entry
0709 12:17:34.901300 (+137776us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:34.901369 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.901494 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:35.626189 (+724695us) raft_consensus.cc:1685] finished
0709 12:17:35.646024 (+ 19835us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:35.682210 (+ 36186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83747,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":154,"thread_start_us":102282,"threads_started":1,"wal-append.queue_time_us":236215}
I20260709 12:17:35.879912  1286 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.884521  1102 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:35.885867  1378 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:17:35.441404  1372 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:35.788892   975 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.943s	user 0.010s	sys 0.001s
I20260709 12:17:35.890092  1373 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.890576   975 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.892707   975 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:35.893690 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.894726 31967 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:17:35.859740 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:35.895861  1383 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.522s	user 0.001s	sys 0.000s
I20260709 12:17:35.900697   933 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.901324   933 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.904975   933 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.830761 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.780483  1034 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.906184 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 445) took 1548 ms. Trace:
I20260709 12:17:35.906601 32660 rpcz_store.cc:276] 0709 12:17:34.357613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.380845 (+ 23232us) service_pool.cc:224] Handling call
0709 12:17:35.659425 (+1278580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.778145 (+118720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.905996 (+127851us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540066e9d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:35.906127 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2939504512}
I20260709 12:17:35.907363  1034 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:35.907711  1329 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.854090  1357 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.847507  1302 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:35.879293 31963 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.002s
I20260709 12:17:35.854256   767 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.854780 32038 rpcz_store.cc:276] 0709 12:17:34.513699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.790855 (+277156us) service_pool.cc:224] Handling call
0709 12:17:35.676283 (+885428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:35.917733 31963 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:35.917888   767 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.918839 32038 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:35.919144 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.920126   767 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.920034 31963 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:35.822841  1268 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.910542 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:35.924335 31970 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.785856   976 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.759658   964 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:35.846304  1333 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.995s	user 0.010s	sys 0.000s
I20260709 12:17:35.856448  1274 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:35.787982   869 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.922465  1373 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:35.932361 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.021s	sys 0.031s
I20260709 12:17:35.759342  1238 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:35.793745  1305 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.925943  1239 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:35.935599  1289 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.857s	user 0.002s	sys 0.008s
I20260709 12:17:35.934163  1104 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:35.935253 31970 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
W20260709 12:17:35.937538 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.022s	sys 0.031s
W20260709 12:17:35.949383 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.09445e+06us
W20260709 12:17:35.979213 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 225429us
W20260709 12:17:35.981173 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:35.945951  1083 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.787s	user 0.006s	sys 0.004s
I20260709 12:17:35.933717  1333 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:35.946446  1238 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.988152 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:35.965312 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:35.939476  1373 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:35.988304 32042 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.001s
I20260709 12:17:35.933535  1377 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:35.947876  1330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.005s	sys 0.076s
I20260709 12:17:36.007841 32115 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:36.008553 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.009605   920 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.010561 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 122) took 1267 ms. Trace:
I20260709 12:17:35.961980  1289 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.014727  1000 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:35.991752  1083 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.028995 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 437) took 2563 ms. Trace:
W20260709 12:17:36.022106  1335 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.150s	user 0.008s	sys 0.000s
W20260709 12:17:36.047369 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.022207  1115 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.834s	user 0.006s	sys 0.002s
I20260709 12:17:36.055186 32121 rpcz_store.cc:276] 0709 12:17:34.742814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.743773 (+   959us) service_pool.cc:224] Handling call
0709 12:17:36.010374 (+1266601us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54003e49d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.010503 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094867712}
I20260709 12:17:36.024746  1333 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:36.029039  1385 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
W20260709 12:17:36.031073  1338 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.002s
I20260709 12:17:36.040740 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:36.003820 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.065932 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.068042 32657 rpcz_store.cc:276] 0709 12:17:33.466102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.467869 (+  1767us) service_pool.cc:224] Handling call
0709 12:17:33.468261 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.769943 (+1301682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.769990 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.770053 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:34.770108 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.028717 (+1258609us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54004c13d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.028776 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.028929 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992919296}
I20260709 12:17:36.015724 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1403) took 2255 ms. Trace:
I20260709 12:17:36.023383   976 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:36.053879  1330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.005s	sys 0.076s
I20260709 12:17:36.043138  1298 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:36.088510  1115 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.063863  1000 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:36.102084 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.102960 31970 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:36.089629  1357 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:36.096740 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 117841us
W20260709 12:17:36.098089 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 157927us
I20260709 12:17:36.099507 32112 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:36.094283  1275 raft_consensus.cc:2764] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:36.106283 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:36.090960  1335 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.095137  1338 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.111907 32045 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:36.100620   976 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.103494 31964 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:36.118304 32121 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:36.120242 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.111594  1248 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.121662  1345 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:36.130051 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1252) took 2544 ms. Trace:
I20260709 12:17:36.130393 31967 rpcz_store.cc:276] 0709 12:17:33.585743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.429853 (+844110us) service_pool.cc:224] Handling call
0709 12:17:34.430256 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.895233 (+1464977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.129871 (+234638us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54001ea0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.130010 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3367248128}
I20260709 12:17:36.131389 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.131937 31967 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:36.132655 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.132745  1330 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.139705 31967 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:36.133383 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.133787   998 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.134678 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.142676 32121 raft_consensus.cc:2368] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:36.134016 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.104583  1298 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.143640 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.094745 32116 rpcz_store.cc:276] 0709 12:17:33.759811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.503087 (+743276us) service_pool.cc:224] Handling call
0709 12:17:34.633099 (+130012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.774607 (+141508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.015405 (+1240798us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002fb054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.015649 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323636480}
I20260709 12:17:36.144198 32121 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.144934 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.146238 32013 leader_election.cc:400] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:36.125870 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.162220 32041 raft_consensus.cc:1217] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Deduplicated request from leader. Original: 1.4->[2.5-2.5]   Dedup: 2.5->[]
I20260709 12:17:36.162842 32041 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:36.163334 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:36.164551 31966 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.794s	user 0.004s	sys 0.004s
I20260709 12:17:36.138270 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 356) took 1331 ms. Trace:
I20260709 12:17:36.164652   964 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:36.166054  1289 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.138774  1333 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:36.166747  1289 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:36.167464  1272 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.167387  1289 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.168651  1298 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.169638 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.170450 31967 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:36.150484  1249 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:36.151288 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1402) took 2391 ms. Trace:
I20260709 12:17:36.152563 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.154227  1335 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.155157  1083 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.208474 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.155211  1115 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.155447  1329 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:36.155771  1358 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.144956  1330 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.156767 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.160531  1338 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.142623 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.170832  1407 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.115368  1395 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:17:36.171345 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.260844 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.261409 31967 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:36.262485 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.263348 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.134246  1000 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.286665 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 353) took 1395 ms. Trace:
I20260709 12:17:36.287048 32658 rpcz_store.cc:276] 0709 12:17:34.891531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.167511 (+275980us) service_pool.cc:224] Handling call
0709 12:17:35.167836 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.686500 (+518664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.286459 (+599959us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.286613 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174057856}
I20260709 12:17:36.287793 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.288018   964 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.172300 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:36.289166 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:36.289436  1335 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:36.289601 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 117364us
I20260709 12:17:36.289886  1335 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:36.290215 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 117) took 2304 ms. Trace:
I20260709 12:17:36.290292  1335 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:36.291568 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.291689 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.295327  1036 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.296399  1036 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.298602  1335 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:36.299125 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.136884  1305 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.300617  1280 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:36.300995  1280 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:36.301127 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:36.301331  1335 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.119745   976 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.214476 32121 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:36.301929 32660 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:36.302549 32043 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:36.302767 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.303296  1220 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:36.128907  1345 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.303872  1220 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.304535 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:36.305433 31971 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1563) took 1830 ms (client timeout 1577 ms). Trace:
W20260709 12:17:36.305805 31971 rpcz_store.cc:269] 0709 12:17:34.475117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.511811 (+ 36694us) service_pool.cc:224] Handling call
0709 12:17:36.304864 (+1793053us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001e9bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.304980 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777205504}
I20260709 12:17:36.216357 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.306891 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:36.307518 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.308221 32118 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:36.308624   976 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.308983 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:36.309120 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1562) took 1834 ms (client timeout 1704 ms). Trace:
I20260709 12:17:36.294121 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.309680 32118 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:36.166189  1397 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.310137 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1259) took 1082 ms. Trace:
I20260709 12:17:36.171185  1396 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:36.164525 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.226980  1407 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.226611  1289 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.162038   869 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.234404 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.218883  1358 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.166747  1333 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.334481  1334 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:36.335023  1396 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:36.334661 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.336233 32046 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:36.191264 32113 rpcz_store.cc:276] 0709 12:17:33.759567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.321915 (+562348us) service_pool.cc:224] Handling call
0709 12:17:36.151102 (+1829187us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5400138654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.151231 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201437568}
I20260709 12:17:36.245115 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1572) took 1767 ms. Trace:
I20260709 12:17:36.165334 32656 rpcz_store.cc:276] 0709 12:17:34.807215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.813364 (+  6149us) service_pool.cc:224] Handling call
0709 12:17:35.203086 (+389722us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:35.203311 (+   225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.203780 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.203825 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:35.204474 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:17:35.344345 (+139871us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:35.347455 (+  3110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.355341 (+  7886us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:36.120415 (+765074us) raft_consensus.cc:1685] finished
0709 12:17:36.121943 (+  1528us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.138198 (+ 16255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":344,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":22961920,"thread_start_us":50740,"threads_started":1,"wal-append.queue_time_us":453038}
I20260709 12:17:36.246119 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.225075  1337 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.219717  1390 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:36.248679  1398 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:17:36.361917  1337 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.367478  1407 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.368059   869 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.301862  1280 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.369899  1280 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:36.256165 32651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 346) took 2387 ms (client timeout 1522 ms). Trace:
I20260709 12:17:36.208724  1249 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:36.265374 32041 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 12:17:36.268010  1400 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:36.267743  1268 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.269063  1377 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.269407 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1345) took 1041 ms. Trace:
I20260709 12:17:36.274425 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.226024 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.275463 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.276396  1330 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:36.279947  1336 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 12:17:36.262290  1373 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.281710  1276 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:36.282079  1359 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.046s	user 0.007s	sys 0.004s
I20260709 12:17:36.376771  1390 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.396463  1359 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:36.284181  1270 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:36.397500 32120 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:36.284868 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 131) took 3098 ms. Trace:
I20260709 12:17:36.398149 32040 rpcz_store.cc:276] 0709 12:17:33.187192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.063896 (+876704us) service_pool.cc:224] Handling call
0709 12:17:34.072824 (+  8928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.877466 (+804642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.284652 (+1407186us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400032ad4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.284809 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818009344}
I20260709 12:17:36.399861 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.400588 32040 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:36.285813 32659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 360) took 1388 ms (client timeout 1625 ms). Trace:
W20260709 12:17:36.401494 32659 rpcz_store.cc:269] 0709 12:17:34.897542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.919545 (+ 22003us) service_pool.cc:224] Handling call
0709 12:17:36.285658 (+1366113us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400543654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.285759 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446299008}
I20260709 12:17:36.289297 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 447) took 1438 ms. Trace:
I20260709 12:17:36.290686 32119 rpcz_store.cc:276] 0709 12:17:34.005191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.740958 (+735767us) service_pool.cc:224] Handling call
0709 12:17:34.741194 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.902831 (+1161637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.902876 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:35.902945 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:35.902990 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.290014 (+387024us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.290056 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.290162 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40004,"raft.queue_time_us":40534,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":2563384064}
I20260709 12:17:36.405324 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.308079 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:36.406234 32657 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.310539 31972 rpcz_store.cc:276] 0709 12:17:35.227505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.293863 (+1066358us) service_pool.cc:224] Handling call
0709 12:17:36.310084 (+ 16221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.165350 31966 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:36.306932 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.147356 32013 leader_election.cc:403] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:36.388814 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 117) took 1202 ms. Trace:
I20260709 12:17:36.312299 31936 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:17:36.314457 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.412732 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.188060 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:36.321882 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 171951us
I20260709 12:17:36.413661 32117 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:36.413875 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.322575 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1267) took 1085 ms. Trace:
I20260709 12:17:36.311247 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.414618 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.415515 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.415642 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.332242 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.310595  1397 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.332897 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:36.329392 32047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 139) took 1813 ms (client timeout 1547 ms). Trace:
W20260709 12:17:36.423816 32047 rpcz_store.cc:269] 0709 12:17:34.516238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.787783 (+271545us) service_pool.cc:224] Handling call
0709 12:17:36.329192 (+1541409us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b540003a054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.329320 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153939712}
I20260709 12:17:36.332899 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:36.339313 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.340996 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:36.426388 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 85491us
I20260709 12:17:36.335739 32115 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.335899  1333 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.418689 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.423435  1120 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:36.428117 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:36.282126  1298 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.391628  1336 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.346138 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.350330 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1354) took 1042 ms. Trace:
I20260709 12:17:36.352331 32039 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:36.435488 31970 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:36.435240 32046 rpcz_store.cc:276] 0709 12:17:35.308006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.317272 (+  9266us) service_pool.cc:224] Handling call
0709 12:17:36.350124 (+1032852us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001937d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.350244 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2341398400}
I20260709 12:17:36.431258 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.436710 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.437402 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.352933 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.351943  1294 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.335903  1396 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.341735  1331 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:36.332269  1220 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.438172 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.446715 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1420) took 2091 ms. Trace:
I20260709 12:17:36.370416 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.448297 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.447214 32114 rpcz_store.cc:276] 0709 12:17:34.355756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.782274 (+426518us) service_pool.cc:224] Handling call
0709 12:17:34.834634 (+ 52360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.834844 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:34.905877 (+ 71033us) log.cc:844] Serialized 14 byte log entry
0709 12:17:36.258425 (+1352548us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b1800347b60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:17:36.439645 (+181220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:36.446371 (+  6726us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:36.446436 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.446529 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.446657 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45523,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":3818,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7346,"spinlock_wait_cycles":2687872,"thread_start_us":213564,"threads_started":2}
I20260709 12:17:36.249980 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.336472 32112 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:36.309506 31969 rpcz_store.cc:269] 0709 12:17:34.474797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.803781 (+328984us) service_pool.cc:224] Handling call
0709 12:17:36.308950 (+1505169us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54001e96d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.309056 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66573696}
W20260709 12:17:36.450162 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 200250us
I20260709 12:17:36.450498 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.451326 31969 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:36.451447 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.452055 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:17:36.452620 31969 raft_consensus.cc:606] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:36.452648 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.453065 31969 raft_consensus.cc:614] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:36.388551 32045 rpcz_store.cc:276] 0709 12:17:35.228115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.342861 (+114746us) service_pool.cc:224] Handling call
0709 12:17:36.269219 (+926358us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b540013ced4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.269346 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722846464}
I20260709 12:17:36.453815  1286 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.390102 31963 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:36.454595 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1574) took 1212 ms. Trace:
I20260709 12:17:36.454818  1220 raft_consensus.cc:2764] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:36.390141 31968 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:36.356431 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.455211  1298 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:36.374627 32651 rpcz_store.cc:269] 0709 12:17:33.868880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.962354 (+ 93474us) service_pool.cc:224] Handling call
0709 12:17:36.255904 (+2293550us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54005eb5d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.256075 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1834585472}
I20260709 12:17:36.456210 31968 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:36.408697 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 262641us
I20260709 12:17:36.449697 32113 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.441582  1391 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:36.459151  1391 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:36.353489  1248 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:36.414450 31967 rpcz_store.cc:276] 0709 12:17:35.236939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.263167 (+1026228us) service_pool.cc:224] Handling call
0709 12:17:36.322499 (+ 59332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.454481  1286 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.460392  1286 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.416594 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:36.461350 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44844.8us
I20260709 12:17:36.376823  1337 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:36.419872 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 107643us
W20260709 12:17:36.393679 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.438566 32043 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:36.432924 32660 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:36.441227  1372 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:36.429348  1082 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:17:36.411604 31964 rpcz_store.cc:276] 0709 12:17:35.184950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.201374 (+ 16424us) service_pool.cc:224] Handling call
0709 12:17:35.201682 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:36.121626 (+919944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.375194 (+253568us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b5400200154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.387020 (+ 11826us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049866496}
I20260709 12:17:36.430228 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.446820 32046 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.454416  1294 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.303897 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.456810 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1258) took 1229 ms. Trace:
I20260709 12:17:36.440209 32040 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:36.492193 31968 rpcz_store.cc:276] 0709 12:17:35.227201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.455636 (+1228435us) service_pool.cc:224] Handling call
0709 12:17:36.455932 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.456579 (+   647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.456759 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.494256 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.460853  1286 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.404912  1276 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.459579  1391 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.416160 32117 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:36.499361 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.499619 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.463724 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.456674 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46882 (request call id 4) took 1112 ms. Trace:
I20260709 12:17:36.483582 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1260) took 1255 ms. Trace:
I20260709 12:17:36.464670 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.431918 32042 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:36.455003 31963 rpcz_store.cc:276] 0709 12:17:35.241938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.225729 (+983791us) service_pool.cc:224] Handling call
0709 12:17:36.454541 (+228812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.392047  1036 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:36.460752 31967 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:36.484655 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.459736 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:36.485955 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.464136 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.486529 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.419231  1373 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.488442 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.431180  1377 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:36.431032  1397 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.497499 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:36.457588  1298 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.497542 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1268) took 1259 ms. Trace:
W20260709 12:17:36.564159 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 72542.5us
I20260709 12:17:36.564311 31968 rpcz_store.cc:276] 0709 12:17:35.238278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.496800 (+1258522us) service_pool.cc:224] Handling call
0709 12:17:36.497074 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.497235 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.497259 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:36.497298 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:36.497328 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.497394 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.497492 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.500710 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.504473 32046 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:17:36.565485 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 105814us
I20260709 12:17:36.566097 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.566848 32046 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:17:36.566991 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102925us
I20260709 12:17:36.428628 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.567867 32119 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:36.568641 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.569478 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.434440  1359 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:36.558032 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72153.2us
I20260709 12:17:36.487869  1381 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.490583 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.560549 32045 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:36.536562 32657 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:36.461973 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1554) took 3314 ms. Trace:
I20260709 12:17:36.560842 31970 rpcz_store.cc:276] 0709 12:17:35.227835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.124300 (+896465us) service_pool.cc:224] Handling call
0709 12:17:36.345927 (+221627us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:36.483515 (+137588us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8154880}
W20260709 12:17:36.565374 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.569633 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.431638  1120 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:36.565497 32121 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.489240   869 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:36.572191 32044 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:36.576713 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.632876 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.578279 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1580) took 1331 ms. Trace:
I20260709 12:17:36.572121  1289 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.557493 32116 raft_consensus.cc:2410] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:36.666675 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:36.667515 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.667979  1345 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.668143 32116 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:36.669142 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.584997 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1266) took 1348 ms. Trace:
I20260709 12:17:36.460867  1331 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.585798 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.344015 31965 rpcz_store.cc:276] 0709 12:17:34.477643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.197832 (+720189us) service_pool.cc:224] Handling call
0709 12:17:35.198401 (+   569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:35.309483 (+111082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.316276 (+  6793us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:35.760513 (+444237us) log.cc:844] Serialized 574 byte log entry
0709 12:17:36.123364 (+362851us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:36.125940 (+  2576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.135932 (+  9992us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:36.244854 (+108922us) raft_consensus.cc:1685] finished
0709 12:17:36.244893 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.245055 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":996,"raft.queue_time_us":20936,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":34512,"thread_start_us":194036,"threads_started":1,"wal-append.queue_time_us":316498}
I20260709 12:17:36.669699 32116 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.670989 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.671692  1331 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.562637 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.672098 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.672611 32118 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:36.487999  1372 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.673254 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:36.640259 32119 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:36.647362  1000 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:36.648396 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.675734 31971 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:36.404654 32654 rpcz_store.cc:276] 0709 12:17:34.851337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.104721 (+253384us) service_pool.cc:224] Handling call
0709 12:17:36.287993 (+1183272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.288196 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.288227 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:36.288271 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:36.288308 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.288381 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.289110 (+   729us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400543650. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.289230 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643051264}
I20260709 12:17:36.650408  1120 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.650627 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.715579 31972 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:36.650867 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.563436  1377 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:36.716451 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.653698 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.654923 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.574093  1407 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.655938 32121 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:36.724169 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.584120 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.658524 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.506268 31969 rpcz_store.cc:276] 0709 12:17:35.352448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.451877 (+1099429us) service_pool.cc:224] Handling call
0709 12:17:36.456603 (+  4726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.657387 31963 rpcz_store.cc:276] 0709 12:17:35.246926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.577338 (+1330412us) service_pool.cc:224] Handling call
0709 12:17:36.577706 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.577899 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.577944 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:36.577996 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:36.578037 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.578104 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.578218 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.463155  1249 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:36.659650 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83016.9us
I20260709 12:17:36.653914 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:36.735924 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 150219us
I20260709 12:17:36.736320 32119 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:36.589761  1298 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:17:36.737004 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.639055 31966 rpcz_store.cc:276] 0709 12:17:33.147385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.204642 (+1057257us) service_pool.cc:224] Handling call
0709 12:17:34.242097 (+ 37455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.175454 (+933357us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54001f07a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14d06f 00007f228c1c5f60 00007f228c1c77de 00007f228c1d1b49 00007f228c1d2416 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62
0709 12:17:36.410119 (+1234665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.443051 (+ 32932us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622011648}
I20260709 12:17:36.662883 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.580091  1330 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.668715  1345 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.669682 31967 rpcz_store.cc:276] 0709 12:17:35.236720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.460313 (+1223593us) service_pool.cc:224] Handling call
0709 12:17:36.460583 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.582422 (+121839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.584944 (+  2522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.751569 31967 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:36.752480 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.752813 31967 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:36.490183  1034 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 12:17:36.675993 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 144) took 1452 ms. Trace:
I20260709 12:17:36.676193 32118 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [1.3-5.4]
I20260709 12:17:36.756691  1345 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.676764 32117 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:36.643079 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:36.677863 32120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1428) took 1492 ms (client timeout 1976 ms). Trace:
I20260709 12:17:36.756431 32039 rpcz_store.cc:276] 0709 12:17:35.223179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.792304 (+569125us) service_pool.cc:224] Handling call
0709 12:17:35.792763 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:36.435351 (+642588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.675779 (+240428us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.675937 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286263040}
W20260709 12:17:36.757802 32120 rpcz_store.cc:269] 0709 12:17:35.185572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.189577 (+  4005us) service_pool.cc:224] Handling call
0709 12:17:36.677661 (+1488084us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001234d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:36.677794 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2621852416}
I20260709 12:17:36.758549 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:36.758663 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 65127.5us
I20260709 12:17:36.759063 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.580614  1255 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.711990 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:36.762799 32658 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:36.558528  1248 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 12:17:36.639380  1356 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.581022  1272 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:36.580303   976 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:36.721022 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.771821 31968 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.726157 32045 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:17:36.773449 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.713008  1104 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:17:36.750344  1333 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:36.588178  1395 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:36.725590 32047 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.738772 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1343) took 1511 ms. Trace:
I20260709 12:17:36.589955 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:36.735754 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.674666  1372 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:36.724181  1377 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.656082  1390 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:36.742570 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 88703.9us
I20260709 12:17:36.745268 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.671806  1358 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.658666  1291 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 12:17:36.744781  1275 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.678449 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:36.759541 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.681589  1359 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:36.588742  1381 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.808001 32046 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:17:36.808525 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72846.9us
W20260709 12:17:36.792443 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 141677us
I20260709 12:17:36.808260 32044 rpcz_store.cc:276] 0709 12:17:35.227162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.899341 (+672179us) service_pool.cc:224] Handling call
0709 12:17:35.899726 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.656661 (+756935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.738568 (+ 81907us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54000339d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:36.738723 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541417216}
I20260709 12:17:36.814649 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:36.761272 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 118300us
I20260709 12:17:36.739939 32041 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:36.758034 31972 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:36.816879 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1274) took 1375 ms. Trace:
I20260709 12:17:36.817305 31972 rpcz_store.cc:276] 0709 12:17:35.441905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.724910 (+1283005us) service_pool.cc:224] Handling call
0709 12:17:36.757749 (+ 32839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.816608 (+ 58859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.816813 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.723712 31964 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.762989 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:36.818861 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55986us
W20260709 12:17:36.821213 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.821133 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.759332 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:36.723093  1300 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:36.584805  1036 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.833833 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.835067 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.767232 32120 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:36.730707   869 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:36.797324  1289 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.774739 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.726301 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.841995 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.776826 32119 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:36.842125 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:36.779013   999 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.779666  1274 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.781035 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:36.843922 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:36.844527 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 63590.9us
I20260709 12:17:36.781749  1286 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.785039 32118 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:36.787729  1294 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.762472  1255 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.764436  1298 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:36.833346 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74087us
I20260709 12:17:36.739042  1407 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.744122 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:36.859463 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:36.852497  1082 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.861389 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:36.847995  1333 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:36.862986 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.863098  1333 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.864143 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.864521  1276 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.850773  1336 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.834868  1345 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.771471  1356 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.767031   964 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.839879 32120 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:36.769065  1272 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.844733 32121 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:36.782485  1245 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:36.850067  1220 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:17:36.853616  1331 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:17:36.866521  1034 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:17:36.847549 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:36.873129  1220 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.873420  1245 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:36.850286 32118 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:36.874107 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:36.874248   976 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:36.837275 31965 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.792801  1268 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.853477 32657 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 1.
I20260709 12:17:36.853396 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:36.790234 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.827713  1334 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.830014  1275 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:36.866607  1372 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:17:36.845448 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.804093  1329 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:36.868438 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.866042 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.853127  1397 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.837859  1402 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.895387  1305 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.841744  1289 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.899591  1305 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.899736  1289 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.900005  1305 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.837567  1300 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:36.878293 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67177.7us
I20260709 12:17:36.866956 31972 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:36.882030 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65350.2us
I20260709 12:17:36.913717 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.876863  1270 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:17:36.847643  1274 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:36.851414  1358 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.860729 32042 raft_consensus.cc:2368] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:36.848322   999 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:36.871425 32116 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:36.801981 32115 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:36.891500 32043 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.891702 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:36.854179  1255 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.955386  1407 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.890384  1276 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:17:36.922058 32114 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:36.890256  1294 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:36.879133  1272 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.925292  1402 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:36.834103 32113 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.959635  1402 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.960139  1402 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.893846 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:36.963151 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 69347.6us
I20260709 12:17:36.964092 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:36.866921  1313 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.920548 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.965904  1313 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.916250  1327 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.966866  1313 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:36.894454  1345 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:17:36.856901 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1579) took 1610 ms. Trace:
I20260709 12:17:36.867478  1356 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.860749  1286 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:36.867549  1390 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.834302  1395 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.970595  1395 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.884285  1338 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.971175  1338 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:36.971230  1294 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.923233  1358 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.971927  1358 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.971868  1294 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.972190  1338 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.879212  1359 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:36.890070  1372 consensus_queue.cc:1243] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:36.890626  1275 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.975389  1372 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.975903  1372 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.976563 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:36.849030   920 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.917534 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:36.884521  1337 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:36.870147  1036 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.977989  1337 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.888908 31966 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:36.884018  1334 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:36.892493  1329 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:36.900991  1280 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:36.929515 31972 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:36.978247 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60810.1us
I20260709 12:17:36.874167  1373 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:36.935286 31968 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:36.930193  1270 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:36.946220 32038 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:36.947729 31965 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:36.909190  1305 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.951349  1377 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.901036   964 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:17:36.946214  1000 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:36.964211  1300 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:36.894541   869 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:36.910445  1289 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.967890  1345 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:36.986202  1334 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.973532  1274 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:36.882583  1248 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:17:36.981818 32623 leader_election.cc:400] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:36.992712  1270 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:37.057534 32623 leader_election.cc:403] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:36.895056 32047 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:37.059128 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.059870 32047 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:36.988799 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.990985 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:36.994850 32045 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
W20260709 12:17:36.978843 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87247.1us
I20260709 12:17:37.062810 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:36.949519  1082 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.973649  1338 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.010838 32043 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:37.016001 32040 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:36.978936  1251 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:37.048314 31972 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:36.974998  1294 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:36.978358  1036 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:36.966101 32041 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:36.868638 32660 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:36.971323 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.006543  1395 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:37.069015 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87318.9us
I20260709 12:17:37.069782 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:36.986034  1337 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.016120  1120 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.090169 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:37.092062 32044 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:36.967816 31964 rpcz_store.cc:276] 0709 12:17:35.246668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.558659 (+1311991us) service_pool.cc:224] Handling call
0709 12:17:36.655423 (+ 96764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.833793 (+178370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.853059 (+ 19266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:37.054720 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:37.093816 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.094044 32045 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:37.097681 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.057818  1000 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.068162  1082 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:37.085734  1300 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:37.070753  1356 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:37.085284  1036 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:37.109496 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 235434us
I20260709 12:17:37.113335 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.071115 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.089423 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.113176 32046 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:37.112048 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:37.114923 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.116205 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.132916 32121 raft_consensus.cc:2410] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:17:37.106297 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 260924us
I20260709 12:17:37.109299  1373 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.102011  1291 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.121102  1300 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:37.121256  1120 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.121582  1270 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.102629  1330 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:37.090062  1251 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:37.125241 32113 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:37.125542  1255 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.107483  1336 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.128728  1381 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.128887 32041 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.099542 32042 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:37.131533 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:37.141152 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.114830 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1270) took 1822 ms. Trace:
I20260709 12:17:37.099527  1274 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.146704 32118 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:37.129766   920 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:17:37.147428 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.152374 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:37.114490  1391 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.539s	user 0.000s	sys 0.009s
I20260709 12:17:37.102514  1331 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.148838 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.217514 31967 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:37.218736 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 363183us
I20260709 12:17:37.218787 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.220239 31967 raft_consensus.cc:2410] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:37.154914 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.161900 32047 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:37.129777  1036 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.150297  1397 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.152889  1291 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.163866 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:37.160084 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:37.093556  1395 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.149173 31965 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:37.155429   964 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:37.155193  1372 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.178174 31972 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:37.179221 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.172363 31971 rpcz_store.cc:276] 0709 12:17:35.292335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.723842 (+1431507us) service_pool.cc:224] Handling call
0709 12:17:36.745969 (+ 22127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.746146 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:36.746445 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.107687 (+361242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.114549 (+  6862us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:37.114596 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.114661 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.114773 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4412,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":1267,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":9979,"thread_start_us":360909,"threads_started":1,"wal-append.queue_time_us":417475}
I20260709 12:17:37.156826 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:37.214607 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 250589us
I20260709 12:17:37.238056 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:37.238344 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 144595us
W20260709 12:17:37.216444 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.166263  1120 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 12:17:37.242296 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 82299.3us
W20260709 12:17:37.242821 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.243110 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.214474 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:37.240353 31972 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:37.213672   976 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.160581  1245 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.235587  1291 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.245179 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.260332 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.225319 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.252513 32657 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:17:37.254745 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.170190  1336 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.114048  1329 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.165427  1330 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.240419 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1360) took 1027 ms. Trace:
I20260709 12:17:37.271950 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:37.243865  1034 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:37.273663 31971 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:37.217324  1391 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.279738 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1269) took 2021 ms. Trace:
I20260709 12:17:37.257339 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.292586 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.280315 31970 rpcz_store.cc:276] 0709 12:17:35.258279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.739261 (+1480982us) service_pool.cc:224] Handling call
0709 12:17:36.758431 (+ 19170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.758632 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:36.770170 (+ 11538us) log.cc:844] Serialized 14 byte log entry
0709 12:17:36.854448 (+ 84278us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b18001e7540. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:17:37.279314 (+424866us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.279382 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:37.279435 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.279532 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.279669 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5129,"prepare.queue_time_us":11399,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":23592,"raft.queue_time_us":16821,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":78,"spinlock_wait_cycles":10681216,"thread_start_us":418200,"threads_started":2,"wal-append.queue_time_us":471973}
I20260709 12:17:37.286141 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1358) took 1076 ms. Trace:
I20260709 12:17:37.183636  1000 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.292495 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1392) took 1074 ms. Trace:
I20260709 12:17:37.296030 32046 rpcz_store.cc:276] 0709 12:17:36.218516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.808780 (+590264us) service_pool.cc:224] Handling call
0709 12:17:37.292222 (+483442us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b540037a354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:37.292418 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620980992}
I20260709 12:17:37.292534 32043 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.298249 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1394) took 1021 ms. Trace:
I20260709 12:17:37.243907  1104 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:37.260658  1397 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.268004  1286 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.301297 32046 rpcz_store.cc:276] 0709 12:17:36.276984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.297209 (+1020225us) service_pool.cc:224] Handling call
0709 12:17:37.297626 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.297846 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.297883 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.297937 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:37.297982 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.298053 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.298183 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:37.301970 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:37.284513   976 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:37.300485  1421 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:17:37.246603  1249 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:37.264793  1377 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.251538  1333 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:17:37.300045 31972 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:37.306275 32117 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:37.274900  1276 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:17:37.302000 32047 rpcz_store.cc:276] 0709 12:17:36.209930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.063702 (+853772us) service_pool.cc:224] Handling call
0709 12:17:37.161580 (+ 97878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.221518 (+ 59938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.285875 (+ 64357us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b54000569d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:37.286063 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224311680}
I20260709 12:17:37.278729  1430 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:37.314014 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71007.9us
I20260709 12:17:37.334897 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.311259   976 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.342197   999 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.343207   976 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.344401   976 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.346436 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.272166  1300 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:37.348140  1426 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:17:37.315636   920 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.315866 32114 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:17:37.352337 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:37.316097  1422 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:17:37.308537  1381 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.300873  1372 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.317718 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 124) took 1020 ms. Trace:
W20260709 12:17:37.318621 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 139503us
I20260709 12:17:37.321147 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1390) took 1138 ms. Trace:
I20260709 12:17:37.371232 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.371520 32040 rpcz_store.cc:276] 0709 12:17:36.183050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.494035 (+310985us) service_pool.cc:224] Handling call
0709 12:17:37.320966 (+826931us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54000348d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:37.321083 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130922240}
I20260709 12:17:37.246585  1251 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.294919  1437 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.305254  1359 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.272672 32041 rpcz_store.cc:276] 0709 12:17:36.210670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.128291 (+917621us) service_pool.cc:224] Handling call
0709 12:17:37.128642 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.177352 (+ 48710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.238443 (+ 61091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:37.311335   869 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:37.324029  1294 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.323951  1245 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:37.328152  1431 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:17:37.356366  1430 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.331450  1391 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.283743  1396 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:17:37.342294  1408 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.311653  1280 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.306900 32113 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:37.404662  1372 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.370860 31968 rpcz_store.cc:276] 0709 12:17:36.297503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.986814 (+689311us) service_pool.cc:224] Handling call
0709 12:17:36.987176 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:36.987368 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.987390 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.014688 (+ 27298us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.238735 (+224047us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:37.238769 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.238827 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.317510 (+ 78683us) raft_consensus.cc:1685] finished
0709 12:17:37.317533 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.317662 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":148061,"threads_started":1,"wal-append.queue_time_us":238450}
W20260709 12:17:37.378582 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 116745us
I20260709 12:17:37.333129  1327 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.313596  1419 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:17:37.400909 32121 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [1.3-5.4]
I20260709 12:17:37.330250  1330 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.371110  1255 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:37.374142  1034 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:37.355829  1082 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.356815  1275 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.372654 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.442591 32040 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:37.443557 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.399770   869 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:37.431993 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.372145  1358 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.408979  1437 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
W20260709 12:17:37.438357 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 144008us
I20260709 12:17:37.377352  1381 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.459272 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:37.460522 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:37.461282 32660 raft_consensus.cc:2376] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5. Re-sending same reply.
I20260709 12:17:37.462682 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:37.449153  1430 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.449648  1407 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:37.440083 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.451113  1336 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.452904  1291 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:37.453678  1438 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:17:37.436264  1395 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:37.422055  1408 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.408010  1294 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.454842 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.434388  1356 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.457266 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1362) took 1246 ms. Trace:
I20260709 12:17:37.464988 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1275) took 1805 ms. Trace:
I20260709 12:17:37.504840 32043 rpcz_store.cc:276] 0709 12:17:36.211324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.159932 (+948608us) service_pool.cc:224] Handling call
0709 12:17:37.292368 (+132436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.298879 (+  6511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.457017 (+158138us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540037a350. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:37.457190 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303859456}
I20260709 12:17:37.467470  1434 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:37.467547 32653 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:37.475250  1402 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.004s
I20260709 12:17:37.434167  1372 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.475610 32044 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:37.483552 32047 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:37.447171   869 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:37.493765 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70561us
I20260709 12:17:37.439911  1276 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:17:37.434376  1272 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.503218 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.455678  1082 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.506557 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.507861  1434 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.466718  1407 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.443338  1034 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.534443 32654 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:37.511907 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 133854us
I20260709 12:17:37.485074   964 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.531503 32040 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:37.422080  1397 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:37.535698 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:37.542128  1298 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:37.536921  1381 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.537895 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.602037  1298 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.603001 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:37.604213  1298 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.468694  1337 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.511336  1036 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.621641  1036 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.624514 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:37.624691  1289 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.538506 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.626561 32658 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.581717  1408 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.627467 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1406) took 1184 ms. Trace:
I20260709 12:17:37.627925 32047 rpcz_store.cc:276] 0709 12:17:36.442771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.325404 (+882633us) service_pool.cc:224] Handling call
0709 12:17:37.325680 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.533106 (+207426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.627261 (+ 94155us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54001a68d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:37.627413 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362552576}
I20260709 12:17:37.629010 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.629961 32047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
W20260709 12:17:37.539768  1428 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:17:37.550668 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:37.634783  1000 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:37.436723  1419 consensus_queue.cc:1243] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:37.503877 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.549144 32113 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6.
I20260709 12:17:37.562529 32039 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:37.571278 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.558866  1434 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.572599  1372 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:37.573035  1274 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.574172   999 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.504701  1331 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.574795  1286 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.647962 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.576344  1457 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.577920 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.649121 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.578594  1220 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:37.649771 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:37.649988 32121 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:37.650462 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.650993 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.579151  1407 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.651585 32118 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.619626  1272 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.652043  1407 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.652776 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.653331 31970 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:37.580608  1443 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:17:37.581027 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.654230 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:37.581820 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.654819 31970 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.655141 31964 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:37.656070 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.656620 31964 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:37.642975  1289 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:17:37.657737 31964 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:37.657902  1289 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:37.583576 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1395) took 1306 ms. Trace:
I20260709 12:17:37.658358  1289 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.587103 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:37.658495 32038 rpcz_store.cc:276] 0709 12:17:36.277312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.429968 (+152656us) service_pool.cc:224] Handling call
0709 12:17:37.583358 (+1153390us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54001946d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:37.583500 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043247872}
I20260709 12:17:37.659282  1289 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.588747  1294 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.591531 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:37.599267  1358 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.600430  1275 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.601169 31966 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:37.604174 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:37.478233  1334 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.567754 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:37.619618  1337 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.625181 31965 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.507550 31969 rpcz_store.cc:276] 0709 12:17:35.651261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.727262 (+1076001us) service_pool.cc:224] Handling call
0709 12:17:36.727641 (+   379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:36.728123 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.728164 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:36.728944 (+   780us) log.cc:844] Serialized 52 byte log entry
0709 12:17:36.930731 (+201787us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:36.954406 (+ 23675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.954494 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.439791 (+485297us) raft_consensus.cc:1685] finished
0709 12:17:37.439815 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.456446 (+ 16631us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7155,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8587,"thread_start_us":185541,"threads_started":1,"wal-append.queue_time_us":377046}
I20260709 12:17:37.631709 32047 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.631608 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:37.763326 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.641000 31971 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.641168 32045 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 5. Ops: [3.4-3.4]
W20260709 12:17:37.642243 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 193962us
I20260709 12:17:37.660422  1434 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.776271  1334 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:37.777395 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.534741  1402 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.648669  1034 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:37.651129 32114 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:37.561030  1448 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.659636 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.785606 32119 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.585210  1356 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:37.590709 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1276) took 1886 ms. Trace:
I20260709 12:17:37.791414 31963 rpcz_store.cc:276] 0709 12:17:35.703879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.729693 (+1025814us) service_pool.cc:224] Handling call
0709 12:17:36.730040 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:36.773488 (+ 43448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.773533 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:36.815938 (+ 42405us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.094937 (+278999us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:37.094996 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.590167 (+495171us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b540034d854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:37.590239 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.590303 (+    64us) raft_consensus.cc:1685] finished
0709 12:17:37.590342 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.590632 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17638,"raft.queue_time_us":5020,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":52590,"spinlock_wait_cycles":89939712,"thread_start_us":236096,"threads_started":1,"wal-append.queue_time_us":386320}
I20260709 12:17:37.654819 31968 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:17:37.692307 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:37.794965 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102701us
I20260709 12:17:37.795879 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:37.695142 32118 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
W20260709 12:17:37.696904 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 129259us
I20260709 12:17:37.699265 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:37.699856  1449 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:17:37.700806  1331 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.650033  1220 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:37.704252 31970 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:37.660444  1289 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:37.660784   964 consensus_queue.cc:1175] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "38c927f0ee104b57972899f5b27aa99b" responder_term: 6 status { last_received { term: 4 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [1.3-5.4]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:37.701768 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1363) took 1490 ms. Trace:
I20260709 12:17:37.709414 32041 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:37.826049 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:37.646080  1251 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.779129 32046 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:37.779397 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.770773 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:37.772466 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.769372 32047 raft_consensus.cc:614] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:37.780354 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:37.781195 32120 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:37.851147 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.851716 32120 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.853797  1408 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:37.854095 32119 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:37.854864  1408 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:37.781966 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.800724 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.797639 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:37.855752 31963 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
W20260709 12:17:37.855918 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58370.6us
I20260709 12:17:37.856262 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:37.815805 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.857417 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:37.857657  1248 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:37.858040 32119 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.858210  1248 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:37.858233  1272 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.839742  1305 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.859068  1272 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.859380  1248 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.860188  1272 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:37.767539  1433 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:17:37.861114  1465 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:17:37.861281  1433 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:37.862286  1433 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.863368 31964 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:37.778786  1434 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.831125 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 361) took 1537 ms. Trace:
I20260709 12:17:37.821853 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1410) took 1246 ms. Trace:
I20260709 12:17:37.837631  1432 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:37.782671  1337 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:37.781543 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.855070 32118 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:37.862725 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:37.830168 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1412) took 1169 ms. Trace:
I20260709 12:17:37.880328 32039 rpcz_store.cc:276] 0709 12:17:36.660944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.833505 (+172561us) service_pool.cc:224] Handling call
0709 12:17:37.829969 (+996464us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b540012fa54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:37.830107 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673420544}
I20260709 12:17:37.864938 31964 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:17:37.864972 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85648.4us
W20260709 12:17:37.867919 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87686.5us
I20260709 12:17:37.884428 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.863431  1433 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.811933  1356 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.823578  1372 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:17:37.819499  1034 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:37.822500  1334 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.855226  1408 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.864398  1291 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:37.893857 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:37.826370  1345 consensus_queue.cc:1175] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ec7f26ea46944ea498ad253b1b1819b0" responder_term: 6 status { last_received { term: 5 index: 4 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:37.875597 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1407) took 1391 ms. Trace:
I20260709 12:17:37.893747 32041 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.935257 32043 rpcz_store.cc:276] 0709 12:17:36.484196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.635259 (+1151063us) service_pool.cc:224] Handling call
0709 12:17:37.635551 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.635750 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:37.636108 (+   358us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.875194 (+239086us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.875255 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:37.875310 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.875403 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.875535 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2939,"prepare.queue_time_us":209,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":3074,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7462,"thread_start_us":235642,"threads_started":1}
I20260709 12:17:37.937774 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:37.881757 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:37.898788 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.870472 32120 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:37.876220  1331 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:37.874660  1454 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:37.909765 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:37.909987 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1440) took 1777 ms. Trace:
I20260709 12:17:37.883486  1432 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:37.908638 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:37.944528 32117 rpcz_store.cc:276] 0709 12:17:36.132066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.839242 (+707176us) service_pool.cc:224] Handling call
0709 12:17:36.839503 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.324954 (+485451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.901563 (+576609us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54002b57d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:37.909915 (+  8352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073052416}
I20260709 12:17:37.928851 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:37.910609   999 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:37.890060  1249 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.939682 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:37.940070 32043 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:37.944875 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46173us
I20260709 12:17:37.948283 32659 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:37.964174 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:37.886031  1450 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:37.949568 31938 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6.
I20260709 12:17:37.965193 31938 leader_election.cc:403] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:37.813583  1448 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.950086 32119 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:37.907595  1034 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:37.918648  1334 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.944244  1331 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.977035 32119 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:17:37.909273  1313 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:17:37.983815 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:38.000090 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.075199 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.944125  1454 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:38.076006  1454 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.946671  1245 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:17:38.078490 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 114398us
I20260709 12:17:38.078928  1245 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.079520 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.080087 32119 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:38.081876  1245 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:37.915146   869 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.951601  1251 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:38.015180 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.874926 32652 rpcz_store.cc:276] 0709 12:17:36.293811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.300247 (+  6436us) service_pool.cc:224] Handling call
0709 12:17:36.300562 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.658057 (+357495us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:37.147095 (+489038us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.378290 (+231195us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.418579 (+ 40289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.418656 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:37.418730 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.830869 (+412139us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54005eabd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:37.830922 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.831062 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4256,"prepare.queue_time_us":188474,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":262704,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":670903552,"thread_start_us":569278,"threads_started":4,"wal-append.queue_time_us":167391}
I20260709 12:17:37.956315  1291 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:38.021987 32660 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:38.003813  1034 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:37.915681  1461 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:17:38.011976  1336 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6.
W20260709 12:17:38.032639 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.038120 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.014176  1419 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.878140 32038 rpcz_store.cc:276] 0709 12:17:36.575815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.659484 (+1083669us) service_pool.cc:224] Handling call
0709 12:17:37.659774 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.659980 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:37.660288 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.816203 (+155915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.816269 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:37.816321 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.816397 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.816936 (+   539us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":623,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":7503,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":122485,"threads_started":1,"wal-append.queue_time_us":352786}
I20260709 12:17:38.003278  1450 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:37.963356   999 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.082227 32043 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:38.154882 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:38.156440 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:38.157750 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:38.004462 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55059.1us
I20260709 12:17:38.055958  1331 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.112907 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.023459  1432 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:38.116546 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.110889  1313 consensus_queue.cc:1243] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:38.112247 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:38.194149 32659 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.051930  1334 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.153017  1249 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:38.179919 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 270228us
W20260709 12:17:38.222558 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.093573  1402 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.169936  1251 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:38.231802 32659 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:38.113895  1300 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:38.118607  1082 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:38.223093  1467 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.288s	user 0.001s	sys 0.001s
I20260709 12:17:37.825932 32042 rpcz_store.cc:276] 0709 12:17:36.211677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.247526 (+1035849us) service_pool.cc:224] Handling call
0709 12:17:37.304553 (+ 57027us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.349575 (+ 45022us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:37.411506 (+ 61931us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.439752 (+ 28246us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.475021 (+ 35269us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.475175 (+   154us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:37.475452 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.508524 (+ 33072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.533879 (+ 25355us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.701547 (+167668us) raft_consensus.cc:1685] finished
0709 12:17:37.701573 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.701710 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26260,"prepare.queue_time_us":43815,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":33950,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":25502,"thread_start_us":18519,"threads_started":1,"wal-append.queue_time_us":142679}
I20260709 12:17:38.202847  1432 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.086473  1408 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.223304  1407 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:38.223654 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:38.243510  1336 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.247236  1419 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:38.219039  1291 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.236703  1249 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.259979 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:38.327639 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1605) took 1179 ms. Trace:
I20260709 12:17:38.327917 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1359) took 2117 ms. Trace:
I20260709 12:17:38.333556 32045 rpcz_store.cc:276] 0709 12:17:36.210286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.104651 (+894365us) service_pool.cc:224] Handling call
0709 12:17:37.146477 (+ 41826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.777913 (+631436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.327500 (+549587us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54001a5ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:38.327843 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133974400}
I20260709 12:17:38.332463 32040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6679c928e92e4f799b88357707650b60"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:38.335988 32040 raft_consensus.cc:606] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.336447 32040 raft_consensus.cc:614] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:38.271013  1300 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:38.333379 31972 rpcz_store.cc:276] 0709 12:17:37.148230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.255182 (+106952us) service_pool.cc:224] Handling call
0709 12:17:37.257714 (+  2532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.299919 (+ 42205us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:37.398800 (+ 98881us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.886705 (+487905us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.886966 (+   261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.887028 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:37.887089 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.327311 (+440222us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b54000b5754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:38.327375 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.327563 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22560,"prepare.queue_time_us":306997,"prepare.run_cpu_time_us":610,"prepare.run_wall_time_us":7337,"raft.queue_time_us":4902,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":11944,"spinlock_wait_cycles":96542464,"thread_start_us":419358,"threads_started":2,"wal-append.queue_time_us":427650}
I20260709 12:17:38.287041  1000 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.313880  1448 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:38.342530 31965 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.566s	user 0.016s	sys 0.000s
I20260709 12:17:38.370077 32045 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:38.371054 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 151) took 1732 ms. Trace:
I20260709 12:17:38.371574 32045 rpcz_store.cc:276] 0709 12:17:36.638915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.345828 (+1706913us) service_pool.cc:224] Handling call
0709 12:17:38.369519 (+ 23691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.370728 (+  1209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.370982 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:38.355656  1475 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:17:38.361831 32038 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:38.360783 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:38.407033 32045 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 3. Ops: []
I20260709 12:17:38.408548 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1367) took 1081 ms. Trace:
I20260709 12:17:38.408888 32045 rpcz_store.cc:276] 0709 12:17:37.327009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.372639 (+1045630us) service_pool.cc:224] Handling call
0709 12:17:38.405510 (+ 32871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.408279 (+  2769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.408489 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:38.352941  1000 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:38.403043  1474 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 12:17:38.367846  1448 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:38.368788 31965 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:38.359543  1407 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:38.412688  1481 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:17:38.432988 32045 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:38.404441   869 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:38.428339  1472 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:17:38.444151 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1368) took 1108 ms. Trace:
I20260709 12:17:38.446105 32045 rpcz_store.cc:276] 0709 12:17:37.335341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.409760 (+1074419us) service_pool.cc:224] Handling call
0709 12:17:38.430196 (+ 20436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.439974 (+  9778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.444079 (+  4105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:38.450603 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 337799us
I20260709 12:17:38.451742 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.432065 32656 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:38.408548  1419 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.357986  1251 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:38.442274   999 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:38.477917  1482 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:17:38.479386  1291 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.456519  1336 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.474917 32045 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:38.427668  1300 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.485829 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1370) took 1149 ms. Trace:
I20260709 12:17:38.488605 32045 rpcz_store.cc:276] 0709 12:17:37.336421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.447142 (+1110721us) service_pool.cc:224] Handling call
0709 12:17:38.474707 (+ 27565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.485278 (+ 10571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.485760 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:38.432829  1448 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:38.490648 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 331512us
I20260709 12:17:38.492347 32656 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:38.493556 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:38.494086  1402 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:38.496295  1402 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:38.496876  1402 consensus_queue.cc:1243] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:38.499339 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:38.500629 32115 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:38.484360 32652 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.532694  1461 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:38.544726 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:38.549588 31967 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
W20260709 12:17:38.544046 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 90885.8us
I20260709 12:17:38.545394 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:38.545956  1490 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:38.547384 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.547729 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61325us
I20260709 12:17:38.484360 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 140) took 1497 ms. Trace:
I20260709 12:17:38.486112  1437 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:38.490345  1272 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:38.559545 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:38.559963 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:38.548919 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 148) took 1951 ms. Trace:
I20260709 12:17:38.553429 31972 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:38.503523  1450 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.490186 32038 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:38.490110   869 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.501078 32115 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:38.560986 32119 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:38.551853  1275 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.588644 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:38.589540 32119 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:38.577271 32045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:38.573408 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:38.551017  1313 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.610653  1313 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.613082  1313 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:38.554957 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:38.550951  1402 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.575423 32047 rpcz_store.cc:276] 0709 12:17:36.597663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.869526 (+1271863us) service_pool.cc:224] Handling call
0709 12:17:37.869898 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.872845 (+  2947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.872893 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.876785 (+  3892us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.942971 (+ 66186us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:37.957992 (+ 15021us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.973034 (+ 15042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.973141 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.548705 (+575564us) raft_consensus.cc:1685] finished
0709 12:17:38.548727 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.548861 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13951,"prepare.queue_time_us":14486,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":17982,"raft.queue_time_us":20523,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":33848,"thread_start_us":65901,"threads_started":1,"wal-append.queue_time_us":170615}
I20260709 12:17:38.613991  1402 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.574584  1437 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.581816   869 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.616181  1402 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:38.574637  1490 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.585439  1496 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:38.591590 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:38.621328 32120 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
W20260709 12:17:38.564394 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.592021 32045 raft_consensus.cc:606] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.622498 32045 raft_consensus.cc:614] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:38.592669 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:38.623306 32047 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:38.624243 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.558434 32112 rpcz_store.cc:276] 0709 12:17:36.986599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.011711 (+ 25112us) service_pool.cc:224] Handling call
0709 12:17:37.620100 (+608389us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:37.620378 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.691147 (+ 70769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.695204 (+  4057us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.778424 (+ 83220us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.888982 (+110558us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:37.889040 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.895001 (+  5961us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.453550 (+558549us) raft_consensus.cc:1685] finished
0709 12:17:38.453582 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.482448 (+ 28866us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25151,"raft.queue_time_us":69499,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":1714,"spinlock_wait_cycles":7372544,"thread_start_us":110089,"threads_started":1,"wal-append.queue_time_us":157275}
I20260709 12:17:38.562168 32042 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:38.597357  1336 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:38.600847  1356 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.000s
I20260709 12:17:38.563374  1249 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.586937  1272 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.579385 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:38.547792  1480 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
W20260709 12:17:38.637868 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.626827  1337 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:38.621831  1484 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:17:38.622561 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1369) took 1286 ms. Trace:
I20260709 12:17:38.623773  1000 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.617100   869 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:38.656181  1437 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:38.657492  1490 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.659186 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:38.678041 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.628865  1313 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:17:38.592790  1275 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:38.689498 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.627679  1497 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.693039 32120 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:17:38.678086  1454 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.005s
I20260709 12:17:38.639586  1356 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.660457  1274 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.660382 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1373) took 1310 ms. Trace:
W20260709 12:17:38.703526 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 148859us
I20260709 12:17:38.687336  1407 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.706059 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.701875 32045 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:38.707716 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.709465 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:38.696681  1437 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.713073 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:38.713110 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.699307  1497 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:17:38.715638 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 142330us
I20260709 12:17:38.717540 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:38.660243 32038 rpcz_store.cc:276] 0709 12:17:37.335906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.438219 (+1102313us) service_pool.cc:224] Handling call
0709 12:17:38.462836 (+ 24617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.577872 (+115036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.618838 (+ 40966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:38.710487  1337 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:38.702450  1454 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.726817 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1393) took 2464 ms. Trace:
I20260709 12:17:38.724434 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:38.714260  1381 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.712308  1336 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.729204 32044 rpcz_store.cc:276] 0709 12:17:36.262343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.257192 (+994849us) service_pool.cc:224] Handling call
0709 12:17:37.257482 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.534424 (+276942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.726596 (+1192172us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400056ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:38.726760 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493793664}
I20260709 12:17:38.733067 32045 pending_rounds.cc:85] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 1
I20260709 12:17:38.733518 32045 pending_rounds.cc:107] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:17:38.733829 32045 raft_consensus.cc:2889] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:38.708827  1490 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:17:38.736196 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:38.721962 32112 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:38.728245  1337 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:38.707692 32042 rpcz_store.cc:276] 0709 12:17:37.338707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.458142 (+1119435us) service_pool.cc:224] Handling call
0709 12:17:38.489964 (+ 31822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.632899 (+142935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.649544 (+ 16645us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:38.739316  1437 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.748595  1272 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:38.743266  1488 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:17:38.752173  1337 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.737965 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:38.717015  1497 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.709295  1274 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.723315  1300 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:38.722609 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1465) took 1305 ms. Trace:
I20260709 12:17:38.757694 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:54746 (request call id 12) took 2107 ms. Trace:
I20260709 12:17:38.736109  1506 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.740074  1498 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:38.742703 32654 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:38.746953  1289 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.921s	user 0.007s	sys 0.012s
W20260709 12:17:38.749733 32041 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.799s	user 0.005s	sys 0.005s
I20260709 12:17:38.760696  1437 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.760725 32115 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.744069  1356 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.794844  1499 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:17:38.802531 32115 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.804417  1337 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.804989  1448 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.805789 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:38.764019  1500 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:17:38.806437 31972 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:38.781977 31970 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:38.782887 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:38.807876 31968 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:38.808030 32120 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:38.751022  1490 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:38.808629  1490 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:38.809068 32652 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:17:38.809207 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:38.809469 32652 pending_rounds.cc:85] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 3
W20260709 12:17:38.810078  1433 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.908s	user 0.009s	sys 0.004s
I20260709 12:17:38.809715  1490 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.811429  1490 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.767426 32113 rpcz_store.cc:276] 0709 12:17:37.416989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.862769 (+445780us) service_pool.cc:224] Handling call
0709 12:17:37.863121 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.863470 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.863511 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.864154 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 12:17:38.053692 (+189538us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:38.058347 (+  4655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.086301 (+ 27954us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.712970 (+626669us) raft_consensus.cc:1685] finished
0709 12:17:38.714198 (+  1228us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.721008 (+  6810us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4481,"raft.queue_time_us":5922,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5153,"thread_start_us":91333,"threads_started":1,"wal-append.queue_time_us":358321}
I20260709 12:17:38.766976  1300 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:38.813822  1300 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:38.749922  1334 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 12:17:38.814677  1334 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.815426  1454 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.816514 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:38.817153 32656 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:38.817230  1334 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:38.760314  1272 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.818320  1272 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.821077  1272 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.821877 32121 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:38.797387 32039 raft_consensus.cc:2410] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:38.798975  1507 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:38.824038  1507 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:38.774844  1331 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.001s
I20260709 12:17:38.826646  1331 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.829298  1331 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:38.829428  1300 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:38.833994 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:38.796839 32041 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:38.774542 32038 rpcz_store.cc:276] 0709 12:17:36.650533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.732029 (+2081496us) service_pool.cc:224] Handling call
0709 12:17:38.757585 (+ 25556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:38.838191 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:38.839190 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:38.836742 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:38.826067  1381 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.830444 31964 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:17:38.745030  1275 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.798069 31963 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:38.843015 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:38.834010 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:38.844648 31968 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:38.845368 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:38.800460  1274 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:38.835122 31970 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:38.846096  1274 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:38.846727  1274 consensus_queue.cc:1243] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:38.818095 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:38.810106  1337 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:38.809711 32652 pending_rounds.cc:107] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 12:17:38.848196 32652 raft_consensus.cc:2917] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 12:17:38.848649 32652 raft_consensus.cc:2889] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:38.848994 32656 pending_rounds.cc:85] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 4
I20260709 12:17:38.849282 32656 pending_rounds.cc:107] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:38.846632 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:38.849536 32656 raft_consensus.cc:2889] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:38.849821 31970 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:38.814155  1034 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.576s	user 0.003s	sys 0.005s
I20260709 12:17:38.799032 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:38.826287  1498 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:38.853238 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:38.856555 31971 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:38.831334  1272 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:38.855401 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:39.208470 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1396) took 2930 ms. Trace:
I20260709 12:17:38.867763 31967 raft_consensus.cc:2410] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 12:17:39.124028 32085 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.752s (SENT) [suppressed 8 similar messages]
I20260709 12:17:39.221480 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 152) took 2571 ms. Trace:
I20260709 12:17:39.220134 32046 rpcz_store.cc:276] 0709 12:17:36.277606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.310047 (+1032441us) service_pool.cc:224] Handling call
0709 12:17:37.310366 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.847595 (+537229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.208145 (+1360550us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54001a59d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:39.208381 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077616000}
I20260709 12:17:38.890949 31970 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:39.221889 32040 rpcz_store.cc:276] 0709 12:17:36.649807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.346024 (+1696217us) service_pool.cc:224] Handling call
0709 12:17:39.220764 (+874740us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:39.220950 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.221138 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:39.221165 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:39.221205 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:39.221237 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.221308 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.221426 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16493952}
W20260709 12:17:39.024755 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.012042 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:39.233517 32626 leader_election.cc:400] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:39.224757 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:38.877465 31971 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:39.234839 32626 leader_election.cc:403] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:39.055265 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:38.977243 32119 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:38.853737  1437 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:38.895288 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53767.7us
W20260709 12:17:38.948493 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.971990 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:39.239912 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 184762us
I20260709 12:17:39.243739 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:38.978134 32118 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:17:39.244920 32118 pending_rounds.cc:85] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 3
I20260709 12:17:39.245146 32118 pending_rounds.cc:107] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 12:17:39.245402 32118 raft_consensus.cc:2917] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 12:17:39.059923 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 372) took 1127 ms. Trace:
W20260709 12:17:38.960497 32660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 365) took 1371 ms (client timeout 1777 ms). Trace:
I20260709 12:17:39.246294 32659 rpcz_store.cc:276] 0709 12:17:37.932553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.983525 (+ 50972us) service_pool.cc:224] Handling call
0709 12:17:39.059747 (+1076222us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400543654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:39.059858 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274562304}
W20260709 12:17:39.246447 32660 rpcz_store.cc:269] 0709 12:17:37.589508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.591223 (+  1715us) service_pool.cc:224] Handling call
0709 12:17:38.960303 (+1369080us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54004e4b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:38.960429 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773047552}
I20260709 12:17:39.245779 32118 raft_consensus.cc:2889] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:38.999521 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 373) took 1064 ms. Trace:
I20260709 12:17:39.247759 32657 rpcz_store.cc:276] 0709 12:17:37.934971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.018853 (+ 83882us) service_pool.cc:224] Handling call
0709 12:17:38.019203 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.346964 (+327761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.346994 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:38.347034 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:38.347061 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.999253 (+652192us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54003bae54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:38.999305 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.999449 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753251584}
I20260709 12:17:39.237931 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:39.165050  1476 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.886s	user 0.001s	sys 0.004s
W20260709 12:17:39.234157 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 222189us
I20260709 12:17:38.958576  1275 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.281864 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:39.253484 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.059870  1483 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:39.248605 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:38.996819 32115 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:38.856484  1507 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:38.856761  1034 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:38.976289  1470 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.958s	user 0.000s	sys 0.015s
I20260709 12:17:39.161523  1220 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:38.973850  1498 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.305987  1220 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.308768  1408 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:39.310961  1516 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.353s	user 0.003s	sys 0.000s
I20260709 12:17:39.281222  1432 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:17:39.314682  1432 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:39.286149  1502 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEARNER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:39.317463 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:39.318217 32657 raft_consensus.cc:2368] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:39.138947  1313 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:39.286298  1491 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:38.869913 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:38.881081  1471 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.950s	user 0.006s	sys 0.000s
I20260709 12:17:39.290608 32120 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:39.319118  1313 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.324846 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.168705 31966 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:39.012336   964 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.292636 32118 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:39.058729 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:38.978633 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:39.293880 32653 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:39.012534  1358 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.880143  1329 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.880733  1300 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.011689 32112 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:39.296865 31968 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:39.325647 31965 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:39.239060  1245 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:38.997965 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 363) took 1537 ms. Trace:
W20260709 12:17:39.297328 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 173342us
I20260709 12:17:38.853935  1497 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.281554  1294 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:39.273991  1437 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.249854  1433 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:39.280309  1286 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:39.286873  1270 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.857802  1419 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:39.304203  1372 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:39.280643  1496 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:17:39.288511 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.232728  1272 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.300551  1345 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.323333  1512 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:39.292297  1483 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:39.326738 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.334611   964 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.304970  1490 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:39.336503 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:39.340858 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:39.294507  1275 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.341668 32117 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:39.351689 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.342738 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:39.381558 32657 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:39.354661 32659 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.341826  1358 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.358956  1294 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:39.359277 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 458) took 1462 ms. Trace:
I20260709 12:17:39.394706 32653 rpcz_store.cc:276] 0709 12:17:37.896605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.901836 (+  5231us) service_pool.cc:224] Handling call
0709 12:17:39.293434 (+1391598us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:39.293674 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.339668 (+ 45994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.359201 (+ 19533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22990208}
I20260709 12:17:39.332357  1506 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:39.395231  1294 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.396071  1506 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.359452  1286 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:39.398533  1286 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.399847  1275 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.367136 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1295) took 1679 ms. Trace:
I20260709 12:17:39.369134  1345 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:17:39.403053  1519 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:17:39.404800  1345 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.369808 31966 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:39.402515 31969 rpcz_store.cc:276] 0709 12:17:37.687952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.770189 (+ 82237us) service_pool.cc:224] Handling call
0709 12:17:37.770724 (+   535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:37.774994 (+  4270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.775048 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:37.775897 (+   849us) log.cc:844] Serialized 52 byte log entry
0709 12:17:38.089327 (+313430us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:38.096781 (+  7454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.114375 (+ 17594us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:39.164316 (+1049941us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:39.355025 (+190709us) raft_consensus.cc:1685] finished
0709 12:17:39.355655 (+   630us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.367063 (+ 11408us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3511,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":189,"spinlock_wait_cycles":16527232,"thread_start_us":235583,"threads_started":1,"wal-append.queue_time_us":472693}
I20260709 12:17:39.368260  1512 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:39.406836 31966 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:39.407200  1512 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.408591  1512 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.249581  1251 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:17:39.355118  1245 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.367522  1249 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:39.357633  1437 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.359302  1330 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.382226 32118 raft_consensus.cc:606] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:39.363202 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:39.378825 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 374) took 1431 ms. Trace:
I20260709 12:17:39.374289  1490 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.439323  1490 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.387187  1429 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:39.442168  1429 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:39.442449  1477 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.443010  1477 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.444479 32658 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:39.431597 32654 rpcz_store.cc:276] 0709 12:17:37.942262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.113101 (+170839us) service_pool.cc:224] Handling call
0709 12:17:38.189542 (+ 76441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.794653 (+605111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.363874 (+569221us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54004c27d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:39.373534 (+  9660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257962752}
I20260709 12:17:39.408624  1506 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.443490  1429 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.446707  1249 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.447814  1429 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.448886  1249 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.431712  1330 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.389528  1358 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.410722 32085 leader_election.cc:400] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:39.450925 32085 leader_election.cc:403] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:17:39.451583 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 66988.9us
I20260709 12:17:39.424292  1245 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.411813 32118 raft_consensus.cc:614] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:39.454762  1506 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.455451  1497 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.456344  1313 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.456951 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:17:39.455499   964 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.432539 31867 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 4, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 4 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.462935 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 366) took 1681 ms. Trace:
I20260709 12:17:39.464026 32658 rpcz_store.cc:276] 0709 12:17:37.781936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.782093 (+   157us) service_pool.cc:224] Handling call
0709 12:17:37.782431 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.444940 (+1662509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.462616 (+ 17676us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540053edd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:39.462854 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3821214592}
I20260709 12:17:39.364578  1433 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:39.351178 32655 rpcz_store.cc:276] 0709 12:17:37.460748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.465229 (+  4481us) service_pool.cc:224] Handling call
0709 12:17:38.018681 (+553452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.018906 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:38.019240 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 12:17:38.412205 (+392965us) log.cc:844] Serialized 14 byte log entry
0709 12:17:38.625266 (+213061us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:38.625333 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:38.625404 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.625498 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.997696 (+372198us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540066e9d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:38.997850 (+   154us) inbound_call.cc:177] Queueing success response
0709 12:17:39.163687 (+165837us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b180006cdc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:17:39.313718 (+150031us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b180006cdc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
Metrics: {"prepare.queue_time_us":990,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":141496,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":4000,"spinlock_wait_cycles":1179546624,"thread_start_us":572101,"threads_started":2,"wal-append.queue_time_us":528937}
I20260709 12:17:39.427109 32659 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:39.428798  1522 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:39.431305 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:39.436708 31869 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 241) took 10512 ms. Trace:
I20260709 12:17:39.438833 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 459) took 1541 ms. Trace:
W20260709 12:17:39.507144 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 75932.3us
I20260709 12:17:39.486006 32043 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:39.460815 31867 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:39.489224 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 207403us
I20260709 12:17:39.365927  1419 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.495043  1034 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.496048 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:39.485109 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:39.499274 32659 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 4
I20260709 12:17:39.522430 32659 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 12:17:39.501670 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:39.502345 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:39.524240 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.522801 32659 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 12:17:39.513736 32120 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:39.527769 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.523499 32658 raft_consensus.cc:2376] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2. Re-sending same reply.
W20260709 12:17:39.531138 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.526374 32659 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:39.514806 31867 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.517449   964 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:39.535038 31867 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 3, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.531864 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:39.501016 31869 rpcz_store.cc:276] 0709 12:17:28.924242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.924404 (+   162us) service_pool.cc:224] Handling call
0709 12:17:39.436604 (+10512200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:35.700890 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:35.852008 (+151118us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:35.852055 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:17:35.852076 (+    21us) tablet.cc:662] Decoding operations
0709 12:17:35.853272 (+  1196us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:35.853368 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:35.853407 (+    39us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:17:35.905087 (+ 51680us) tablet.cc:701] Row locks acquired
0709 12:17:35.905119 (+    32us) write_op.cc:260] PREPARE: finished
0709 12:17:35.905349 (+   230us) write_op.cc:270] Start()
0709 12:17:35.905515 (+   166us) write_op.cc:276] Timestamp: P: 1783599455905257 usec, L: 0
0709 12:17:35.905546 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:17:35.922883 (+ 17337us) log.cc:844] Serialized 3965 byte log entry
0709 12:17:36.646596 (+723713us) op_driver.cc:464] REPLICATION: finished
0709 12:17:37.149245 (+502649us) write_op.cc:301] APPLY: starting
0709 12:17:37.154119 (+  4874us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:37.182098 (+ 27979us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:37.227873 (+ 45775us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:37.340480 (+112607us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:37.340698 (+   218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:17:37.340745 (+    47us) write_op.cc:312] APPLY: finished
0709 12:17:37.341017 (+   272us) log.cc:844] Serialized 77 byte log entry
0709 12:17:37.341184 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:37.341365 (+   181us) write_op.cc:454] Released schema lock
0709 12:17:37.341422 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:37.341803 (+   381us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:37.341831 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5351,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":453293,"apply.run_cpu_time_us":4332,"apply.run_wall_time_us":228329,"num_ops":10,"prepare.queue_time_us":566311,"prepare.run_cpu_time_us":21713,"prepare.run_wall_time_us":477847,"raft.queue_time_us":368341,"raft.run_cpu_time_us":30470,"raft.run_wall_time_us":273948,"replication_time_us":740896,"thread_start_us":987079,"threads_started":4,"wal-append.queue_time_us":288393,"wal-append.run_cpu_time_us":69659,"wal-append.run_wall_time_us":2129506}]]}
I20260709 12:17:39.537611 32040 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:39.533722 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.536248 32119 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:39.539306 32120 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:39.539350 31867 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 4, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 4 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:39.521124 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 235867us
I20260709 12:17:39.541036 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:39.541299 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.498562  1313 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:17:39.518944 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:39.544109 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:39.544086 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:39.553383 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.552220 31867 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:39.562873 32044 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 12:17:39.559103 31867 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:39.519383  1251 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:17:39.565979 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.565950 31867 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 5, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.567575 32047 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:39.569209 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:39.570122 32047 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:39.568101 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:39.574870 31867 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 4, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:39.576915 31867 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 3, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.578863 31867 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 5, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.581846 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:39.582595 32040 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.576046 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:39.582824 31867 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:39.584017 32046 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:39.585335 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:39.586005 32046 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.588866 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:39.593760 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:39.594518 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.506515 32651 rpcz_store.cc:276] 0709 12:17:37.896925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.901410 (+  4485us) service_pool.cc:224] Handling call
0709 12:17:39.357913 (+1456503us) spinlock_profiling.cc:243] Waited 9.95 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:39.438767 (+ 80854us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22886784}
I20260709 12:17:39.554958  1461 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.598096 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:39.608036 32044 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.555008  1345 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:39.522779  1524 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:17:39.603128 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:39.602993 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:39.603909 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:39.616600 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80217.9us
I20260709 12:17:39.573827  1272 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.619889 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.624584 32040 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:39.610143 32046 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:39.584453 31867 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 3, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), now has a pending config: VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2), VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:39.542892  1358 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:39.556566  1407 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:17:39.600885  1497 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:39.626940 31867 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:39.633956 32044 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:39.627300 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:39.627350 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:39.516891  1483 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.610205  1345 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.646685 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:39.677363  1497 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:39.613467  1330 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.613416 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:39.616201  1524 consensus_queue.cc:1243] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:39.645257  1289 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:39.615685 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:39.601017  1082 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 12:17:39.754611  1532 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.071s	user 0.002s	sys 0.004s
I20260709 12:17:39.675374 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.755822 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.756402 31965 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:39.757149 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.621045  1408 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.758733  1220 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.758687  1408 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.759341  1220 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.760864  1408 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.761078  1506 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:39.761271  1220 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.761785  1408 raft_consensus.cc:2764] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:39.626295   964 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.681447  1419 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.618603 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.758173 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.762847 31969 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:39.763473 31965 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:39.643778 31966 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:39.596254  1433 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.764767   964 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.734153 31867 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 4, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 4 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:39.766763 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 139482us
I20260709 12:17:39.767926 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:39.767846 31867 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 3, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.769081  1407 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:39.683319  1272 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.615738  1461 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:39.736343 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 192245us
I20260709 12:17:39.732591  1530 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.694226  1539 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:39.738847 31918 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3836f9b5cd36488da56d4540dc9d80ec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:17:39.738914  1529 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:39.692672  1286 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.677289 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:39.609922  1498 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:39.764341 31969 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:39.683079 32655 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:39.769985 31867 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:39.770877 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 132) took 1903 ms. Trace:
I20260709 12:17:39.793210 31867 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:39.793284 31964 rpcz_store.cc:276] 0709 12:17:37.867312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.884315 (+ 17003us) service_pool.cc:224] Handling call
0709 12:17:37.906293 (+ 21978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:38.842535 (+936242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.770642 (+928107us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b54001e7dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:39.770808 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124516736}
W20260709 12:17:39.737409 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.771541 32046 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:39.775511 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.742566  1330 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:39.796060  1330 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.736928  1345 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.798242  1330 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:39.784584 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 184121us
W20260709 12:17:39.786952 31918 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:17:39.757441  1490 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.806325 32116 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.804450 31867 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 3, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 3 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.732688  1313 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.773144 32042 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.770784 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:39.812820 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 135618us
I20260709 12:17:39.772250  1289 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.787354  1530 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.818540  1358 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:39.820389  1330 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:39.820705  1289 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.821135  1330 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:39.789742  1539 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.822968  1286 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.823251  1539 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.823850 32113 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:39.824681  1286 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.825227  1286 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.826514  1313 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:39.794222  1498 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.829929  1498 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.830183  1286 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.831187  1477 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.831838  1497 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.803575  1542 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 12:17:39.834574  1534 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:17:39.836685  1313 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:39.783128  1272 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.839020  1272 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.839759 32038 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:39.804669 31918 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:17:39.842285  1492 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:39.842823  1492 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:39.850347  1245 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:39.852686  1245 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.787572  1529 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:39.853564 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.813496 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:39.864343 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48634.1us
I20260709 12:17:39.856439 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.872089 32118 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:39.873083 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.873733 32118 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:39.875641 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62268.5us
I20260709 12:17:39.875783 32659 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:39.932384 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:39.931324 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:39.823083  1535 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:39.853574  1529 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:39.941386 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 128112us
I20260709 12:17:39.946382 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:39.931638 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56459.4us
W20260709 12:17:39.886592 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.929994  1490 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.946632 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:39.942410 32118 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.805289   964 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:39.820421  1530 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:39.834067  1437 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:39.843886  1492 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.000486 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1636) took 1027 ms. Trace:
I20260709 12:17:39.967306 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:40.002637 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:40.017524 31963 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:40.017652 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50566.1us
I20260709 12:17:39.929775  1498 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.016883 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70338.5us
I20260709 12:17:39.939352  1245 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:39.873442  1407 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:40.031442 32039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 160) took 2174 ms (client timeout 2203 ms). Trace:
I20260709 12:17:40.016930 32116 rpcz_store.cc:276] 0709 12:17:38.973127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.341574 (+368447us) service_pool.cc:224] Handling call
0709 12:17:39.355185 (+ 13611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.806738 (+451553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.000207 (+193469us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54003e4ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:40.000408 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037014400}
I20260709 12:17:40.033702 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:40.018656 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:39.974026 31936 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:39.882284 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:40.050376 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.876657 32114 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:17:40.092741 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73774.8us
I20260709 12:17:40.092820 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:39.870743  1433 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:39.973430  1512 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.564s	user 0.016s	sys 0.000s
W20260709 12:17:40.019677  1503 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.282s	user 0.003s	sys 0.000s
I20260709 12:17:40.109035 32120 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:40.124518 32118 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:40.125098 31963 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
W20260709 12:17:40.091756 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 209581us
I20260709 12:17:40.125550 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:40.127254 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.129508 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:39.983211  1525 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.013285  1499 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.106189 32039 rpcz_store.cc:269] 0709 12:17:37.856980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.939385 (+ 82405us) service_pool.cc:224] Handling call
0709 12:17:40.019779 (+2080394us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54000334d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.031359 (+ 11580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930278272}
W20260709 12:17:40.096740 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 122828us
I20260709 12:17:40.131083  1270 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.994827 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:40.090845  1498 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.106982  1220 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:39.972404  1289 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.145591 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.107331  1490 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:39.983783  1334 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.162s
I20260709 12:17:40.107496  1291 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.119848  1536 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:17:40.136677  1275 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:17:40.137712  1492 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.032027  1524 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:40.123230  1530 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.108461   964 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.968329 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.137512  1294 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.050763  1506 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.131s
I20260709 12:17:40.030630  1432 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.146665 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.172266 31964 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:17:39.944078  1508 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.199s
I20260709 12:17:40.172071 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.159703 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.175566 32046 raft_consensus.cc:2368] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
W20260709 12:17:40.163255 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 129621us
W20260709 12:17:40.155903 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.176409 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.176663 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:40.169423 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:40.177484 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.138921  1525 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:40.178889 32046 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:40.179733 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.180858 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.181317 32046 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:40.182042 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.146247 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:39.986084  1249 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.187s
I20260709 12:17:40.195681 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.202370 32658 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:17:40.150782  1547 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:17:40.146590  1270 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.156968 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.143090  1433 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.150135  1286 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 12:17:40.159942 32042 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:40.149039  1220 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:40.301599 32084 leader_election.cc:400] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:40.202020 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.355955 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.186841 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:40.365206 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63869.1us
W20260709 12:17:40.365916 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 179192us
I20260709 12:17:40.366019 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:40.149507  1490 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.148553  1498 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.156214  1524 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:40.158087 32038 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:40.163017  1358 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.218s
I20260709 12:17:40.219586  1408 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.161698 32115 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:17:40.159482  1437 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.164835  1512 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:40.299192 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.298209  1429 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:40.178766  1506 consensus_queue.cc:1243] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:40.369175  1508 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:40.174031 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:40.162580  1461 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:40.370693 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:40.395723 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 222143us
I20260709 12:17:40.396523 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:40.199359  1525 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.173624  1477 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.171375  1289 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.192s
I20260709 12:17:40.220618  1419 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.399020  1506 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.401017  1419 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.401361  1506 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.403235  1419 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.195963 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.313290  1432 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.404606  1432 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.313629 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.406473  1508 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:40.406644 32653 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:40.173467  1407 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.199s
I20260709 12:17:40.407414  1508 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.370781  1220 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.409708  1220 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:40.412410  1345 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.614s	user 0.006s	sys 0.003s
I20260709 12:17:40.412901  1345 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.414083  1432 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.365217 32039 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.381981  1497 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.418651  1497 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.420783  1497 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.421336  1294 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.422032  1551 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:40.422501  1551 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:40.422906  1551 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.168780  1499 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.387195  1429 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.369964  1524 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.392709 32045 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:40.398029 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:40.274781 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1622) took 2745 ms. Trace:
I20260709 12:17:40.414186 32047 raft_consensus.cc:2368] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5.
I20260709 12:17:40.195698  1543 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:40.403293  1506 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.150596  1291 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.425137 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:40.422039 32039 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:40.374543  1498 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.372309  1490 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.427580 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.427873 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:40.424818 32121 rpcz_store.cc:276] 0709 12:17:37.462912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.942879 (+479967us) service_pool.cc:224] Handling call
0709 12:17:37.956758 (+ 13879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.822388 (+865630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.207673 (+1385285us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540013a454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:40.207835 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1988996352}
W20260709 12:17:40.425096  1547 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:17:40.440798 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.442828 32039 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:40.396936  1437 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:40.400554  1408 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.439345 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.433357  1432 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.454675 32659 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:40.383912  1529 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.431222  1294 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:40.427627 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1635) took 1454 ms. Trace:
I20260709 12:17:40.466352 32115 rpcz_store.cc:276] 0709 12:17:38.972814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.337058 (+364244us) service_pool.cc:224] Handling call
0709 12:17:40.161085 (+824027us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:40.161305 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.380617 (+219312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.425580 (+ 44963us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5109504}
I20260709 12:17:40.467944  1274 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:40.468525  1274 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:40.470750  1034 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:40.471297  1034 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:40.472378  1034 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.474983 32654 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:40.434348  1543 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:40.465394  1529 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:40.461364  1294 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.441015  1506 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:40.426550  1477 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.469610  1274 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.489825  1506 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.486394  1294 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.496348 32118 pending_rounds.cc:85] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 1
I20260709 12:17:40.502084 32118 pending_rounds.cc:107] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:40.502499 32118 raft_consensus.cc:2889] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:40.452816  1437 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:40.498128  1533 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.816s	user 0.002s	sys 0.003s
I20260709 12:17:40.511307  1506 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:40.512293 32119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1640) took 1538 ms (client timeout 1596 ms). Trace:
W20260709 12:17:40.512859 32119 rpcz_store.cc:269] 0709 12:17:38.974156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.237687 (+263531us) service_pool.cc:224] Handling call
0709 12:17:40.512061 (+1274374us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.512210 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589804416}
I20260709 12:17:40.514961 32119 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:40.515410 32119 pending_rounds.cc:85] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 1
I20260709 12:17:40.515681 32119 pending_rounds.cc:107] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:40.515972 32119 raft_consensus.cc:2889] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:40.524380  1329 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:40.525278  1291 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.526037  1543 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.521888  1429 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.535537  1437 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.521425  1274 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.522651  1270 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.520996  1345 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:40.542940  1499 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:40.556710  1483 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.551915  1433 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:40.558212  1294 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.560979  1294 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.553907  1512 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:40.608616 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 180519us
I20260709 12:17:40.611928 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:40.612546 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1613) took 1774 ms (client timeout 1801 ms). Trace:
W20260709 12:17:40.613205 31967 rpcz_store.cc:269] 0709 12:17:38.837938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.838999 (+  1061us) service_pool.cc:224] Handling call
0709 12:17:40.610891 (+1771892us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b54000b6b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.612251 (+  1360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41628928}
I20260709 12:17:40.611965 31971 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:40.616638 31968 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1611) took 1779 ms (client timeout 2364 ms). Trace:
W20260709 12:17:40.617179 31968 rpcz_store.cc:269] 0709 12:17:38.837507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.845163 (+  7656us) service_pool.cc:224] Handling call
0709 12:17:40.616177 (+1771014us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54001e91d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.616440 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036641024}
W20260709 12:17:40.620191 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.621385 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:40.623128 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1310) took 1342 ms. Trace:
I20260709 12:17:40.444289 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.625810 32039 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.398983 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.627771 32042 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:40.629134 32045 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:40.629485 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:40.624032 32043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1384) took 2324 ms (client timeout 1770 ms). Trace:
W20260709 12:17:40.630678 32043 rpcz_store.cc:269] 0709 12:17:38.115233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.157463 (+ 42230us) service_pool.cc:224] Handling call
0709 12:17:40.439273 (+2281810us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001950d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.439412 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2964679040}
I20260709 12:17:40.631719 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.631901 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.632475 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.633198 32042 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:40.633602 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.633749 31971 rpcz_store.cc:276] 0709 12:17:39.281051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.372351 (+ 91300us) service_pool.cc:224] Handling call
0709 12:17:40.611277 (+1238926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.615985 (+  4708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.622745 (+  6760us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000b6b50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.622978 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821189888}
I20260709 12:17:40.638185 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1435) took 1058 ms. Trace:
I20260709 12:17:40.633440 32043 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:40.638748 32040 rpcz_store.cc:276] 0709 12:17:39.579663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.787553 (+207890us) service_pool.cc:224] Handling call
0709 12:17:39.803134 (+ 15581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.835522 (+ 32388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:39.835569 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:39.835627 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:39.835678 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.637601 (+801923us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54002f8d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:40.637686 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.637930 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73876352}
I20260709 12:17:40.640789 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.639559 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.641811 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:40.642905 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.643826 32043 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:40.644865 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.644802 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.645812 32038 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:40.647104 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.647965 32038 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:40.649953 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.652096 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.653353 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1625) took 1059 ms. Trace:
I20260709 12:17:40.653826 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:40.636024 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.655460  1562 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:40.627498 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.655534 32041 raft_consensus.cc:2376] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6. Re-sending same reply.
I20260709 12:17:40.656772 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1422) took 1095 ms. Trace:
I20260709 12:17:40.652130 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.657313 32041 rpcz_store.cc:276] 0709 12:17:39.561064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.566420 (+  5356us) service_pool.cc:224] Handling call
0709 12:17:40.635657 (+1069237us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:40.656668 (+ 21011us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6395648}
I20260709 12:17:40.641057 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.659643 32038 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:40.653790 31970 rpcz_store.cc:276] 0709 12:17:39.594360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.651376 (+1057016us) service_pool.cc:224] Handling call
0709 12:17:40.653256 (+  1880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.659982 32041 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:40.657967 31971 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:40.659881 32040 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:40.663059 32041 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:40.663810 32040 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:40.665156 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.665712 32040 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:40.667127 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:40.660455  1562 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.667886  1562 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.661448 32045 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:40.665812 32041 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:40.670862 32041 pending_rounds.cc:85] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 4
I20260709 12:17:40.671113 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.671274 32041 pending_rounds.cc:107] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:17:40.672027 32041 raft_consensus.cc:2889] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:40.642748 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:40.669723 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.670169 32045 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:40.668021 31969 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:40.670251  1565 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.677050  1565 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.677597  1565 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.700647 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.701336 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1626) took 3063 ms. Trace:
I20260709 12:17:40.701792 32117 rpcz_store.cc:276] 0709 12:17:37.637951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.947794 (+309843us) service_pool.cc:224] Handling call
0709 12:17:37.948073 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.384719 (+1436646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.700616 (+1315897us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000807d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:40.701041 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3204344192}
I20260709 12:17:40.703037 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.704419 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1645) took 1191 ms. Trace:
I20260709 12:17:40.704826 32120 rpcz_store.cc:276] 0709 12:17:39.513123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.565750 (+ 52627us) service_pool.cc:224] Handling call
0709 12:17:40.703998 (+1138248us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400149bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.704218 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246303104}
I20260709 12:17:40.705389 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:40.706800 32113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 154) took 1874 ms (client timeout 2058 ms). Trace:
I20260709 12:17:40.707571 32115 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:40.708122 32115 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:17:40.708475 32115 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 12:17:40.708653 31972 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:40.708958 32115 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 12:17:40.714811 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.660424 32043 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:40.679638 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.680325 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1433) took 1109 ms. Trace:
I20260709 12:17:40.682276 32112 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:40.688879 32042 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:40.690981 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:40.726612 31963 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.710124 32115 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:40.727397 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:40.697484 31971 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:40.723804 32047 rpcz_store.cc:276] 0709 12:17:39.570520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.588581 (+ 18061us) service_pool.cc:224] Handling call
0709 12:17:40.678236 (+1089655us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54001a54d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.680143 (+  1907us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891928320}
W20260709 12:17:40.707365 32113 rpcz_store.cc:269] 0709 12:17:38.832038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.832866 (+   828us) service_pool.cc:224] Handling call
0709 12:17:40.706355 (+1873489us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540012bbd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:40.706563 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108666112}
I20260709 12:17:40.732919 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.734234 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:40.734784 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:40.735364 31966 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.737527 32046 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:40.738631 31969 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:40.747620 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:40.736687 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 340102us
I20260709 12:17:40.725179 32039 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:40.759445 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.760110 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.762722  1381 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.763510  1381 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.765560  1336 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.766804  1433 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:40.767354  1433 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.767824  1433 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.768229  1272 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.769007  1272 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:40.772074  1286 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.772414  1249 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.772640  1286 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:40.760514 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.773208  1286 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.773273  1249 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:40.774571 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.774822  1492 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.775554  1492 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.776350  1249 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.776652  1372 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:40.769228  1275 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.778460  1270 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:40.780176  1270 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:40.565052  1490 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:40.782739  1300 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:40.783649  1300 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.784274  1490 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.785660  1270 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.787999  1270 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.790719  1492 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.796754  1381 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.796907  1565 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.798911 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.800554  1565 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:40.800482 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:40.801537  1477 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:40.813525 31963 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:40.791059 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:40.799361 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:40.793030  1336 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:40.846689 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 221360us
W20260709 12:17:40.863893 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 127137us
I20260709 12:17:40.874442  1245 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:17:40.876065 32115 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:40.867398 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 50165.8us
I20260709 12:17:40.880952 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.883638 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:40.884092 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:40.884778 32657 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.888585 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.889881 32654 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:40.890210 32658 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.848048  1300 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:40.891360 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 173732us
I20260709 12:17:40.892215 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.892400 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:40.842111  1372 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.893282 31972 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:40.893844  1245 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:40.894589  1245 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:40.894963  1034 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:40.895515 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.895949  1245 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.897037 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.897727 31968 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:40.899451 32652 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:40.914870 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:40.915828 32660 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:40.916303  1249 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:40.918377  1477 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:40.927800 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 136931us
I20260709 12:17:40.933712 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:40.938808 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:40.939100  1563 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:40.939713 32113 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:40.939814  1563 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:40.944660 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51776.7us
I20260709 12:17:40.944754 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:40.940456  1563 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:40.893570  1372 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:40.965795 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.972316 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:40.991946 31972 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:17:40.996790 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 343205us
W20260709 12:17:41.012991 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47000.4us
I20260709 12:17:40.949611  1433 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:41.011847 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 49307.1us
I20260709 12:17:41.016664 32120 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
W20260709 12:17:41.009745 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 282716us
I20260709 12:17:40.993752  1249 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:41.029402 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:41.028000 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53651.4us
I20260709 12:17:41.024345  1249 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:41.021235  1336 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.042590 31968 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:41.042277  1572 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:17:41.042459  1291 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:17:41.046865 32115 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 12:17:41.053143 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:41.058115 32117 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:41.027478  1270 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:41.066635 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1489) took 1061 ms. Trace:
I20260709 12:17:41.074954  1564 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:17:41.075632  1251 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:17:41.076387  1372 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.077677 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.119040 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:41.097378 32113 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:41.102641 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.106619  1557 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.107244 32660 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:41.107959 32046 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:41.100967 32115 pending_rounds.cc:85] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 1
I20260709 12:17:41.112452 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:41.115630 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1487) took 1334 ms. Trace:
I20260709 12:17:41.091964 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:41.118850 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46903us
I20260709 12:17:41.072558 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:41.125573 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 113896us
W20260709 12:17:41.058125  1274 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.006s
I20260709 12:17:41.132325 32657 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:41.100980  1336 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.127632 32115 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 12:17:41.139376 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 108769us
I20260709 12:17:41.142923 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:41.143980  1483 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.568s	user 0.005s	sys 0.007s
I20260709 12:17:41.144819  1294 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.145347 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:41.145561  1294 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.146160 32117 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:41.220965  1551 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.797s	user 0.007s	sys 0.007s
W20260709 12:17:41.222643 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 78853.9us
I20260709 12:17:41.224551 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:41.225392  1559 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:41.225629  1548 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:41.226239  1559 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.226866  1289 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:41.227272  1289 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:41.228384  1289 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.229753  1289 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:41.151067 32116 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:41.231510 31967 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:41.231467 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.232689 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1646) took 1719 ms. Trace:
I20260709 12:17:41.233266 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.233217 32116 rpcz_store.cc:276] 0709 12:17:39.513359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.092498 (+579139us) service_pool.cc:224] Handling call
0709 12:17:41.232424 (+1139926us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540012c0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:41.232589 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363443968}
I20260709 12:17:41.127130  1220 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:41.235165 31969 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:41.234871  1220 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.236801 31969 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:17:41.154924  1249 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.239041 31969 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:41.239435  1249 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.163574  1275 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:41.242782  1275 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.164476  1565 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:41.247609  1565 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:41.167896 32042 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:17:41.169306  1524 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.736s	user 0.005s	sys 0.005s
I20260709 12:17:41.249804  1524 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.253700 31964 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:41.170337 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:41.127398 32114 rpcz_store.cc:276] 0709 12:17:39.780840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.388296 (+607456us) service_pool.cc:224] Handling call
0709 12:17:40.726774 (+338478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:40.727495 (+   721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:40.727539 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:40.728280 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 12:17:40.750100 (+ 21820us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:40.750176 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.750282 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:40.872765 (+122483us) raft_consensus.cc:1685] finished
0709 12:17:40.872794 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.115064 (+242270us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b5400138150. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:41.115384 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44034,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"spinlock_wait_cycles":777399680,"thread_start_us":20673,"threads_started":1,"wal-append.queue_time_us":21737}
I20260709 12:17:41.171622 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:41.172555 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:41.173097  1557 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.141804 32115 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:41.175714  1574 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:17:41.176087  1581 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:41.176376  1329 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:41.135768 32652 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:41.143478  1286 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.292625  1329 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:41.294433  1286 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.294037  1329 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.141054  1336 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.296073  1329 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.296938  1336 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:41.180642 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 140311us
I20260709 12:17:41.180851  1433 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.181722 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:41.299604 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 118014us
I20260709 12:17:41.300278 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.301156 32116 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:41.181978  1251 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.302446  1564 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:41.302232  1251 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.303010  1564 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.181722  1381 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.099828 32118 rpcz_store.cc:276] 0709 12:17:39.872176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.495582 (+623406us) service_pool.cc:224] Handling call
0709 12:17:40.496047 (+   465us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:40.686835 (+190788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:40.686887 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:40.687718 (+   831us) log.cc:844] Serialized 52 byte log entry
0709 12:17:40.815729 (+128011us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:40.815863 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.842642 (+ 26779us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:40.933274 (+ 90632us) raft_consensus.cc:1685] finished
0709 12:17:40.933294 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.933477 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":183222,"raft.queue_time_us":267657,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":279,"thread_start_us":126553,"threads_started":1,"wal-append.queue_time_us":242203}
I20260709 12:17:41.183698  1331 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.184684  1300 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:41.185988  1529 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:41.175778 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1651) took 1379 ms. Trace:
I20260709 12:17:41.188086 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.189752 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.311411 31963 raft_consensus.cc:2410] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:41.191699 31972 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:41.194861 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:41.197912  1525 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.769s	user 0.000s	sys 0.010s
I20260709 12:17:41.139161  1270 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.202034  1562 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:41.202607 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.319344 32117 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:41.203750  1291 raft_consensus.cc:2764] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:41.205924 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.207065  1566 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:41.134204 31966 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:41.206053  1477 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:41.221597  1551 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.267268  1274 raft_consensus.cc:2764] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:17:41.267210 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 95680.9us
I20260709 12:17:41.268734 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:41.270013 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:41.334138 31969 raft_consensus.cc:2410] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:41.271441 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.272593  1557 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.273419 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:41.274664 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.282783 31964 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:41.282974 32120 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:41.275530 32115 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:41.283514  1565 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.340983 31965 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.284192 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.284792  1275 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.238114  1220 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.285228 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:41.285405  1524 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.292021  1559 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.356408  1551 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.357169  1551 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.357578  1551 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.357952  1551 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.358188 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:41.358479 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.327044  1331 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:41.360551 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:41.334436  1525 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.362725  1275 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:41.362918 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 77820.3us
I20260709 12:17:41.363443 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:41.179219 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:41.364750 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.364952 32113 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:41.297989 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:41.304513 32658 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:41.305085 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:41.366740 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 68865.6us
I20260709 12:17:41.178786 32044 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:41.321342 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:41.368256 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 47001.9us
I20260709 12:17:41.311846 31972 pending_rounds.cc:85] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 2
I20260709 12:17:41.369165 31972 pending_rounds.cc:107] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:41.369457 31972 raft_consensus.cc:2889] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:41.369880  1492 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.333791 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.370384  1492 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.370592 31970 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.370898  1492 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.366770  1557 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.312886  1300 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.331318 31968 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:41.374456  1564 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.353895 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:41.375618 31963 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:41.375717  1300 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.355561 32656 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:41.377578  1548 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.377776  1270 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.368338 32116 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:41.309334 32121 rpcz_store.cc:276] 0709 12:17:39.796537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.455391 (+658854us) service_pool.cc:224] Handling call
0709 12:17:40.455897 (+   506us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:40.683158 (+227261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:40.683230 (+    72us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:40.684769 (+  1539us) log.cc:844] Serialized 78 byte log entry
0709 12:17:40.757723 (+ 72954us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:40.963086 (+205363us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.964460 (+  1374us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.991937 (+ 27477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.016101 (+ 24164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.175387 (+159286us) raft_consensus.cc:1685] finished
0709 12:17:41.175408 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.175611 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56066,"prepare.queue_time_us":494082,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":326374,"raft.run_cpu_time_us":511,"raft.run_wall_time_us":28923,"thread_start_us":173490,"threads_started":2,"wal-append.queue_time_us":344244}
I20260709 12:17:41.330746  1566 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:41.332896  1272 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:41.374532  1220 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:41.312963  1529 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.341959 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.363714  1525 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.383529 32120 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 12:17:41.385776 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:41.398936 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.379273  1220 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:41.351732 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:41.341622 32115 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:41.403710 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.411378 31965 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:41.338932  1381 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.379170  1272 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:41.414095  1300 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:41.414458  1419 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:41.415001 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:41.416452 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:41.339257  1477 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.347167 32114 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:41.384663 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:41.381569  1331 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.421039 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.380960  1562 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.380964  1529 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.399485 32112 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:41.410818  1220 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:41.425295  1564 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.421411  1381 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.430475 31967 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:41.420701 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 119046us
I20260709 12:17:41.438797 31964 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.429114 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:41.379344  1566 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:41.456035 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.457237  1381 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.461499  1548 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:41.462805 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 119548us
I20260709 12:17:41.441861  1270 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:41.446188  1557 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.445075 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.479645 32120 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:41.486923 32044 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:41.485829 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:41.472616  1548 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.491513 32046 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:41.475911  1289 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.493556  1562 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:41.500873 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:41.505916 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 337331us
I20260709 12:17:41.490028 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:41.536525 32623 leader_election.cc:400] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:41.502491  1419 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:41.539656 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:41.516950  1566 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.507149 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:41.521302 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:41.520787 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:41.519193  1563 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.003s
I20260709 12:17:41.504877  1289 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:41.558847 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45130.9us
I20260709 12:17:41.501762  1562 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.507375  1492 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.518623 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:41.540867 32042 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:41.571579 32046 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:41.575603 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:41.578802 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:41.580526 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:41.582559  1559 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:41.547088  1291 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.520870  1272 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.585157  1551 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.579766 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:41.589579  1557 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.590389  1270 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.537070  1294 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.594058 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.536360  1581 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:17:41.602140 32040 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:41.603430 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:41.605649 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45952.6us
I20260709 12:17:41.591497  1419 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.615993 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.609858 32047 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
W20260709 12:17:41.622195 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 217362us
I20260709 12:17:41.628654 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:41.557987  1563 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.617555  1557 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.617058  1529 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.620886  1270 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:41.620702  1274 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.620996  1419 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.647112 32120 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:41.648123 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72650.6us
W20260709 12:17:41.673434 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 419992us
I20260709 12:17:41.633383  1559 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:41.687022 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:41.693169 32121 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.688445 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:41.645768 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.710389 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:41.713593 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71816.7us
I20260709 12:17:41.691884  1419 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.695389  1559 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:41.695993  1034 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:41.720786 32113 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:41.683568  1529 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.723784 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:41.729050 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 179731us
I20260709 12:17:41.729336  1581 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:41.728629  1593 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.712481  1477 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.730275 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:41.728415   964 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:17:41.733430 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:41.754148  1034 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.759054  1492 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.754240  1581 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:41.764309  1529 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.724622 32112 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:41.720907  1249 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:41.725554  1289 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.779402 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:41.783252  1581 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.786795  1611 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:41.790151  1612 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:41.793680  1611 raft_consensus.cc:1064] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:41.796051  1034 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.717450  1274 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.797767  1611 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.799105  1274 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:41.799366  1329 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.002s
I20260709 12:17:41.805250  1529 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:41.806985  1563 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:41.812803 32043 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:41.823573 31964 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:41.824476 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 126967us
W20260709 12:17:41.825841 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 97785.4us
W20260709 12:17:41.827033 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 105935us
I20260709 12:17:41.828970 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.805883  1529 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:41.830591 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 113385us
W20260709 12:17:41.833664 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53597us
I20260709 12:17:41.835987 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:41.806255  1270 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.832262  1563 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:41.844185 32659 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:41.829769  1249 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:41.857820  1596 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.858709  1529 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.858594  1596 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.866149 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:41.867677 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.839231 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:41.875648  1607 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:17:41.875736 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.844882  1274 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.877339  1275 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.877291  1274 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.878046  1275 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.879506  1274 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.877219 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:41.885069 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.865365  1593 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:41.886511 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.870911  1563 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:17:41.887238 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.889969  1562 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.862123  1381 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:41.880368  1275 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:41.900481 32121 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:41.845620 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.910585 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.914404 32659 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:17:41.917121 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70989.5us
I20260709 12:17:41.919342  1508 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
W20260709 12:17:41.919821  1508 raft_consensus.cc:1068] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Unable to promote non-voter ec7f26ea46944ea498ad253b1b1819b0: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:41.919997 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.835677  1559 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.920732 32655 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:41.907680  1524 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:41.925279 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:41.926287 32658 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:17:41.929170 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.933085 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:41.905169  1525 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.914549  1381 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:41.940258 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:41.941598 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.941854  1381 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.942615 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:41.944651  1270 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:41.944917 32114 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:41.945339 32114 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:41.923516 32656 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:41.907372  1563 consensus_queue.cc:1243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:17:41.954612 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.927138  1565 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.951628 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:41.938086 32651 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:41.950996 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.956650 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:41.935318  1524 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:41.930874  1289 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
W20260709 12:17:42.008078 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67936.4us
I20260709 12:17:41.982625 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.010473 32113 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:41.975302 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:41.951375  1270 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:41.996842 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:42.014112 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.951719  1251 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.016427  1610 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:17:42.016808 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:42.009040  1249 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.011286  1245 raft_consensus.cc:1027] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: attempt to promote peer ec7f26ea46944ea498ad253b1b1819b0: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:17:42.019639 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68118.7us
W20260709 12:17:41.989866 31966 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.001s
I20260709 12:17:42.011958  1272 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:42.076253 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58008.8us
W20260709 12:17:42.015476  1548 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.004s
I20260709 12:17:41.950282  1557 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:41.970745  1034 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:41.975608  1563 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.946135 32114 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:41.961517  1525 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.014204  1270 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:42.087914  1274 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.113068 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.113730 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.117833 32117 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:42.086951 31966 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:42.098443  1611 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.092267  1563 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.126132 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.099817  1270 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.134508  1602 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.134886 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.135524 32121 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:42.142119  1617 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.142755  1617 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:42.143639  1617 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.145392  1604 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:42.147111  1595 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:17:42.148361  1599 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.172s
I20260709 12:17:42.150521 32114 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.152735  1557 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.154393  1587 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:17:42.155915  1508 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:42.156417 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.157310  1508 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.157606 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:42.158267  1621 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:17:42.158653  1381 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.124536  1433 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:17:42.159082  1381 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:42.159511  1381 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.098934  1272 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.162825  1272 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.163192 32651 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:42.128790  1563 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.099921  1548 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.157720  1508 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.169827  1508 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.136906  1524 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.174302  1548 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.141368  1611 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.176903  1611 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.157428  1419 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.179543 31935 leader_election.cc:400] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5.
I20260709 12:17:42.179890 31935 leader_election.cc:403] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:17:42.137622  1220 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.001s
I20260709 12:17:42.120820  1274 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.196736  1274 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:42.198529 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85685.7us
I20260709 12:17:42.199736 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.200479 32112 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:42.200879 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.200968 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.201615 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.201869  1336 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:42.209717  1251 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.208981  1492 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:42.211095  1587 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:42.211272  1336 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.212849  1419 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:42.211683  1587 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:42.220957  1336 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.213615  1433 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:42.221910  1587 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.227820  1294 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:42.228287  1294 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:42.228703  1622 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:17:42.229192  1294 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.230984 32119 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:42.231076 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.232060 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:42.232664 32114 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:42.233626  1559 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:42.227831 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.235718 32119 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:42.206995  1272 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:42.237468 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 138190us
I20260709 12:17:42.237703 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:42.242014 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.207495  1602 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5.
I20260709 12:17:42.232396 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.236589  1294 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.193769  1220 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:42.237392  1372 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:42.247691 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.207168  1286 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
W20260709 12:17:42.250517 32010 leader_election.cc:336] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.843s (SENT)
I20260709 12:17:42.258469 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.256635 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.249280  1599 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.261715 32113 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:42.249830  1433 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.239691  1272 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.271145 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.244627  1270 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.273298  1220 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:42.250550  1581 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.270411  1525 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.278968  1331 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:42.280920 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46508.2us
I20260709 12:17:42.234258  1508 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.292219 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.293088  1419 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:42.293409  1508 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.293494  1419 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:42.293867 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.294535  1581 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:42.295037  1274 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.288991  1272 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.294051  1419 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.295584  1581 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.296492 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.280256  1381 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.297235  1286 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.278584  1565 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.261723  1602 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.285472  1599 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEARNER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.299634  1286 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:42.299646  1381 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:42.301259 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52774.2us
I20260709 12:17:42.301182 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.303472 32118 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:42.307057  1632 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.307474  1632 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.307814  1632 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.310382  1551 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.314828  1270 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:42.315148 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:42.314952  1563 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.315994 32115 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 3: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:42.317214  1563 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.318957  1632 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.320930  1632 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.331316 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.335366  1602 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:42.336313  1331 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:42.336448 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.337184 32112 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:42.338165 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:42.340466 32114 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.344141  1562 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:42.344691  1562 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.347259  1562 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.336624 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:42.351125 32114 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:42.352241  1300 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.352698  1300 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.340425 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.353729 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.376533 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:42.378607 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74656.7us
W20260709 12:17:42.385557 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45199us
I20260709 12:17:42.400080 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:42.304694  1565 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.401185 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:42.403887 32114 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 12:17:42.387058 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:42.405659  1543 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.375401  1300 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:42.408322  1565 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.464010  1565 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.465070 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.423646  1593 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:42.465773  1593 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.399839  1381 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:42.456993 32010 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.801s (SENT)
I20260709 12:17:42.413863  1529 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 12:17:42.476488 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89550.2us
I20260709 12:17:42.413472  1492 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:42.385573 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 82537.2us
I20260709 12:17:42.354482  1599 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 12:17:42.457983 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71819.4us
I20260709 12:17:42.459530 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.485005 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.485525 32114 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:42.470780 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:42.487150 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:42.475862  1543 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.489414  1543 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:42.489670  1572 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:42.490551  1615 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:42.470511  1331 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.492027 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.456601  1294 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.490123  1572 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:42.484760  1286 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:42.501114 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.502699 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:42.493155  1565 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.496760  1548 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:42.496938  1331 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.513487 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:42.513782 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.500174  1562 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:42.490868  1543 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.518689 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:42.529680 32120 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:42.531133 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74180.9us
I20260709 12:17:42.510932  1565 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.539345 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.512220  1272 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.509166  1563 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.512235  1616 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.528818  1564 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.519199  1331 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.534785  1525 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:42.507720  1294 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.519256  1615 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:17:42.542224  1565 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:42.549587 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 148496us
I20260709 12:17:42.512965  1548 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:42.545733  1576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.021s	sys 0.029s
I20260709 12:17:42.544265  1611 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.536075  1562 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.550288  1564 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.605544 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:42.628088 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:42.631732 32660 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:17:42.638660 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51648.5us
W20260709 12:17:42.623600  1576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.022s	sys 0.034s
I20260709 12:17:42.642251 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.589032  1604 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.604293  1548 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:42.655804  1627 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.425s	user 0.001s	sys 0.005s
I20260709 12:17:42.658524  1642 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:42.659305 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:42.662067  1300 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.668351  1300 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.669337  1596 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.672976  1433 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.583436  1572 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.673669 31971 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:42.674204  1572 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.640153  1564 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.676713 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.677549 31972 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:42.683576  1294 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.684125  1564 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:42.684628  1245 consensus_queue.cc:1175] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "38c927f0ee104b57972899f5b27aa99b" responder_term: 6 status { last_received { term: 2 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.2-5.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:42.686947  1604 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.687170  1331 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.689291  1331 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.689657  1286 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:42.690021  1270 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:42.690416 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 190892us
I20260709 12:17:42.690728  1286 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.691279 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:42.691530  1220 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:17:42.691727 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 120951us
I20260709 12:17:42.692049  1220 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.692842 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:42.693311 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.694234  1372 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:42.694414  1581 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.694895  1581 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.643834  1381 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:17:42.695374  1270 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.644946 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:42.699863 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55001.4us
I20260709 12:17:42.645713 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:42.645686  1628 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 12:17:42.623783  1525 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:42.625782  1566 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:17:42.645802  1289 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.648990  1249 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:42.648808  1419 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.651235 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:42.652742 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.654145 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:42.733318  1576 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:42.673631 31965 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:42.587841  1272 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:42.708163 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:42.695485  1581 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:42.655654 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:42.723361 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 81189.9us
I20260709 12:17:42.805383 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.686141  1564 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.819324 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.694140  1220 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:42.783445 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.784516 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:42.633266  1613 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.694840  1372 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:42.645751 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:42.696019  1381 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.789628 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:42.741189  1642 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:17:42.789189 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:42.790444  1596 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:42.804862  1617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.001s	sys 0.017s
I20260709 12:17:42.791731 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:42.797679 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 74350us
I20260709 12:17:42.792742 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:42.792999 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:42.799642 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65778.5us
I20260709 12:17:42.655618  1562 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:42.802170 31964 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.797266 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.788192  1294 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.820058 31963 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:42.820624 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:42.843448 32120 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.788587  1289 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.821125 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:42.819588  1551 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.847019 31965 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:17:42.791078 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.849854 32660 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.848266 32118 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:17:42.863127 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 169857us
I20260709 12:17:42.864415 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:42.866632 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46108.7us
I20260709 12:17:42.867120 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.867861 31972 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:17:42.868862 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62120.7us
I20260709 12:17:42.799811  1615 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:42.853396  1617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.002s	sys 0.017s
I20260709 12:17:42.872799 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:42.873442 32658 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:17:42.874336 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.842751  1628 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:42.875996 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.877189 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.877859 32043 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:42.842876  1381 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:42.886725 32047 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:42.803079  1593 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.801501  1249 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.852079 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:42.884133 32116 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.2-6.2]
I20260709 12:17:42.886180 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:42.890031 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.900353 31971 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:42.901086  1628 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:42.901401  1372 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.891312 32119 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:42.807322  1251 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:17:42.852613 32656 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:42.898275 32044 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:42.872946  1336 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.927699 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.842742  1613 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.902833 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:42.930644 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.953127 32043 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:42.926926 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.948005 32039 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:42.865514  1289 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.886749  1274 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.897990  1633 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.956480 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.970695 31963 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:42.957131 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:42.902554  1615 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.963152 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.895654  1576 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.964282 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.979709 31968 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:42.956380 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.981048 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:42.981313 31964 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:42.981670 32039 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.876240  1551 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.966562 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.979882 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.993383 31969 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 3.
I20260709 12:17:42.967000 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.994748 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:42.996008 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:42.997078 31963 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:42.998799 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:42.999585 32045 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.964450 32040 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:42.969805 31972 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 12:17:42.927544 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:42.927423 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:42.965451 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:42.983692 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.984279 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:42.874053  1294 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:42.931357  1300 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.019456 32046 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:42.970305  1633 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:43.023702 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:43.024309 32118 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.025462 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.029930 32657 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:43.031721 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:42.994626 32115 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:43.034834 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.035455 32040 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.973613  1274 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.056593 32039 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:43.021436 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.058553 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:43.061650 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:43.050566 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:42.956259  1272 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:42.948988  1525 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.063282 32038 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:43.047783  1610 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.029973  1245 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:17:43.065866 32653 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:43.050872  1563 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:42.952561  1628 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.104264  1651 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:43.104785  1643 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.204s	user 0.000s	sys 0.002s
W20260709 12:17:43.105432  1625 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.893s	user 0.005s	sys 0.011s
I20260709 12:17:43.105173  1565 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.105952  1286 raft_consensus.cc:2764] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:43.106271  1644 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:17:43.106858  1572 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:43.002080  1576 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.068887  1419 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.068933  1249 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.107692  1604 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.033715  1300 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.110884  1615 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.119800  1593 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.012665  1381 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.115156  1543 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:17:43.115048  1632 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:17:43.130789  1551 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:43.138845 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52855.7us
I20260709 12:17:43.213231  1294 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.213991  1632 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:43.214360  1632 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:43.215138  1632 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.217516  1576 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.219740 32120 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:43.138935 31965 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:43.145440 32040 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:43.174935 32118 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:43.085376  1633 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.193230 32660 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:43.196180 32047 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:43.085121 31966 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:43.177279  1559 raft_consensus.cc:670] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:43.187207 32045 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:43.212677  1543 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:43.225973 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.231105 32040 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:43.227589 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 363274us
I20260709 12:17:43.229660  1647 raft_consensus.cc:2764] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:43.231024 31965 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
W20260709 12:17:43.226437 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 266498us
I20260709 12:17:43.224408  1632 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.235220 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:43.241899 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:43.136111  1610 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.244706 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.245782 31967 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.245664 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:43.232378  1381 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:43.268250 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.259992 31965 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.239248  1572 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:43.284309 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 258957us
I20260709 12:17:43.234253  1604 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.286413 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.289073 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:43.290089 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.290880 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.241626  1593 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:43.295706 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.304087 32043 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:43.304500 31965 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.306768 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:43.307407 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.309039 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:43.287685  1419 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:43.287103  1664 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:43.311347 31967 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.310134 32043 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:43.293184  1559 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:43.315091 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 79922.9us
I20260709 12:17:43.295876  1551 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.319070 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.324508  1672 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.324622  1610 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.325083  1672 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.325537  1672 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.324661  1662 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:17:43.291883  1572 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.327877 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.310110  1565 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.332335 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.313989  1294 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.335752 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.349030 32043 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:43.354229  1270 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.000s
I20260709 12:17:43.354698  1270 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.354911  1664 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:43.355257  1289 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.355967  1664 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.309861  1593 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.333045 31966 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
W20260709 12:17:43.361413  1548 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.705s	user 0.005s	sys 0.004s
I20260709 12:17:43.313290  1419 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:43.363634  1419 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:43.363962  1529 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:43.364331  1529 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:43.365087  1529 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.366042  1529 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.367689  1572 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.368191  1572 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.368649  1572 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.358016  1289 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.345405  1654 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.345992  1274 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.377357  1275 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:43.377830  1275 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:43.378772  1275 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.379719  1275 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:43.380105  1581 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.003s
W20260709 12:17:43.388103  1656 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:17:43.322158  1559 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:43.391530  1559 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.307574  1666 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:43.321688  1286 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.395695  1286 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:43.397408 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.397725  1286 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.361924  1548 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.399065 32040 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:43.400231 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.401247 31967 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:43.407971 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.411355 32043 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:43.320326  1613 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:43.436740 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 117742us
W20260709 12:17:43.440402 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 108137us
I20260709 12:17:43.441991 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.443020  1613 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.449725 32660 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:17:43.450258 32660 pending_rounds.cc:85] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 5
I20260709 12:17:43.450579 32660 pending_rounds.cc:107] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:17:43.450873 32660 raft_consensus.cc:2889] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:43.457180 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.417402  1572 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:43.468080 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.470172 32043 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:43.471406 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.472084 32043 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:43.439393  1289 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.474263 32046 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:43.501210 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.504666 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:43.466589  1286 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.434476  1581 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.504623 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:43.472150  1613 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.473377  1596 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.472252 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:43.504113  1548 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.473717  1665 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.514976  1593 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.530718 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.499600  1632 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:17:43.533766  1331 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:43.525996  1274 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:43.525756  1596 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.541424  1286 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:43.559432 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:43.525032  1613 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.544413  1274 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:43.586683 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.590268  1274 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.594880  1613 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.547158  1331 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.615026 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:43.618791 32040 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:43.647001 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:43.621268   964 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.680323   964 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.680922   964 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.660905  1660 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:43.687867  1660 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:43.688912  1660 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.691992 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.694703 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:43.601804  1419 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:43.701598 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 143000us
I20260709 12:17:43.704939 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:43.714484 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.715204 31969 raft_consensus.cc:2368] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:43.717823 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.700758  1657 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.724406 31965 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:43.725126 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.726774 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.724025 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.727907 31972 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.728158 31967 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.729285 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.730572 31969 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 6 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:43.713033  1551 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.731716 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.735229 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:43.732362 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58305.4us
I20260709 12:17:43.711531  1581 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.738606 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.738808  1559 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:43.740232  1676 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:17:43.740352 31969 raft_consensus.cc:2410] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:43.741782 32084 leader_election.cc:400] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:43.762394 32084 leader_election.cc:403] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:43.742470  1286 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.763870 32659 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:43.764501 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 398) took 1030 ms. Trace:
I20260709 12:17:43.764878 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.764889 32659 rpcz_store.cc:276] 0709 12:17:42.733821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.734016 (+   195us) service_pool.cc:224] Handling call
0709 12:17:43.763474 (+1029458us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:43.763686 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.764243 (+   557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.764432 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8891648}
I20260709 12:17:43.763759  1286 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.768069 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.768841 31969 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.769161  1286 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.770058 31969 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.709419  1664 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:17:43.764240 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:43.795512 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58800.7us
I20260709 12:17:43.735751  1551 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:43.810194 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.823204  1581 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:43.823905  1596 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.819957  1286 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:43.822558 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.826004  1286 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.826838  1669 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 12:17:43.827790 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.829833 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:43.827980 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:43.831007 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.771296  1602 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.834228  1581 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.766089 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.838234 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.795291 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.840720 31966 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.841847 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:43.843284  1683 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:17:43.843739 32041 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:43.844594 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:43.821844  1551 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.845783 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.847568 32047 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:43.834389  1602 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.848601 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.775319  1613 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:17:43.833434  1286 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.852118 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.852981 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.853493 32047 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:43.796582  1672 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:43.854780 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.844669 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.858990 32041 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:43.845005  1419 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.865511 32047 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.839066  1581 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:17:43.867395 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.871796 32047 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.872967 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:43.873554 31966 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.875073 32652 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.875437 32652 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:43.883299 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:43.884166  1419 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:43.883903 32045 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
W20260709 12:17:43.886083 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.868587  1559 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.884574  1419 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:43.883026 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.893695 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:43.920826  1419 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:43.922113  1419 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.924998 31966 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:17:43.927897 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.928719 32660 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:43.885517 32652 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.908555  1672 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.930837 32041 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:43.919687  1602 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.958815 32652 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.968856 31972 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:17:43.958634  1689 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:17:43.966516 31966 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:17:43.975225  1649 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.973s	user 0.003s	sys 0.003s
I20260709 12:17:43.938879   964 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:43.978884 31969 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:43.953888  1602 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:17:43.980207 32654 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:43.982549  1300 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:43.985167  1664 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:43.986403  1559 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:43.986595 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:43.989820 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:43.966952  1274 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:43.996088  1640 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.093s	user 0.000s	sys 0.007s
I20260709 12:17:43.997731  1274 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.992558  1220 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:43.998649  1220 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.001127  1220 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:43.992767  1660 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:44.002823  1581 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.003504  1581 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.007871 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:44.009969 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.013425 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.015476 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:44.006107  1581 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:43.987475 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.017619 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:44.018287  1664 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:44.018654 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.019181  1593 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:44.019737  1274 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.019857   964 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:44.016659  1220 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.023217 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.011828 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.026640 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:44.027856 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.028600 32042 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:44.025653  1581 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.029480 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.019798 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 400) took 1219 ms. Trace:
I20260709 12:17:44.031369 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.035004 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.036387  1220 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.037772 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.032253 32654 rpcz_store.cc:276] 0709 12:17:42.797851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.800615 (+  2764us) service_pool.cc:224] Handling call
0709 12:17:43.975587 (+1174972us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:43.979875 (+  4288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.991955 (+ 12080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.014968 (+ 23013us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9537792}
I20260709 12:17:44.042804 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.042680 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1538) took 1059 ms. Trace:
I20260709 12:17:44.043463 32116 rpcz_store.cc:276] 0709 12:17:42.974491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.986056 (+ 11565us) service_pool.cc:224] Handling call
0709 12:17:42.986588 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.005061 (+ 18473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.005092 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.929767 (+924675us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:43.930181 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 12:17:43.969985 (+ 39804us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:43.970024 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.970113 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.030258 (+ 60145us) raft_consensus.cc:1685] finished
0709 12:17:44.030288 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.034019 (+  3731us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20872,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":11668,"spinlock_wait_cycles":20777344,"thread_start_us":39411,"threads_started":1,"wal-append.queue_time_us":86047}
I20260709 12:17:44.042371 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.046375 32042 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:17:44.045723 31969 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:44.024427 32041 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.041894  1651 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.032413  1664 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:44.055120  1664 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.058720 32042 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:17:44.059126 32042 pending_rounds.cc:85] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 2
I20260709 12:17:44.059362 32042 pending_rounds.cc:107] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:17:44.059603 32042 raft_consensus.cc:2889] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:44.060772  1551 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.062927  1551 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:44.034643  1581 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.073170 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:44.079561 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.080109 32118 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.081228  1274 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:44.082896 32116 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:44.084225 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.091434 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.096421 32044 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:44.097988 32044 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:44.100503 32652 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:44.078685  1660 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:44.114576  1599 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:44.122589 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.118865 32652 pending_rounds.cc:85] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 5
I20260709 12:17:44.125371 32118 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:44.116225 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 397) took 1397 ms. Trace:
I20260709 12:17:44.144296 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.145272 32121 raft_consensus.cc:2410] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:44.081970  1274 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.130295 32652 pending_rounds.cc:107] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:44.151171 32652 raft_consensus.cc:2889] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:44.140002 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.154717 32112 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.156312 31968 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:44.153014 31972 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:44.160377  1704 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:44.145659  1336 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.162755  1336 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.163486  1705 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:44.164285  1705 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.164990  1336 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:44.166384 32654 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.167917  1705 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:44.168606  1692 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.274s	user 0.000s	sys 0.002s
I20260709 12:17:44.169588  1647 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.170077  1647 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:44.171506 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 187866us
I20260709 12:17:44.148476 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.176800 32119 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:44.176870 31968 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:44.177980 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:44.123579  1660 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:44.179903 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 106835us
I20260709 12:17:44.183384 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:44.186638 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:44.181591 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:44.174506  1666 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:17:44.174641  1275 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:17:44.182510  1525 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:44.183996 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.194972 31968 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:44.197713 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:44.200382 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.201999 32044 raft_consensus.cc:2368] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:44.199846  1272 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.189014  1662 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:44.212095 32045 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:44.189309  1419 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:44.201814 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.223495 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.201493  1713 raft_consensus.cc:2764] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:44.188524  1220 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:17:44.207005  1664 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.226505  1664 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.226701 32112 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:44.226925  1664 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:44.227334 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.228665  1419 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:44.229022 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 397) took 1530 ms. Trace:
I20260709 12:17:44.229746  1419 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.206817  1613 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:44.136694  1599 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.229485 32653 rpcz_store.cc:276] 0709 12:17:42.699009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.722231 (+ 23222us) service_pool.cc:224] Handling call
0709 12:17:42.838058 (+115827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.128338 (+290280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.228762 (+1100424us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540053edd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:44.228950 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273908608}
I20260709 12:17:44.211257  1662 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.211460 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:44.230677 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:44.244344 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.209151 31966 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:44.245860 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49583.8us
I20260709 12:17:44.200062  1525 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.254006 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.214444  1272 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.231156  1647 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:44.257205 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 75650us
I20260709 12:17:44.246625  1662 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.260506 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:44.262769 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:44.246387  1613 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:44.264418  1715 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.265090  1581 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.265340  1713 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:44.266837  1372 raft_consensus.cc:2764] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:17:44.268966 32652 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:44.274663 32042 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.275920 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.279557 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.279718 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.280432  1581 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.280758 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.280887  1581 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.281489  1581 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.281513  1717 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:44.283876  1685 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.543s	user 0.001s	sys 0.003s
I20260709 12:17:44.280865 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.285347 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.273475  1525 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:44.259728   964 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:17:44.286659  1714 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.287094   964 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:44.287302  1714 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.287606   964 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:44.287909  1714 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.288470   964 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.289561   964 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.286813  1274 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:44.290941  1613 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.269469  1286 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:17:44.137413 32655 rpcz_store.cc:276] 0709 12:17:42.719069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.719867 (+   798us) service_pool.cc:224] Handling call
0709 12:17:43.376813 (+656946us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:43.377040 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.377331 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:43.507237 (+129906us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.081852 (+574615us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b180035eb40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:17:44.114043 (+ 32191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:44.114175 (+   132us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:44.114302 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.115183 (+   881us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.115951 (+   768us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140510,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":1110,"raft.queue_time_us":196,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":40836224,"thread_start_us":127988,"threads_started":2,"wal-append.queue_time_us":614427}
I20260709 12:17:44.299075 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.299942  1286 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:44.300410  1660 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:44.301646 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.308251 32041 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:44.309356  1274 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:44.309402  1660 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.309978  1272 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:44.308879 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:44.318370 32084 leader_election.cc:400] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:44.319387 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.319841 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.316676  1664 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.326637 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.327257 32121 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:44.328141 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.317533  1660 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:17:44.329782 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:44.318259  1710 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:17:44.330425 31966 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.323632  1372 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:17:44.311350  1596 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.323796  1272 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.333662  1670 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:44.333897  1272 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:44.334103  1670 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:44.334264 32038 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.334993  1272 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.335198  1670 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.336504  1670 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.325193  1564 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:44.316203 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.337109 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 175) took 1368 ms. Trace:
I20260709 12:17:44.340746 32038 rpcz_store.cc:276] 0709 12:17:42.968644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.242621 (+1273977us) service_pool.cc:224] Handling call
0709 12:17:44.315746 (+ 73125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.334783 (+ 19037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.336050 (+  1267us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54000578d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:44.336913 (+   863us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b54000578d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:44.337054 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139501696}
I20260709 12:17:44.349519 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.353717 32112 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:44.356607 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:44.340639  1662 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.356786  1664 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:44.360564 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.361091  1651 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:44.362746  1686 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.587s	user 0.000s	sys 0.003s
I20260709 12:17:44.360623  1714 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.362882  1651 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:44.363914  1651 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.364552  1704 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:44.361816  1647 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:17:44.367888 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.360806  1581 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.370395 32047 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:44.370949 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.376446 31966 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
W20260709 12:17:44.378450 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58690.8us
I20260709 12:17:44.373732 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.383105 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.383428 32047 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:44.380002 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:44.386662 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.321276  1715 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:44.398476 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.365140  1651 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.369261  1717 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:44.410023 32117 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:44.410678 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.413286 31971 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:44.412111 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.382072  1618 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:44.393352  1286 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:44.386077  1660 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.395373  1593 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.420958  1618 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:44.424896 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:44.428668 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.412249  1717 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.437913 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.429430 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:44.435045 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.393157  1713 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:44.448876 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:44.452596 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.452597 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.453576 32042 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:44.454417 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.455468 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.456204 32042 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:44.456977 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.425940  1566 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.461530 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.462356 32041 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:44.430480  1613 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:44.490228 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60895.7us
I20260709 12:17:44.491940  1665 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:44.496395 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.430166 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.448280  1632 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.419847  1551 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:17:44.502667 32120 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:44.511978 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.503424 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.448155  1717 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:44.488742  1721 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:17:44.451866  1593 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:44.452128  1548 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.458889  1697 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.460605  1272 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:44.489840  1565 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.514783 32114 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:44.520692 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 173) took 1562 ms. Trace:
I20260709 12:17:44.499073  1372 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:44.510979  1286 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:44.530603 32046 rpcz_store.cc:276] 0709 12:17:42.955301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.226593 (+271292us) service_pool.cc:224] Handling call
0709 12:17:43.226952 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.505153 (+278201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.517319 (+1012166us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54001941d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:44.517516 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567688448}
I20260709 12:17:44.512589 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.534911 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.535610 32119 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:44.536294 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.504889  1613 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.542176 31971 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:44.543370 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.547072 31972 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.543859 32119 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:44.551218 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:44.528115  1272 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:44.514472  1632 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.554291 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.558910 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.558816 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:44.562856 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.563127 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.563236 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.524755  1618 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.564301 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.564497  1618 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.564723 32119 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.565342 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.566113 32119 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:44.566612  1593 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.570518 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.571099 31964 raft_consensus.cc:2376] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6. Re-sending same reply.
I20260709 12:17:44.571887 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.572168 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:44.533118  1372 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:44.566301 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:44.581113 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.540017  1714 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:44.627944 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:44.589186 31972 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:44.591606 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.628605 31971 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.545205  1529 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.629304 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:44.559866  1272 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.571386  1336 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:44.518003  1715 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.628456  1566 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:44.590549  1372 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.533751  1697 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:44.627879  1716 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:17:44.562135  1718 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:44.520881 32121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:44.656098 31963 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.661832 32121 raft_consensus.cc:606] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:44.647321 32119 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:44.657044 32114 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:44.662194 32121 raft_consensus.cc:614] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:44.636859  1529 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.669028 31869 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 4, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:44.670074  1529 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:44.674616 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.675194 31963 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:17:44.639067  1581 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:44.676565  1720 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:17:44.678129  1719 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:17:44.681741 31969 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:44.676755 31963 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:44.686523 31966 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:44.647188  1336 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:44.706990 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.708053 32121 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:44.699883 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:44.709035 32112 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:44.655429  1714 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:44.633265  1562 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:17:44.711264 32119 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:44.713377 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:44.719538  1419 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:44.720116  1419 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.721436 32119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:37262
W20260709 12:17:44.721693  1722 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:17:44.721998 32119 raft_consensus.cc:606] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.710269  1336 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.722596  1714 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.724062  1419 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:44.722424 32119 raft_consensus.cc:614] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:44.724697  1714 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:44.686234  1718 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.664054  1331 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:17:44.735564 32118 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.741178 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.742235 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1423) took 1027 ms. Trace:
I20260709 12:17:44.742655 32043 rpcz_store.cc:276] 0709 12:17:43.714300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.740025 (+ 25725us) service_pool.cc:224] Handling call
0709 12:17:44.740964 (+1000939us) spinlock_profiling.cc:243] Waited 936 us on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:44.742161 (+  1197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152320}
I20260709 12:17:44.748706 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 178) took 1279 ms. Trace:
I20260709 12:17:44.749099 32040 rpcz_store.cc:276] 0709 12:17:43.469607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.469750 (+   143us) service_pool.cc:224] Handling call
0709 12:17:44.748556 (+1278806us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b5400032ad4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:44.748652 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214756224}
I20260709 12:17:44.749490 32043 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.750399 32040 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:44.750784 32043 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:44.799947 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:44.805214 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.704005  1664 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:17:44.726974  1647 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:44.808569  1300 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:44.809113  1300 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.811146  1300 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:44.689252  1286 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:44.806373 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.816186  1735 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:44.820374  1272 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.817654  1372 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:44.826776 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:44.830752 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 270882us
I20260709 12:17:44.832638 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.831998 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.806717  1714 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:44.832991  1723 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 12:17:44.879572 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:44.880553 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:44.880806 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:44.881371 32043 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:44.894799 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:44.895380 32043 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.871932  1331 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:44.953601 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72861.1us
I20260709 12:17:44.954551 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:44.925037  1697 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:44.984916  1697 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.894562  1632 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.031348 32117 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.952827 32115 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.879487  1613 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:44.940699  1286 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.917791   964 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:44.959487  1618 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:45.040761 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:45.042090 31971 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:45.049906  1618 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.050671  1730 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.051321  1730 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:45.052559  1618 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:45.053253  1670 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.000s
I20260709 12:17:45.053550  1564 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:45.053957  1564 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:45.054677 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1471) took 2160 ms. Trace:
I20260709 12:17:45.054867  1564 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.043499  1581 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.056061  1564 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.055052 32039 rpcz_store.cc:276] 0709 12:17:42.893926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.069821 (+175895us) service_pool.cc:224] Handling call
0709 12:17:43.070139 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.070480 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.070517 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:44.352139 (+1281622us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b2c0005c228. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:44.352560 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.425721 (+ 73161us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:44.425754 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.425825 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.054389 (+628564us) raft_consensus.cc:1685] finished
0709 12:17:45.054412 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.054623 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61961,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":9900288,"thread_start_us":72823,"threads_started":1,"wal-append.queue_time_us":104643}
I20260709 12:17:45.056246  1581 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.056949 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.058846  1581 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.060941  1336 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:45.063803  1733 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:17:45.058941  1728 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.066804  1728 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.069535  1728 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.025763  1697 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:45.071082  1697 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:45.073560 32013 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.605s (SENT) [suppressed 10 similar messages]
I20260709 12:17:45.049849 32043 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:45.032109  1734 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.098716  1602 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:45.099332  1602 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:45.099433  1336 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:45.100893  1717 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.580s	user 0.005s	sys 0.005s
I20260709 12:17:45.101488  1717 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.101807 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.102552 32119 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:45.098857  1734 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.103652 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.103874  1717 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.104842 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.104991  1734 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.043953  1331 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:45.105943  1602 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:45.101915 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.076892 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.100827  1599 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.054037  1730 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.105991 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.227100 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:45.227782 32041 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:45.229266 31971 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:17:45.108487 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67771.2us
I20260709 12:17:45.108871 32112 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:45.058801  1739 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:17:45.108844 32042 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:45.272428  1752 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:17:45.280293 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:45.094343  1632 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:45.159591  1529 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:45.099742  1657 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:45.131882  1581 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:45.042742  1286 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.226711  1372 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.309624  1274 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.278405  1336 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:45.326581  1336 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:45.327476  1336 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.130807  1718 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.297581 32655 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:17:45.325189  1286 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.106976  1272 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:45.322083  1651 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.000s
I20260709 12:17:45.351078 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.318735  1657 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:17:45.338536 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 384041us
I20260709 12:17:45.354512 32120 raft_consensus.cc:2410] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:45.316406  1581 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.045166   964 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:45.348445  1565 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.004s
I20260709 12:17:45.362859 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:45.344990  1275 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.348987  1572 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:45.106391  1331 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.386224 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.375671  1657 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.377180 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1446) took 1188 ms. Trace:
I20260709 12:17:45.359108  1581 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:45.335330  1274 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.383577  1275 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:17:45.408951 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.368104  1565 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:45.418409 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.418565 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.419157 32119 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:45.420866  1699 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:45.379364  1752 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:45.409690 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.387756 32044 rpcz_store.cc:276] 0709 12:17:44.188771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.269662 (+ 80891us) service_pool.cc:224] Handling call
0709 12:17:45.377001 (+1107339us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54001950d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:45.377124 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102680832}
I20260709 12:17:45.413236  1272 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:45.413834  1697 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.415136  1669 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:45.387413  1748 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:17:45.429234 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.441560 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:45.431442  1731 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:45.431478  1272 consensus_queue.cc:1243] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:45.459123  1565 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:45.459591  1565 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:45.460597  1565 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.435767  1742 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:45.443452 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:45.466861  1742 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.448046  1275 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.449594  1274 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.454257  1669 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.467338  1742 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.456597  1660 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:45.469532  1669 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.469765  1660 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:45.457455 32655 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:45.473443 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1483) took 1049 ms. Trace:
I20260709 12:17:45.474951 31969 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:45.477181 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:45.478714 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:45.480857  1732 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:45.481283  1732 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:45.481805 32654 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:45.482424 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.483158 32660 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:45.484627  1593 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:45.485052  1593 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:45.394953  1572 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:45.485508 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.485901  1593 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.487251  1593 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:45.482074  1732 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.496220  1732 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:45.498731 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 218505us
I20260709 12:17:45.498956 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:45.486057  1572 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.522784 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:45.473883 32042 rpcz_store.cc:276] 0709 12:17:44.423923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.477188 (+ 53265us) service_pool.cc:224] Handling call
0709 12:17:44.477569 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.282779 (+805210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.473237 (+190458us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54000587d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:45.473388 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451281792}
I20260709 12:17:45.521016 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.462070  1565 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.526852 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:45.538719 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.470613  1742 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.541520 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:45.542729  1742 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:45.526685  1272 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.543743  1272 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.544111  1742 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.544984  1752 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:45.545109  1599 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:45.545660  1752 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.545866  1272 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.546931 32039 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:45.547446 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.543216 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.551034 32041 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:45.540758 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.571249  1272 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.571538 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.542725  1697 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.572217 32659 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:45.571875  1272 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:45.572865  1740 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.539s	user 0.000s	sys 0.003s
I20260709 12:17:45.573549 32659 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:17:45.573936 32659 pending_rounds.cc:85] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 6
I20260709 12:17:45.553694 32044 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:45.544608  1742 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.574263  1272 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.576176  1742 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.578547 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.578761 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1369) took 1167 ms. Trace:
I20260709 12:17:45.579375 32656 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:45.580367 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 191) took 1082 ms. Trace:
I20260709 12:17:45.580708 32040 rpcz_store.cc:276] 0709 12:17:44.498301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.801276 (+302975us) service_pool.cc:224] Handling call
0709 12:17:44.807405 (+  6129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.807764 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.807793 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:44.808137 (+   344us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.001607 (+193470us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:45.001672 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.001889 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.580177 (+578288us) raft_consensus.cc:1685] finished
0709 12:17:45.580196 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.580314 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":75961,"threads_started":1,"wal-append.queue_time_us":741966}
I20260709 12:17:45.581611  1581 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.579109 31968 rpcz_store.cc:276] 0709 12:17:44.411201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.411322 (+   121us) service_pool.cc:224] Handling call
0709 12:17:44.411772 (+   450us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:44.426555 (+ 14783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.426602 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:44.436889 (+ 10287us) log.cc:844] Serialized 78 byte log entry
0709 12:17:44.778667 (+341778us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:44.778728 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.778806 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.550267 (+771461us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:45.578459 (+ 28192us) raft_consensus.cc:1685] finished
0709 12:17:45.578480 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.578703 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10062,"spinlock_wait_cycles":4374784,"thread_start_us":267820,"threads_started":1,"wal-append.queue_time_us":380876}
I20260709 12:17:45.582818 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 171) took 1082 ms. Trace:
I20260709 12:17:45.584142 32118 rpcz_store.cc:276] 0709 12:17:44.500608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.501698 (+  1090us) service_pool.cc:224] Handling call
0709 12:17:44.735394 (+233696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.739170 (+  3776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.582596 (+843426us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54004349d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:45.582750 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536734080}
I20260709 12:17:45.586134  1599 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:45.586979  1599 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:45.587361  1599 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:45.588200  1599 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.546154  1752 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.589331  1599 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.590966  1752 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.578156 32652 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:45.593782  1662 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.594481  1714 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.595124  1714 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.597373  1714 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.597826  1274 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.599659  1274 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.601620  1274 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.470772  1660 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.602391  1660 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.607230 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:45.609426 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:45.587142  1755 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:45.609822 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.611888 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:45.612629 32660 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.597707 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:45.574151 32659 pending_rounds.cc:107] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:17:45.619055 32659 raft_consensus.cc:2889] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:45.630690 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:45.640336 31869 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:45.642123 31869 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 3 to 4, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.646631 31869 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 6, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 6 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.654269 32040 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:45.654477 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:45.656399 31964 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:45.551990  1572 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.669138 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:45.670627 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:45.672060 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:45.672731 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:45.680459 32660 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:45.653527  1596 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:45.688928  1747 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:45.689707 32658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.050s	sys 0.118s
I20260709 12:17:45.661590  1755 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:45.658820  1372 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:45.690449 32658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.050s	sys 0.118s
I20260709 12:17:45.673075  1286 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:17:45.695369 32013 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.000s	sys 0.012s
W20260709 12:17:45.695792 32013 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.000s	sys 0.012s
W20260709 12:17:45.696167 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 622574us
I20260709 12:17:45.698661  1672 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:45.674733  1419 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:45.701100 31869 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:17:45.710086 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.661458  1730 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:17:45.690541  1596 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:45.734001 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.754732  1596 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:45.719702  1632 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:45.756399  1632 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.691059  1664 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.667870  1746 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:17:45.699918  1286 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.765107  1286 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:45.765475  1274 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.766098  1274 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.768697  1274 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.769795  1730 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:45.770223  1651 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:45.751492  1754 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:45.775842 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 414551us
I20260709 12:17:45.722010  1662 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:45.779820 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 82369.2us
I20260709 12:17:45.722241  1372 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:45.722041  1419 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:45.759815 32038 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:45.785384 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:45.759263  1725 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:45.787472 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.793392 32114 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.793717 32121 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:45.745085   964 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.795879 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1681) took 1251 ms. Trace:
W20260709 12:17:45.792229 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 410299us
I20260709 12:17:45.793052 32120 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:45.790457 31869 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 3, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 3 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.721828  1747 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:17:45.805236 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:45.796222 31972 rpcz_store.cc:276] 0709 12:17:44.544232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.629253 (+ 85021us) service_pool.cc:224] Handling call
0709 12:17:44.629530 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.629714 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:44.648125 (+ 18411us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.420742 (+772617us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b18003cb9e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:17:45.795639 (+374897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:45.795688 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:45.795728 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.795771 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.795840 (+    69us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3374,"prepare.queue_time_us":17941,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2292,"spinlock_wait_cycles":24711424,"thread_start_us":371287,"threads_started":1}
I20260709 12:17:45.770236  1730 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:45.801949 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.794416  1419 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:45.819119 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.819733 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:45.826521 32121 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.830042 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.830907 32043 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:45.774818  1664 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.830121 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.832808 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.834956 32118 raft_consensus.cc:2410] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:45.833972 32114 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:45.836119 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.791620  1336 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:45.839444 32116 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:45.794646  1666 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 12:17:45.843400 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.858438 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:45.858438 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:45.860638 32119 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:45.862445 32043 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
W20260709 12:17:45.863682 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 357259us
I20260709 12:17:45.863813 32041 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:17:45.865635 32041 pending_rounds.cc:85] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 5
I20260709 12:17:45.865944 32041 pending_rounds.cc:107] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 5.6
W20260709 12:17:45.865883 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53529.6us
I20260709 12:17:45.866478 32041 raft_consensus.cc:2889] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:45.789965  1662 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.867141 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:45.869462 32121 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:45.829921  1275 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.878623 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:45.878849 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:45.883180 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.805979  1725 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.834067  1660 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.899482 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:45.833060  1564 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.835886  1612 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.835889  1576 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:45.838783  1717 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.858578 32655 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:45.862504  1754 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:45.895339  1581 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:45.920841 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:45.914739  1754 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.913214  1612 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.925298  1612 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:45.872330  1613 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:17:45.908269  1746 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.933604 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:45.907521  1433 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:45.901741  1759 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:45.904167 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:45.898005  1275 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.934734 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:45.909476  1703 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:45.930371  1670 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.017s	sys 0.049s
W20260709 12:17:46.012384 32010 leader_election.cc:336] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.167s (SENT)
I20260709 12:17:45.925302  1754 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:45.920984  1717 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:46.035166 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 343528us
I20260709 12:17:45.907953  1660 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.929255  1593 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.935885  1705 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:45.944247 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:46.039403 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101745us
W20260709 12:17:46.040099 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 105458us
I20260709 12:17:45.910614  1564 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:46.045682 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 373054us
I20260709 12:17:45.939030  1336 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.916803  1576 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:45.938884  1699 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:46.030954  1670 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.018s	sys 0.049s
I20260709 12:17:45.945387  1759 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:46.068110 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:45.909328  1372 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:45.936692  1742 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.121s
I20260709 12:17:45.910145  1730 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.075973 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:46.038513  1713 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:46.110275 31869 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 5, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 5 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:46.058495  1433 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:46.117331 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.056891  1581 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.944393  1746 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.055056  1705 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:46.055235  1300 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:17:46.065532  1336 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.076804  1703 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.076108 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:46.109050  1728 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:46.160297 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.120534  1750 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:46.138188  1742 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:46.187812 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:46.120723  1220 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:46.077606  1759 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.066756  1593 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:46.136582 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:46.192239 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 313590us
W20260709 12:17:46.194566 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 116672us
I20260709 12:17:46.142046  1705 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:46.223069 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:46.142083  1599 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.228222 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:46.113301  1651 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 12:17:46.173280  1336 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.119252  1699 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:17:46.174295  1741 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:17:46.246968  1274 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:46.247701  1764 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 12:17:46.247514  1274 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.248975 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1716) took 1173 ms. Trace:
W20260709 12:17:46.249748  1732 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.001s
I20260709 12:17:46.249965  1220 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:46.250101 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.227066  1272 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:46.250569  1220 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.251006 32039 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:46.251631 32044 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:46.252615  1220 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:46.253278  1220 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:46.253332 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.254002 32041 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:46.140724  1713 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:46.255488 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.256016  1713 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:46.256632  1713 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:46.195006  1572 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.001s
I20260709 12:17:46.257457 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.258083 31968 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:46.251009  1272 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:46.259030 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:46.260223 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.249743 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.260869 31969 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:46.259198  1272 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.262434  1272 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.262712  1717 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:46.188923  1703 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:46.194658  1728 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.264024  1728 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.264845  1703 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:46.265615  1703 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:46.265944  1703 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:46.219790 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 164013us
I20260709 12:17:46.252550 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:46.266685 32045 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.238566 32655 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:46.268117  1660 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:46.268421 32040 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:46.238112  1772 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:46.141804  1581 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:46.269029  1596 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.000s
I20260709 12:17:46.269181  1772 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.269627  1772 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.269438  1596 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:46.240716  1657 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.270654  1581 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:46.271237  1275 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:46.273207  1576 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:46.240424  1381 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.256927 31964 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:46.253772  1220 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.192627  1730 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:46.247104  1756 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
W20260709 12:17:46.282279 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.278784 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:46.239306 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 295105us
W20260709 12:17:46.283443 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 227226us
I20260709 12:17:46.287377 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.289801 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:46.289975 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:46.290800 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.291567 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.288100 31966 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:46.292454 31969 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:46.293644 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:46.274739  1286 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 12:17:46.296098 32121 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.242862  1633 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.238032  1750 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:46.240010  1780 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.299492 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:46.249963  1274 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:46.302577 32117 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:46.281971  1300 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.316877 32119 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [4.2-6.3]
W20260709 12:17:46.384271 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:46.393164 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53975.4us
I20260709 12:17:46.399806 32047 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:46.386300 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.385815 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:46.407073 32121 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:46.385046 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:46.283697  1651 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:46.293195  1758 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
I20260709 12:17:46.401373 32044 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:46.410848 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.300041  1713 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.297729  1779 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.279340  1331 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 12:17:46.311267  1275 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:46.386240  1274 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.432369 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 415) took 1364 ms. Trace:
I20260709 12:17:46.427264 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:46.427155 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.434192 32039 raft_consensus.cc:2368] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:46.310619 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.299754  1660 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.432871 32658 rpcz_store.cc:276] 0709 12:17:45.068183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.074425 (+  6242us) service_pool.cc:224] Handling call
0709 12:17:45.092927 (+ 18502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.093418 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:45.093444 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.093888 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.692735 (+598847us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:45.692775 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.692844 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.432113 (+739269us) raft_consensus.cc:1685] finished
0709 12:17:46.432134 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.432295 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5079,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":585226,"threads_started":1,"wal-append.queue_time_us":785195}
I20260709 12:17:46.312322  1286 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.436528 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:46.437552 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:46.433929 32047 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:46.296850  1787 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 12:17:46.316444  1381 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:17:46.411587  1300 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.430680 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:46.425881 31964 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:46.467622 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:46.467429 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:46.468142 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.469659 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:46.429670  1419 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.001s
I20260709 12:17:46.417776 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:46.423121  1713 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.442440 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:46.316558  1750 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:46.432188 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:46.317935  1780 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.470625 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:46.481802 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:17:46.902781 32660 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:46.679613 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 249183us
I20260709 12:17:46.908496 31963 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:46.906108 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:46.917232 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.918434 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.902827 31969 raft_consensus.cc:606] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:46.920431 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:46.249316 32115 rpcz_store.cc:276] 0709 12:17:45.075503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.093333 (+ 17830us) service_pool.cc:224] Handling call
0709 12:17:45.267349 (+174016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.349745 (+ 82396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.352562 (+  2817us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.384609 (+ 32047us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.754153 (+369544us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:45.754211 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.754299 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.248768 (+494469us) raft_consensus.cc:1685] finished
0709 12:17:46.248785 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.248918 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30979,"raft.queue_time_us":38839,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5358,"thread_start_us":334730,"threads_started":1,"wal-append.queue_time_us":518376}
I20260709 12:17:46.919275 31969 raft_consensus.cc:614] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:46.907085 31970 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:46.922349 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.922829 32039 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:46.923146 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:46.924057 32114 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:46.925487 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:46.926265 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.926715 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:46.910339  1754 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.440676  1274 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.938205  1754 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.495944  1220 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:46.939296  1657 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.669s	user 0.007s	sys 0.001s
I20260709 12:17:46.940004 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.940919 31969 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:17:46.935880 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.936666 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:46.937158 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 515437us
I20260709 12:17:46.941911 32043 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:46.938519 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.940138  1274 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:46.439318  1596 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:46.467869  1564 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:46.941533 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.937386 31970 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:46.935734 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.439327 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:47.031930 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:47.032820 31970 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:47.032958  1785 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.033214 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:47.033545  1785 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.943243 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:47.034636 31965 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.443222  1581 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.035606  1785 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:47.036227  1789 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.498s	user 0.001s	sys 0.001s
I20260709 12:17:47.036017  1581 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.037436 32119 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:46.431910  1779 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.018932 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:46.428680  1669 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.507s
I20260709 12:17:47.039706  1779 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.040429  1800 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:47.043922  1750 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.045727  1300 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.047348  1286 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:47.048327  1633 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.750s	user 0.005s	sys 0.002s
I20260709 12:17:47.049031  1581 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:47.053747  1272 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.791s	user 0.007s	sys 0.000s
I20260709 12:17:47.055142  1754 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:47.055272 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:47.056298  1705 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.816s	user 0.007s	sys 0.000s
I20260709 12:17:47.019264 31963 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:47.056682  1705 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.965646 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:46.430222  1651 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.057871 32047 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:46.431434  1275 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.943423  1672 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.697381  1746 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:46.937640 32655 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:47.060182  1275 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.019268 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:47.060541 32047 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.495692  1782 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.659s
I20260709 12:17:47.061959 32047 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:47.019737 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:46.973304 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.003s	sys 0.008s
W20260709 12:17:47.088629 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.004s	sys 0.008s
W20260709 12:17:46.934679 32084 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.003s	sys 0.009s
I20260709 12:17:46.942845 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:47.091828 32084 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.003s	sys 0.009s
W20260709 12:17:47.092334 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 812269us
I20260709 12:17:46.941267 32114 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.977671 32653 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:47.032003 32652 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:17:47.034682 31966 raft_consensus.cc:2376] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6. Re-sending same reply.
I20260709 12:17:47.039628  1806 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:17:47.142690 32042 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.003s	sys 0.007s
W20260709 12:17:47.143244 32042 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.003s	sys 0.007s
I20260709 12:17:47.056805 32039 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:46.922387  1713 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.957077 32112 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:17:47.146065 32045 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.009s	sys 0.003s
I20260709 12:17:46.927069  1419 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:47.049074  1780 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.018821 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:47.034101 31970 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:47.067435  1651 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:46.919623  1660 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:46.950902 32660 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:47.099922 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.591873 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:46.948714 32044 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:17:47.146435 32045 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.009s	sys 0.003s
I20260709 12:17:47.148317 31967 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:46.439453  1738 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.554s
W20260709 12:17:47.043951  1759 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.815s	user 0.007s	sys 0.001s
I20260709 12:17:47.153043 31964 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:47.157045 31968 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.023s	sys 0.026s
I20260709 12:17:46.919685  1763 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:47.145262  1669 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.160408 31969 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:47.031961  1783 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.189s
W20260709 12:17:47.348549 31968 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.035s	sys 0.052s
W20260709 12:17:47.349355 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.202862 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:46.936582 32654 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:47.350653 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 257018us
I20260709 12:17:47.350930 31970 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:17:47.250845 32120 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:47.350142 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:47.353271 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 410470us
I20260709 12:17:47.355966 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:47.145571  1806 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.150565  1800 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.359951  1651 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:47.360859  1596 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.149654  1779 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:47.353549 32624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.008s	sys 0.008s
W20260709 12:17:47.362628 32624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.008s	sys 0.008s
I20260709 12:17:47.363698 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:47.364565 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 398014us
I20260709 12:17:47.351802 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1473) took 1103 ms. Trace:
I20260709 12:17:47.360956  1787 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:47.360097 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:47.363094 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:47.368386 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:47.290539  1797 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
W20260709 12:17:47.324364  1798 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
W20260709 12:17:47.347280  1784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.008s	sys 0.009s
W20260709 12:17:47.357969  1772 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.083s	user 0.006s	sys 0.006s
I20260709 12:17:47.196178  1780 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.347654  1664 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.350546  1763 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:47.368578 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:47.357533  1731 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.362527  1806 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.361974  1564 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.357227  1672 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.398276 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:47.250200  1633 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.361654  1596 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.354447  1669 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:47.375054  1784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.008s	sys 0.009s
I20260709 12:17:47.361562  1272 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:17:47.356819  1735 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:17:47.378836  1651 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:47.401163 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.379000 32115 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:47.359071  1759 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.375789 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 414) took 1055 ms. Trace:
I20260709 12:17:47.398011  1274 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:47.407747 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:47.416441 32039 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:47.397920  1738 consensus_queue.cc:1243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 5.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:47.373162  1779 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:17:47.369891  1800 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.428684  1783 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:47.429657 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.425455 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.374028  1786 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.417081 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.401878  1672 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:47.458926  1790 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 12:17:47.460166  1672 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.415597  1651 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.417814  1596 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.422151 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:47.404300  1664 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:47.395754  1731 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.424022  1669 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:47.399729  1300 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:47.363416  1791 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.539s
I20260709 12:17:47.401670  1286 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.396091  1804 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.476294  1300 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:47.404105  1705 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.414570  1784 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.505491 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:47.471987  1581 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:47.466958 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44862.8us
I20260709 12:17:47.375250  1787 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:47.435585  1728 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.171s	user 0.005s	sys 0.006s
W20260709 12:17:47.443307  1765 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.322s	user 0.007s	sys 0.014s
I20260709 12:17:47.425158  1754 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.399124  1780 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.419724 32654 rpcz_store.cc:276] 0709 12:17:46.314774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.317200 (+  2426us) service_pool.cc:224] Handling call
0709 12:17:47.358688 (+1041488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:47.504751  1750 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.404999  1633 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.439739  1274 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:47.488356  1419 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:47.429070  1783 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.493680 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1568) took 1196 ms. Trace:
W20260709 12:17:47.496027  1730 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.213s	user 0.010s	sys 0.002s
I20260709 12:17:47.462520  1275 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:17:47.469527  1651 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.434881  1763 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.396142  1772 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.431783  1742 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:17:47.462837 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1572) took 1155 ms. Trace:
I20260709 12:17:47.414780  1811 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:47.471808 32038 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:47.504276  1786 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:47.518754 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 156534us
I20260709 12:17:47.484143 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.519073  1286 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:17:47.474301  1664 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.507745  1300 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.523352  1759 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.515201  1660 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:47.508747  1581 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.507516  1735 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.530499 32112 rpcz_store.cc:276] 0709 12:17:46.297680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.297954 (+   274us) service_pool.cc:224] Handling call
0709 12:17:46.298229 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:47.146149 (+847920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.493390 (+347241us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54001482d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:47.493615 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514380160}
I20260709 12:17:47.532298 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 168) took 1099 ms. Trace:
W20260709 12:17:47.537407 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 164051us
W20260709 12:17:47.540082 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 132413us
I20260709 12:17:47.527657  1529 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:47.544925 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.531872  1783 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.520202 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1477) took 1234 ms. Trace:
W20260709 12:17:47.550714 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 173063us
I20260709 12:17:47.518838  1728 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.521117  1754 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.521181  1784 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.529398  1750 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.592032 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:47.541409 32115 rpcz_store.cc:276] 0709 12:17:46.306950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.148094 (+841144us) service_pool.cc:224] Handling call
0709 12:17:47.276848 (+128754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.419054 (+142206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.437492 (+ 18438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:47.620863 32655 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:47.621340  1735 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.621851  1771 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.621966  1735 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.587985 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.543331  1811 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.619093 32047 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:47.567179 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:47.557080 32113 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:47.622411  1771 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.549149  1564 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:47.619781 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:47.555047  1300 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.623217  1581 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.676854  1651 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.592142  1807 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.589190  1772 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.623658 32038 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:47.553462  1780 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.620096 32041 rpcz_store.cc:276] 0709 12:17:46.285940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.286055 (+   115us) service_pool.cc:224] Handling call
0709 12:17:46.286349 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.286583 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:46.471967 (+185384us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.519720 (+1047753us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.519849 (+   129us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:47.519939 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.520006 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.520147 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18494,"prepare.queue_time_us":208337,"raft.queue_time_us":84296,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":18636160,"thread_start_us":750944,"threads_started":3,"wal-append.queue_time_us":995364}
I20260709 12:17:47.554409  1669 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:47.656111 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:47.690577 32047 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.658691 32654 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:47.646804  1813 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:17:47.692780  1813 consensus_queue.cc:1243] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:47.692852  1804 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.664851  1784 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.665370  1731 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.694856  1664 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.697166  1725 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.698432  1669 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.667557  1750 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:47.668071 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:47.670137 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:47.576687 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:47.685302  1286 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.675545 32658 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:17:47.675788 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1733) took 1265 ms. Trace:
I20260709 12:17:47.710976 32112 rpcz_store.cc:276] 0709 12:17:46.410876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.669499 (+1258623us) service_pool.cc:224] Handling call
0709 12:17:47.675282 (+  5783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.675473 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.675496 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.675544 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:47.675574 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.675624 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.675735 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:47.676118  1564 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.631914  1738 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.591051  1632 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:47.589991  1787 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.654361 31968 rpcz_store.cc:276] 0709 12:17:46.433129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.434326 (+  1197us) service_pool.cc:224] Handling call
0709 12:17:46.434685 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.434935 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:47.349127 (+914192us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.531976 (+182849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.532027 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:47.532073 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.532142 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.532246 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23126,"prepare.queue_time_us":999389,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":31165,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":1066892,"threads_started":3,"wal-append.queue_time_us":122452}
W20260709 12:17:47.685235 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 65545.2us
I20260709 12:17:47.716068 32047 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:47.588639  1633 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.631973  1732 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.723593  1732 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:47.672945  1713 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.002s
I20260709 12:17:47.725745  1732 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.725683  1713 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.727754  1713 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:47.654690  1811 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:47.710784  1286 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:47.729909  1286 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:47.730643 32044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1495) took 1326 ms (client timeout 1667 ms). Trace:
W20260709 12:17:47.731072 32044 rpcz_store.cc:269] 0709 12:17:46.404695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.410624 (+  5929us) service_pool.cc:224] Handling call
0709 12:17:47.730480 (+1319856us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001a59d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:47.730588 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235566464}
I20260709 12:17:47.590231  1274 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.685170  1807 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.732273 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.732854 32044 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:47.732920 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:47.685860  1660 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:47.734088  1660 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.734488  1660 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.689230 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.691347 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:47.738716 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47437.3us
I20260709 12:17:47.739401 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:47.739828  1804 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:47.740579 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:47.619393  1529 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:47.691849 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:47.699043  1336 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.424s	user 0.007s	sys 0.004s
I20260709 12:17:47.700500 32043 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:47.700639 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46441.4us
W20260709 12:17:47.671420 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 104344us
I20260709 12:17:47.623646  1806 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.730500  1731 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:17:47.689270  1786 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.730139  1783 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.704711  1771 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.705390  1576 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.706013  1651 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:47.707152  1728 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:47.708993  1770 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.519s	user 0.002s	sys 0.022s
I20260709 12:17:47.709213  1662 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:47.710371 32113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1564) took 1417 ms (client timeout 1812 ms). Trace:
I20260709 12:17:47.788587 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:47.733692  1807 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:47.793134 32113 rpcz_store.cc:269] 0709 12:17:46.292974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.293429 (+   455us) service_pool.cc:224] Handling call
0709 12:17:47.710198 (+1416769us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540043add4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:47.710299 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2863909632}
I20260709 12:17:47.755059 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1573) took 1447 ms. Trace:
I20260709 12:17:47.365603 32042 rpcz_store.cc:276] 0709 12:17:46.248205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.251755 (+  3550us) service_pool.cc:224] Handling call
0709 12:17:46.252039 (+   284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:46.281814 (+ 29775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:46.281845 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:46.282467 (+   622us) log.cc:844] Serialized 52 byte log entry
0709 12:17:46.465687 (+183220us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:47.143699 (+678012us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.351356 (+207657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.351465 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.351543 (+    78us) raft_consensus.cc:1685] finished
0709 12:17:47.351564 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.351729 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71584,"prepare.queue_time_us":886612,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":32917,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":15387,"thread_start_us":747230,"threads_started":2,"wal-append.queue_time_us":858913}
I20260709 12:17:47.742165 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:47.794387 32114 rpcz_store.cc:276] 0709 12:17:46.307175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.754061 (+1446886us) service_pool.cc:224] Handling call
0709 12:17:47.754453 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.754676 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.754711 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.754764 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:47.754809 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.754875 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.754996 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:47.795172  1787 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:47.795068 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52961us
I20260709 12:17:47.797688 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:47.788201 32038 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:47.806253 32117 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:47.807508 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:47.808573 32117 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:47.798070 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:47.813032 32047 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:47.814847 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:47.837000 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:47.838958 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:47.839455 32658 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:17:47.840111 32114 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:47.842389  1787 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:47.844375 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46771.3us
I20260709 12:17:47.845239 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:47.845713 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:47.845849 32113 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:47.731055  1754 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:47.722967  1632 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:47.749190  1599 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.847131 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:47.792920  1576 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.849334 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1453) took 3423 ms. Trace:
I20260709 12:17:47.793299  1781 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.740577  1804 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.742476  1718 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.836172  1750 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.850051 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:47.846482 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 74112.9us
I20260709 12:17:47.855160 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:47.793408  1602 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.856839 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:47.814126  1774 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:47.845281 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:47.835395  1783 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.843758  1697 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.797308  1336 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.867636 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:47.742116  1529 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.847342  1633 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:47.837999  1662 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:17:47.847126  1807 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:17:47.846726  1824 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:17:47.850971 32660 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:47.923035 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1347) took 4201 ms (client timeout 1998 ms). Trace:
I20260709 12:17:48.037145 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:47.859294 32046 rpcz_store.cc:276] 0709 12:17:44.424563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.730460 (+305897us) service_pool.cc:224] Handling call
0709 12:17:44.730879 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.731330 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.731371 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.482556 (+751185us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b2c0005c228. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:45.482950 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.867661 (+384711us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:45.867713 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.867799 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.708383 (+1840584us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:47.847431 (+139048us) raft_consensus.cc:1685] finished
0709 12:17:47.847450 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.847569 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6575,"spinlock_wait_cycles":9835008,"thread_start_us":384298,"threads_started":1,"wal-append.queue_time_us":589142}
W20260709 12:17:48.037235 31967 rpcz_store.cc:269] 0709 12:17:43.721637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.731927 (+ 10290us) service_pool.cc:224] Handling call
0709 12:17:47.922779 (+4190852us) spinlock_profiling.cc:243] Waited 3.28 s on lock 0x7b5400314a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:47.922932 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7550898432}
I20260709 12:17:48.037961 32113 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.924817 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:47.868992 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:47.870328 32652 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:17:47.939038 32043 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.028947 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:47.866472 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 200) took 1554 ms. Trace:
W20260709 12:17:48.040635 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 185547us
W20260709 12:17:48.041769 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 196547us
I20260709 12:17:48.041165 32045 rpcz_store.cc:276] 0709 12:17:46.311735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.311856 (+   121us) service_pool.cc:224] Handling call
0709 12:17:46.312135 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.312367 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:47.741189 (+1428822us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b18003f7840. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf
0709 12:17:47.741301 (+   112us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.865886 (+124585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.866063 (+   177us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:47.866121 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.866210 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.866399 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":937513,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":1184,"raft.queue_time_us":108,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":54592,"spinlock_wait_cycles":12239232,"thread_start_us":1078736,"threads_started":2}
I20260709 12:17:48.042769 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.044262 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.046034 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:47.868521 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.048094 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:48.054157 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:48.027456  1767 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.832s	user 0.001s	sys 0.023s
I20260709 12:17:47.863411  1576 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.730898  1738 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.062139 32660 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:47.942950  1331 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.040622  1806 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.052048  1613 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:48.047508  1780 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:47.919577  1772 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.036978  1774 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.862440  1735 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.043788  1816 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:47.865181  1731 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:47.944829  1593 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.870560  1771 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.063352 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:48.063503 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:47.865664  1813 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.059163 32113 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:48.040323  1664 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.068548  1725 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.006090  1632 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:48.097607 32119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1574) took 1790 ms (client timeout 2176 ms). Trace:
I20260709 12:17:47.865512  1804 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:47.946338  1669 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:47.879577  1714 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:48.107750  1725 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.870137  1779 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:48.047320  1746 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:48.056746  1672 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:47.864194  1581 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.091965  1274 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:48.063167  1718 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.863629  1599 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.924396  1286 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:47.881156  1781 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.063587  1817 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.044178  1602 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.045058  1750 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.079180 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.044229  1633 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.082558 32113 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [1.3-6.5]
I20260709 12:17:48.071393  1772 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.071527  1774 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:48.091041  1757 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 2.319s	user 0.005s	sys 0.018s
I20260709 12:17:48.070528  1331 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:47.858748  1660 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:48.098140 32119 rpcz_store.cc:269] 0709 12:17:46.307390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.423843 (+116453us) service_pool.cc:224] Handling call
0709 12:17:48.097369 (+1673526us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002fab54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:48.097518 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311384960}
I20260709 12:17:48.080992  1780 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.143698 31963 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
W20260709 12:17:48.101377  1828 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:17:48.118185  1836 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:48.108389  1754 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:48.096287  1664 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.122061  1725 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.113837  1746 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.115325  1779 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.097012  1813 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.124596  1823 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.126011  1735 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.130105 32116 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:48.130538  1274 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:48.130460  1576 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.130935 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:48.131979 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.162005 32658 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.162146 32660 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:48.071777  1811 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.108234  1714 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:48.134227 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49316.8us
I20260709 12:17:48.110672  1804 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.076815  1731 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.081813  1771 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.139482 32114 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:48.140528 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61452.3us
I20260709 12:17:48.142874 32113 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:48.046506  1697 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.136452  1817 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.151633  1725 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.141824  1662 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.177649  1633 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.179251  1771 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.175287  1817 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.180047  1781 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.181879 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.183426  1714 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.183697  1576 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.184350 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.184852  1602 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.185613  1738 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.185456  1725 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.186889  1331 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.187265  1286 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.187805  1725 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.188098  1599 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.165591 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:48.189082 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.149119  1581 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:48.151661  1746 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:48.153337  1651 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.171218 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:48.171221 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:48.175108 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.160261  1336 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.176128 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.196230 32113 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:48.191677 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.192392 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.192788 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.151664  1836 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.153064  1754 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.203369 32658 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.205583 32114 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:48.226688 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.228633 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.229024 32119 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:48.229494 32045 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:48.236202 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.236882 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.235945 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.237488 32044 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.232718 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:48.238226 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.239261 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.240324 32045 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:48.162395  1823 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.175580  1718 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:17:48.246626 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:48.320742 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.259603 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.307060 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.274940 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.323791 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.319535  1763 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.325035 32119 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:48.325157 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.281522 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.199579  1725 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.259318 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.313289 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 192) took 2513 ms. Trace:
I20260709 12:17:48.300595  1529 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:48.344588 32040 rpcz_store.cc:276] 0709 12:17:45.800293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.800470 (+   177us) service_pool.cc:224] Handling call
0709 12:17:45.800735 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.268742 (+468007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.313016 (+2044274us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b5400033ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:48.313222 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074860928}
I20260709 12:17:48.228894  1672 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.235899  1660 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:48.346589 32113 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:48.300747  1834 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:48.229480  1816 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:48.245978  1613 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:48.293207  1830 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:17:48.399329 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.420900 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:48.399847 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.422698 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:48.423486 32117 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:48.404995 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.301880  1813 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:48.299285  1768 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 2.185s	user 0.011s	sys 0.017s
I20260709 12:17:48.343489  1787 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:48.270356  1274 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.258905 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:48.226802  1662 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.427114 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.228634  1651 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.352849  1750 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:48.414656  1785 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:48.456627  1785 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:48.463320 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.464002 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.464695 32045 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.466328 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.341215  1717 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.258653  1837 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:48.433158 32118 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:48.346814  1564 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:48.433831 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.473935 32119 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:48.342710  1772 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.434154 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.429179 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.430735  1331 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.436419  1811 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.480069 32623 leader_election.cc:400] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:48.396607  1672 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:48.438750 32120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1725) took 2634 ms (client timeout 1514 ms). Trace:
I20260709 12:17:48.439018  1754 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.430440 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.494729 31967 raft_consensus.cc:2368] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:48.432914  1662 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:48.430923  1787 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.446899  1836 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.235389  1703 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.418507  1816 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.448354 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.448635 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:48.475414  1750 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.454692  1697 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.455662 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.471133 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.458016  1785 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.469213 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.516001 32039 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:48.484122  1837 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:48.474550 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.347218  1725 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.477507 31969 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.518141 31965 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.482517 32623 leader_election.cc:403] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:48.498780  1787 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:48.483567 32120 rpcz_store.cc:269] 0709 12:17:45.801367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.801695 (+   328us) service_pool.cc:224] Handling call
0709 12:17:48.432548 (+2630853us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540012bbd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:48.432691 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3312004480}
I20260709 12:17:48.447146 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.487318 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:48.488278  1651 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:48.538750 31966 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:48.483325  1331 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.539172  1672 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.493484  1774 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.496129 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.519523  1725 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.541740 32659 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.497115 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:48.542924 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45897.3us
I20260709 12:17:48.427742  1613 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.503630 32044 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:48.477211  1717 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.450616 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.509096 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.510018 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.558858 31967 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:48.510942 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:48.471787 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:48.559995 32657 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:48.513075 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:48.513082 32038 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.513767 32045 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:48.543016  1774 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.345604  1699 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:17:48.498934  1662 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.499675  1823 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.522758 32117 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:48.524845 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.573580 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:48.574527 32117 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:48.523449 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.527631 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.519769 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.529201 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.529646  1836 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:48.531929 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.584453 32041 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.535110 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.536834 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:48.515069  1750 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.540998 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.548071 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:48.578550 32120 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:48.585326 31966 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:48.548511 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:48.586133 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:48.352458  1664 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.559947 32119 raft_consensus.cc:2376] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4. Re-sending same reply.
I20260709 12:17:48.594465 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.594544 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.575443 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.527441 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.549177  1785 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.527415  1559 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5.
I20260709 12:17:48.562402  1717 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.577104 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.622898 32044 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:48.577472 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:48.579592 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50479.2us
I20260709 12:17:48.623903 31964 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.582827 31965 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:48.583532 32657 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:48.553701  1672 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.533464  1800 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:48.585306 32040 raft_consensus.cc:2410] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:17:48.590775 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71058.1us
I20260709 12:17:48.631552 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.631736 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.632650 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.556964 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:48.584290  1836 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:48.606920  1669 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:48.598142  1804 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.599668  1817 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.608119 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.640460 31966 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:48.641322 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.614446  1811 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.609386 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.612884 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1734) took 1523 ms. Trace:
I20260709 12:17:48.614166 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.642472 32116 rpcz_store.cc:276] 0709 12:17:47.090048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.701110 (+611062us) service_pool.cc:224] Handling call
0709 12:17:47.705646 (+  4536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.160205 (+454559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.612633 (+452428us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:48.612828 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975397760}
I20260709 12:17:48.617928 32120 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.625666 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.644060 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.613922  1750 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:48.628038 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.645779 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.628870 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.633739 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:48.593062 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56320.1us
I20260709 12:17:48.594678  1823 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.651525 31964 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:17:48.634641 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 77421.1us
I20260709 12:17:48.586840  1780 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.593981  1699 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:48.637287 32041 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 2.
I20260709 12:17:48.596560 32117 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:48.647109 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.657486 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.657963 31966 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:48.658933 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.659844 32117 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:48.608970  1774 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.647040 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.662446 31965 raft_consensus.cc:2376] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5. Re-sending same reply.
I20260709 12:17:48.663683 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.664408 31965 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.665406 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.653465 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.667763 31969 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:17:48.667798 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.668879 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.675894 32120 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:48.670627 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.639240  1664 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.639860  1836 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.654488  1660 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:48.681816 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.692598 31969 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:48.682523 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.639420  1718 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:48.642190  1669 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:48.722688  1669 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.662300 32656 raft_consensus.cc:2410] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:48.693981 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.727699 32043 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:17:48.727885  1718 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:48.728487 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.729225 32043 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:48.728761  1718 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.729962 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.730986 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.731549 31966 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:48.736300  1746 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.003s
I20260709 12:17:48.736763  1746 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.743208  1731 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.744693 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:48.744874  1662 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.748601 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1575) took 2441 ms. Trace:
I20260709 12:17:48.666208  1754 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:48.749279  1754 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.749881  1754 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.754598  1669 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.749722  1787 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.644942  1811 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.700923 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.701562 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.765358 32116 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:48.641270  1837 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:48.710408  1804 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.711364 32041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:48.715967  1613 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.683898 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.661998  1774 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.749028 32118 rpcz_store.cc:276] 0709 12:17:46.307622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.348263 (+1040641us) service_pool.cc:224] Handling call
0709 12:17:48.432740 (+1084477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.472881 (+ 40141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.748401 (+275520us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540012bbd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:48.748529 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440846592}
I20260709 12:17:48.755685 31966 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.776108 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.759330 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.777011 32038 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:48.702188 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.777197 31966 raft_consensus.cc:2368] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:48.778214 32112 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:48.695761 32040 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:48.779069 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.779618 32040 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:48.780392 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.780622 32040 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:48.780956 31966 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.781832 32040 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 12:17:48.746171 32046 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:17:48.654654  1823 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.768198 32041 raft_consensus.cc:606] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.782967 32041 raft_consensus.cc:614] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:48.643940  1800 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:17:48.771361 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87538.1us
I20260709 12:17:48.721536  1660 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:48.772069 32659 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:48.774569 32044 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:48.800503 32043 raft_consensus.cc:2368] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5.
I20260709 12:17:48.782014 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.692169  1672 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.785776 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:48.697862  1714 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.797036 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.813139 32038 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:48.814174 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.763803  1791 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:48.800982 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.767119  1811 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.801471 31966 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.808970 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:48.816738 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 417) took 1250 ms. Trace:
I20260709 12:17:48.820012 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:48.817111 32652 rpcz_store.cc:276] 0709 12:17:47.566293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.592166 (+ 25873us) service_pool.cc:224] Handling call
0709 12:17:47.681706 (+ 89540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.039739 (+358033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.816506 (+776767us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:48.816686 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378097536}
I20260709 12:17:48.783866  1774 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.796581  1823 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.795025  1750 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:48.799276 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:48.829252 32040 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 12:17:48.834282 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.819509  1672 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:48.874840 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 507) took 2455 ms. Trace:
I20260709 12:17:48.833863 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.814731  1714 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.885382 31966 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:17:48.878192 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:48.917348 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.892019 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:48.950069 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.891534 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1586) took 1079 ms. Trace:
I20260709 12:17:48.951918 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.951895 32114 rpcz_store.cc:276] 0709 12:17:47.811705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.846906 (+ 35201us) service_pool.cc:224] Handling call
0709 12:17:48.891349 (+1044443us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:48.891466 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649621376}
I20260709 12:17:48.827020  1713 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:48.879802 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.008780 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.009109 32112 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:49.009220 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 172) took 1142 ms. Trace:
I20260709 12:17:48.849530 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:48.877756 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 243988us
I20260709 12:17:49.051151 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:48.953199 31966 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:49.053007 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:48.979744 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:49.079232 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:48.887734 32653 rpcz_store.cc:276] 0709 12:17:46.419282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.422771 (+  3489us) service_pool.cc:224] Handling call
0709 12:17:46.439479 (+ 16708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.116619 (+677140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.874530 (+1757911us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400544054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:48.874750 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152463744}
W20260709 12:17:48.888223 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71180.6us
I20260709 12:17:48.849100 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.019006  1662 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.998040  1714 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.948402  1816 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:48.826979  1791 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.017963  1672 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:48.882081 32651 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:48.886226  1746 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:49.032478 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.132104 32046 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:49.132993 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:49.133520 32046 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.877426  1750 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.009829 31970 rpcz_store.cc:276] 0709 12:17:47.866508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.866690 (+   182us) service_pool.cc:224] Handling call
0709 12:17:47.866983 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.072254 (+205271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.072278 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:48.072766 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:17:48.142965 (+ 70199us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:48.143010 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.890701 (+747691us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54001e96d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:48.890766 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:48.890820 (+    54us) raft_consensus.cc:1685] finished
0709 12:17:48.890831 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.009139 (+118308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3896,"spinlock_wait_cycles":471201536,"thread_start_us":69881,"threads_started":1,"wal-append.queue_time_us":100989}
W20260709 12:17:49.080433 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 100777us
I20260709 12:17:49.087651 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:49.016122  1811 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:49.094527 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.018406  1845 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:17:49.014189  1717 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.125352 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:49.153985 32038 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:49.125562 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74526.7us
I20260709 12:17:48.879490 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:49.089509 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:49.155941  1791 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:49.156149 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 66703.5us
I20260709 12:17:49.156806 31964 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:17:49.157333 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:48.833329  1725 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.161609 32118 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:49.162441  1725 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:49.163008  1274 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.079818  1718 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:49.139952 31967 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:49.125918 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.141388 31970 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:49.131629  1800 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:49.091138 32655 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:49.125758  1660 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.136445 32044 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:49.146914  1849 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:49.139370  1750 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.128723  1850 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:49.164760 32112 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:49.186904 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:49.164990  1651 raft_consensus.cc:2764] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:49.187451 32084 leader_election.cc:400] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
W20260709 12:17:49.177321  1846 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:17:49.173687  1669 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:49.187860  1800 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:49.191934  1529 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:17:49.192123  1848 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 12:17:49.194587 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:49.193252 31967 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:49.195596 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:49.198992 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:49.200081 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:49.203907 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:49.211153 32044 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:49.212148 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.223176 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:49.184988  1817 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:49.220530 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.222229 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:49.222468 32038 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:49.220544 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:49.225433 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.206377 32115 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 4: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:49.226948 32038 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:49.227836 31972 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:49.179688  1718 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.205051  1728 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:49.233367 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:49.229431 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:49.195320  1529 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:49.197026  1787 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:49.205302  1791 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.231860 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.235921 31972 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:49.307175 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:17:49.329694 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.309582 32120 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:49.389025 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1747) took 1109 ms. Trace:
I20260709 12:17:49.234766  1780 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.291975 32626 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:49.390443  1780 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.231186  1651 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.391126 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.391438  1651 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:49.391587 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 99710.1us
I20260709 12:17:49.230818 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:49.243039  1718 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:49.392917 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.390950  1850 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.391944  1651 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.394591  1783 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:49.394968  1783 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:49.395815  1783 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.396791  1783 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:49.399359  1853 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.386s	user 0.002s	sys 0.000s
W20260709 12:17:49.399597  1763 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.053s	user 0.004s	sys 0.010s
I20260709 12:17:49.291356 31968 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:17:49.401423 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:49.394186 32041 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:49.401990 32044 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:49.389483 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1749) took 1082 ms. Trace:
I20260709 12:17:49.214661  1847 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:49.404909 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 175538us
I20260709 12:17:49.389500 32120 rpcz_store.cc:276] 0709 12:17:48.279451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.309106 (+1029655us) service_pool.cc:224] Handling call
0709 12:17:49.309396 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.388781 (+ 79385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.388960 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:49.391891 32039 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:49.260124 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.388833 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.409538 32046 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:49.407725 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:49.410275 31972 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:17:49.410684 31972 pending_rounds.cc:85] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 2
I20260709 12:17:49.410913 31972 pending_rounds.cc:107] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:17:49.411151 31972 raft_consensus.cc:2889] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:49.411209 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:49.403951 32044 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:17:49.253572  1754 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.002s
I20260709 12:17:49.225419  1713 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:49.234843  1725 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:49.412531  1836 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.702s	user 0.002s	sys 0.009s
W20260709 12:17:49.503794 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 176) took 1636 ms (client timeout 2178 ms). Trace:
I20260709 12:17:49.408442  1763 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:49.308182 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.236578 32114 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:49.308017 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:49.510098 31963 rpcz_store.cc:269] 0709 12:17:47.867724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.868222 (+   498us) service_pool.cc:224] Handling call
0709 12:17:49.503520 (+1635298us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b540034d854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:49.503714 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220537728}
I20260709 12:17:49.239580  1717 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:49.513165 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.263484 32116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:17:49.430871 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:49.518488 31963 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:49.519611 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1745) took 1253 ms. Trace:
I20260709 12:17:49.467355 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1736) took 1302 ms. Trace:
I20260709 12:17:49.247094 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.517057 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:49.412955 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.462536 31968 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:49.427932 32046 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:49.240269  1664 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.545s	user 0.006s	sys 0.002s
I20260709 12:17:49.520031 32114 rpcz_store.cc:276] 0709 12:17:48.266701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.235839 (+969138us) service_pool.cc:224] Handling call
0709 12:17:49.236176 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.519250 (+283074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.519558 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:49.556758 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.520807 32045 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:17:49.521522 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 90763.4us
I20260709 12:17:49.439785 32120 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:17:49.558916 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1601) took 1230 ms. Trace:
I20260709 12:17:49.486925  1747 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:49.403767 31967 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:49.525113 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1730) took 1045 ms. Trace:
I20260709 12:17:49.255738  1811 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:49.412413  1859 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.560642 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:49.562570 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.440229  1725 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:49.403127  1728 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.520330 32112 rpcz_store.cc:276] 0709 12:17:48.165049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.124869 (+959820us) service_pool.cc:224] Handling call
0709 12:17:49.164527 (+ 39658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.192718 (+ 28191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.467086 (+274368us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54004344d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:49.467268 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54308096}
I20260709 12:17:49.612632 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:49.540669 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.357831  1714 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:49.518615  1774 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 12:17:49.559774  1699 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.391148  1780 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.521184  1660 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:49.562479  1664 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.559656  1669 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:49.431550  1433 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 12:17:49.512571  1836 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.620478 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:49.551537 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:49.603369  1738 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.434801  1602 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.184s
I20260709 12:17:49.559548  1855 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:49.629875 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:49.630976 32112 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:49.609691  1725 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:49.619732  1823 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.727s	user 0.003s	sys 0.007s
I20260709 12:17:49.618209  1728 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.441349  1754 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.404839 32115 rpcz_store.cc:276] 0709 12:17:48.307508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.340690 (+1033182us) service_pool.cc:224] Handling call
0709 12:17:49.360175 (+ 19485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.376027 (+ 15852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.377294 (+  1267us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:49.378012 (+   718us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:49.389205 (+ 11193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.389277 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.389421 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:49.503268  1529 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.647791 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1604) took 1200 ms. Trace:
I20260709 12:17:49.620604  1859 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:49.560935 31970 rpcz_store.cc:276] 0709 12:17:48.479323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.140644 (+661321us) service_pool.cc:224] Handling call
0709 12:17:49.140999 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.141254 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:49.289335 (+148081us) log.cc:844] Serialized 14 byte log entry
0709 12:17:49.524691 (+235356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:49.524755 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:49.524817 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.524912 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.525046 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7234,"thread_start_us":234880,"threads_started":1,"wal-append.queue_time_us":338593}
W20260709 12:17:49.611375 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50771.9us
I20260709 12:17:49.660182 32112 rpcz_store.cc:276] 0709 12:17:48.447254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.612999 (+1165745us) service_pool.cc:224] Handling call
0709 12:17:49.630746 (+ 17747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.647491 (+ 16745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.647725 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:49.663040 32119 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:49.629042  1669 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:49.665508 32084 leader_election.cc:400] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:49.250963  1800 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.665828 32084 leader_election.cc:403] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.560111 32120 rpcz_store.cc:276] 0709 12:17:48.328484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.439043 (+1110559us) service_pool.cc:224] Handling call
0709 12:17:49.439476 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.558521 (+119045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.558847 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:49.674212 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:49.620100  1774 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.625811  1699 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.633507  1811 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:17:49.682401 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:49.691112 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:49.670267  1669 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.668305  1855 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:49.695971 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.619889  1713 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:49.674288  1529 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:49.694082 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 89730.3us
I20260709 12:17:49.464633  1331 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:49.651629  1738 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.689380  1813 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:49.681274  1581 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6.
I20260709 12:17:49.693764 32117 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:49.684525  1837 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.675971  1763 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:49.780503  1868 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:17:49.652848  1823 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:49.781589  1864 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:17:49.782639  1867 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:17:49.783689  1866 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:17:49.784845  1863 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:17:49.787555  1774 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:49.795190  1714 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:49.795379  1754 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:49.801553  1850 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.727756 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:49.851393 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 123748us
I20260709 12:17:49.852108 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:49.696604  1274 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:49.707720 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:49.703171  1785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.018s	sys 0.000s
W20260709 12:17:49.853653  1785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.020s	sys 0.001s
W20260709 12:17:49.811480 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 455112us
I20260709 12:17:49.706377  1858 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:49.819216 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 183052us
W20260709 12:17:49.829481  1862 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:17:49.710773  1813 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:49.717972  1837 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:49.710544  1602 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
I20260709 12:17:49.809368  1779 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:49.816053  1873 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:49.855894 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:49.910773 31963 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:49.859870 32117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 187) took 1178 ms (client timeout 1546 ms). Trace:
W20260709 12:17:49.911580 32117 rpcz_store.cc:269] 0709 12:17:48.681521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.700668 (+ 19147us) service_pool.cc:224] Handling call
0709 12:17:49.859653 (+1158985us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54002fa654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:49.859809 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135837568}
I20260709 12:17:49.833768  1651 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.930500 32653 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:49.931356  1274 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:49.932269  1651 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:49.932344  1274 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:49.893034 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 185352us
I20260709 12:17:49.936304 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:49.936645 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:49.894703  1878 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:49.905557  1867 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:49.929756  1873 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:49.937676 32117 raft_consensus.cc:2368] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:49.965058 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:49.905987  1837 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:49.904968  1865 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:17:49.891126  1754 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:49.896291 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:49.977975 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 81811.5us
I20260709 12:17:49.978237 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1727) took 1499 ms. Trace:
I20260709 12:17:49.978647 31966 rpcz_store.cc:276] 0709 12:17:48.478525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.088885 (+610360us) service_pool.cc:224] Handling call
0709 12:17:49.089252 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:49.336393 (+247141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.336440 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:49.337283 (+   843us) log.cc:844] Serialized 52 byte log entry
0709 12:17:49.595615 (+258332us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:49.595654 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.595723 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.978016 (+382293us) raft_consensus.cc:1685] finished
0709 12:17:49.978037 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.978182 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":246625,"raft.queue_time_us":247888,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":5576,"thread_start_us":245476,"threads_started":1,"wal-append.queue_time_us":317220}
I20260709 12:17:49.948977 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.949645 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.939944  1632 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.903818  1703 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.965795 32117 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:49.938406 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:49.969450 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:49.971519 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1402) took 1492 ms. Trace:
W20260709 12:17:49.972235 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 106576us
I20260709 12:17:49.992069 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:49.975759 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:49.991622 31965 rpcz_store.cc:276] 0709 12:17:48.478880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.097077 (+618197us) service_pool.cc:224] Handling call
0709 12:17:49.120216 (+ 23139us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.344734 (+224518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.344771 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.345235 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.544559 (+199324us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:49.544599 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.544660 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.971253 (+426593us) raft_consensus.cc:1685] finished
0709 12:17:49.971285 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.971443 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":226322,"raft.queue_time_us":225110,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":199019,"threads_started":1,"wal-append.queue_time_us":305781}
W20260709 12:17:49.941195 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84691.4us
I20260709 12:17:49.849723  1855 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:49.965343  1874 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.983830 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:49.940012  1602 consensus_queue.cc:1243] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:49.939261  1836 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:49.900946  1581 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.993994 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:50.001956 32117 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.911458  1774 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:17:49.939836  1878 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:49.981164  1754 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.998327  1331 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:49.986055  1877 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.013631 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:49.911242  1763 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.027766 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.024344 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:49.999190  1874 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.996338  1717 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:50.011008  1750 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.815s	user 0.010s	sys 0.002s
I20260709 12:17:50.031139 32652 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:50.034441 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.035815 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:50.041823 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.042552 32045 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.002547 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:50.009963  1878 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.036841 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:50.048650 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54169.6us
I20260709 12:17:50.048910 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1733) took 1568 ms. Trace:
I20260709 12:17:50.038076 32043 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:50.049397 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.050725 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.016003  1754 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.051887 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.006549  1787 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:17:50.009419 32655 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:17:50.037261 32046 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:49.985276  1813 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.047992 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:50.036576 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.003252  1651 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:17:50.002130  1816 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:17:50.026144  1529 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.055001 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:49.994704  1336 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:17:50.031188  1331 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:50.051079 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:50.063352 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.004979  1804 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:17:50.048092 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:50.048398  1887 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.049365 31964 rpcz_store.cc:276] 0709 12:17:48.480317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.157704 (+677387us) service_pool.cc:224] Handling call
0709 12:17:49.158006 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.345959 (+187953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.345987 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.346436 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.633560 (+287124us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:49.633601 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.633667 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.048700 (+415033us) raft_consensus.cc:1685] finished
0709 12:17:50.048721 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.048858 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":187618,"raft.queue_time_us":188662,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":286733,"threads_started":1,"wal-append.queue_time_us":340401}
I20260709 12:17:50.074241 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:50.054896  1714 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:50.006844  1713 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:17:50.031118  1699 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.053577 32117 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:50.038612  1779 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:17:50.238965 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:50.047773 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:50.038625  1874 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:50.069574 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71691.6us
I20260709 12:17:50.062958 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.068071 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.075055 32115 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:50.072358 31968 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.058794 31966 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.075212 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:50.260291 32045 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:50.073403  1877 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:50.267803 32045 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:17:50.271106 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.086478  1823 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:50.045055  1717 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.054121  1811 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:50.242491  1785 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:50.081382 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:50.083472  1841 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.524s	user 0.007s	sys 0.017s
I20260709 12:17:50.240967 32117 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:17:50.061055  1651 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:50.253238 32039 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.286110  1713 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.304864  1713 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:50.306002  1725 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.643s	user 0.000s	sys 0.007s
I20260709 12:17:50.254855 32013 leader_election.cc:400] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:50.317201 32013 leader_election.cc:403] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:50.049722  1750 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.279757 31966 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:50.281351 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 213333us
I20260709 12:17:50.318728 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.241384  1787 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.244378  1813 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.290198  1779 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.076262 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.074733  1890 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.215s
I20260709 12:17:50.307574  1717 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.062963 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:50.241257  1714 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.319401  1895 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.238688  1887 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.080345  1331 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:50.357345 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 102634us
W20260709 12:17:50.357649 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 281441us
I20260709 12:17:50.316061  1823 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:50.307435  1713 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:50.371804 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 308885us
I20260709 12:17:50.288363  1774 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:17:50.366816 32039 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:50.385635 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.337327  1787 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:50.390004 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:50.352958  1785 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:50.395190  1699 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.395905  1892 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:50.338687  1725 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.396906 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.383893  1874 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.381592 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.400049 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1732) took 4004 ms. Trace:
I20260709 12:17:50.386547 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.358863  1873 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.358741  1779 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:50.401172 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 322093us
W20260709 12:17:50.391728  1669 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.000s
I20260709 12:17:50.427831  1738 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.667791  1738 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.667567  1669 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.668285  1738 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.669790  1669 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.560108 31968 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:50.670432  1738 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:50.492532 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 173875us
I20260709 12:17:50.569473 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 525) took 1259 ms. Trace:
I20260709 12:17:50.672942 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:50.375871  1888 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:17:50.672537 32658 rpcz_store.cc:276] 0709 12:17:49.310301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.311154 (+   853us) service_pool.cc:224] Handling call
0709 12:17:49.344943 (+ 33789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.463008 (+118065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.463041 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:49.463082 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:49.463115 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.569252 (+1106137us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54004c13d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:50.569292 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.569415 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270999808}
W20260709 12:17:50.577452 32113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1594) took 2263 ms (client timeout 1632 ms). Trace:
W20260709 12:17:50.671676 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 274858us
W20260709 12:17:50.641191 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.675014 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:50.375999  1529 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:50.372443  1895 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:50.673962 32113 rpcz_store.cc:269] 0709 12:17:48.314427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.428787 (+114360us) service_pool.cc:224] Handling call
0709 12:17:50.577271 (+2148484us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400147dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:50.577387 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544490240}
I20260709 12:17:50.374894  1894 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.374121  1670 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:50.495999  1780 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.864s	user 0.011s	sys 0.000s
I20260709 12:17:50.374524  1717 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:50.390861  1728 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.726s	user 0.009s	sys 0.002s
I20260709 12:17:50.694607 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.374871  1858 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.392379  1713 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:50.565500  1750 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:50.688261  1738 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:50.678007  1599 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.692s	user 0.056s	sys 0.100s
I20260709 12:17:50.675990 32121 rpcz_store.cc:276] 0709 12:17:46.396495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.429784 (+1033289us) service_pool.cc:224] Handling call
0709 12:17:47.430082 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:47.430424 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.430451 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:47.436197 (+  5746us) log.cc:844] Serialized 26 byte log entry
0709 12:17:47.982556 (+546359us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b180004bc40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:17:48.427343 (+444787us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:48.427386 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.427456 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.049063 (+1621607us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:50.399809 (+350746us) raft_consensus.cc:1685] finished
0709 12:17:50.399828 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.399986 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":494,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5280,"spinlock_wait_cycles":17328256,"thread_start_us":444581,"threads_started":1,"wal-append.queue_time_us":1045962}
W20260709 12:17:50.561364  1837 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
W20260709 12:17:50.712932 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 326454us
I20260709 12:17:50.714177  1914 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:50.714825 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.004s	sys 0.020s
W20260709 12:17:50.757928 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.004s	sys 0.021s
W20260709 12:17:50.737800  1855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.005s	sys 0.025s
W20260709 12:17:50.758566 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 774779us
I20260709 12:17:50.760082 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:50.718302 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 62909.1us
I20260709 12:17:50.719020  1913 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:50.719306 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 337798us
I20260709 12:17:50.721658  1670 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.633250  1836 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:50.632253 32656 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:50.723127  1895 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.459255 32115 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:50.374636  1893 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:17:50.710676 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.653581  1887 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:50.731501 32116 raft_consensus.cc:606] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:50.402451  1811 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:50.472934  1714 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:50.459476  1800 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.678988  1699 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:50.384299  1823 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:50.707439  1859 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:50.791777  1699 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.709882  1877 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:50.793366  1823 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.711690  1529 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:50.794498  1529 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.747062  1903 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:17:50.765218 32119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1746) took 2487 ms (client timeout 2235 ms). Trace:
I20260709 12:17:50.738379  1651 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:50.765834 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 185484us
W20260709 12:17:50.804075 32119 rpcz_store.cc:269] 0709 12:17:48.277706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.593995 (+316289us) service_pool.cc:224] Handling call
0709 12:17:50.765023 (+2171028us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002f9c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:50.765153 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454993792}
I20260709 12:17:50.732613  1779 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.770221 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:50.806761 32113 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:50.770609 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:50.747120  1713 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:50.513379  1785 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.808979 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:50.774775 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 64168.2us
I20260709 12:17:50.504655  1787 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.776859 32116 raft_consensus.cc:614] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:50.758798  1738 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:50.740594  1780 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.749495  1331 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:50.778676 32652 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:17:50.771956  1791 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.367s	user 0.013s	sys 0.000s
I20260709 12:17:50.747692  1728 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.770051  1892 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.732122  1894 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.763401 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:50.847484 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 84173.8us
I20260709 12:17:50.778874  1891 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:50.799448  1747 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:50.771719  1836 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:50.752701  1599 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.044s	user 0.056s	sys 0.100s
W20260709 12:17:50.860445  1813 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.000s
I20260709 12:17:50.776160 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 433) took 1577 ms. Trace:
I20260709 12:17:50.807498 32120 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:50.764794  1855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.005s	sys 0.026s
I20260709 12:17:50.772338  1274 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:17:50.775137  1755 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:17:50.845660  1791 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.798590  1893 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:50.802233  1886 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.496s	user 0.001s	sys 0.000s
I20260709 12:17:50.774972  1750 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:50.881494 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 528) took 1170 ms. Trace:
I20260709 12:17:50.882776 32659 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:50.825489 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:50.802403  1811 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:50.885957 32046 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:50.887283 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.887853 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.888214 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.808115  1859 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:50.822962  1873 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.849344 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:50.851017  1917 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:50.860580  1754 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.798s	user 0.006s	sys 0.002s
I20260709 12:17:50.765687  1837 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.852644  1651 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.852711  1747 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.882853  1874 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.868885 32044 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:50.908732 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:50.854552  1632 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.859s	user 0.011s	sys 0.001s
I20260709 12:17:50.823400 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.889679 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:50.882110 32657 rpcz_store.cc:276] 0709 12:17:49.711413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.711605 (+   192us) service_pool.cc:224] Handling call
0709 12:17:49.712078 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.821327 (+109249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.821356 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.824304 (+  2948us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.911946 (+ 87642us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:49.912035 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.912308 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.801713 (+889405us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:50.881039 (+ 79326us) raft_consensus.cc:1685] finished
0709 12:17:50.881075 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.881400 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108765,"raft.queue_time_us":112423,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":6275,"spinlock_wait_cycles":6977920,"thread_start_us":53369,"threads_started":1,"wal-append.queue_time_us":396058}
W20260709 12:17:50.913079 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89742.2us
I20260709 12:17:50.869202 32651 rpcz_store.cc:276] 0709 12:17:49.198781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.205425 (+  6644us) service_pool.cc:224] Handling call
0709 12:17:49.833761 (+628336us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:17:49.833961 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.834202 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:49.981779 (+147577us) log.cc:844] Serialized 14 byte log entry
0709 12:17:50.335448 (+353669us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b18003be8a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:17:50.719168 (+383720us) log.cc:844] Serialized 14 byte log entry
0709 12:17:50.741041 (+ 21873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:50.752545 (+ 11504us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:50.764188 (+ 11643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.767130 (+  2942us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.773578 (+  6448us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161072,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":3193,"raft.queue_time_us":579,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":21703680,"thread_start_us":513024,"threads_started":2,"wal-append.queue_time_us":730513}
I20260709 12:17:50.915778  1912 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.875135  1755 consensus_queue.cc:1243] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:50.919981  1873 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:50.920682  1905 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:17:50.864792  1858 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:17:50.798292  1800 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.840701  1738 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.878691  1725 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:50.897821  1660 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:50.916463  1915 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:50.905967  1728 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.915333 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 436) took 1513 ms. Trace:
I20260709 12:17:50.913830 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:50.897320  1906 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:17:50.897517  1903 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.900381  1581 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:50.953826  1581 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:50.895164  1836 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.955281  1836 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.899290  1914 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:50.957245  1914 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.895834  1699 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.951007  1915 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.961357  1699 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:50.929637  1660 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.899461  1917 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:50.931578  1861 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.252s	user 0.007s	sys 0.006s
I20260709 12:17:50.963480  1581 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.910487  1754 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.931663 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:50.932037  1783 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.967028  1754 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:50.967371  1783 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:50.968523  1783 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.865970  1713 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:50.923907  1908 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:17:50.956455 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 526) took 1645 ms. Trace:
I20260709 12:17:50.971203  1895 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.971462 32655 rpcz_store.cc:276] 0709 12:17:49.310763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.335966 (+ 25203us) service_pool.cc:224] Handling call
0709 12:17:49.345442 (+  9476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:50.060575 (+715133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.955080 (+894505us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b540067e1a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d2a02 00007f228c1cedd6 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:50.956107 (+  1027us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b540067ded4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:50.956376 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544372864}
W20260709 12:17:50.932547  1703 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.937s	user 0.012s	sys 0.000s
I20260709 12:17:50.932776  1859 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.977713  1731 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.884811  1893 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:50.933859  1763 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:50.978428  1763 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:50.934849  1878 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.876s	user 0.011s	sys 0.000s
I20260709 12:17:50.979393  1878 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.979276  1763 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:50.980147  1813 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:50.980687  1813 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:50.981663  1878 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.935855  1894 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.937520  1800 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:50.906592  1811 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.933184  1837 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:50.986627  1787 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.981679  1813 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.993687  1780 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:50.996227 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 435) took 1595 ms. Trace:
I20260709 12:17:50.998239 32659 rpcz_store.cc:276] 0709 12:17:49.401309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.401780 (+   471us) service_pool.cc:224] Handling call
0709 12:17:49.402041 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:50.883353 (+1481312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.995900 (+112547us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:50.996140 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3404747136}
I20260709 12:17:51.001008  1670 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:51.011123  1914 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.012094  1763 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.014436  1731 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.015087  1893 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.015960  1917 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:51.016023  1783 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.020489  1779 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.021973  1713 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.179330  1780 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:51.180392  1780 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:51.094398 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.183389 31971 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:51.184286  1780 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:51.184926  1780 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.897742 31966 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:51.185774  1780 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:50.902784 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:50.901079 31938 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.632s (SENT) [suppressed 23 similar messages]
W20260709 12:17:51.103046 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.234534 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:51.101541 31965 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.236893  1780 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.237478 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.110245 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 214) took 1141 ms. Trace:
W20260709 12:17:51.124751 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.239631 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:51.240159 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.239735 32044 rpcz_store.cc:276] 0709 12:17:49.969045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.969187 (+   142us) service_pool.cc:224] Handling call
0709 12:17:51.109463 (+1140276us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b5400032fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:51.109916 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006075264}
I20260709 12:17:51.241396 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.203442  1893 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.243007 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:50.893307 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44049.2us
I20260709 12:17:51.243076  1893 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:50.896561 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 86083.4us
I20260709 12:17:51.249015 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:50.911224 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.250416  1780 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:51.250945 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.251083  1780 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:51.251904 32114 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:51.252944  1780 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.253793  1895 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:51.245965  1787 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:51.258852  1928 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:51.259634  1787 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.143396 32112 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:51.144302 32117 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.148703 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37262 (request call id 9) took 1909 ms. Trace:
W20260709 12:17:51.148885 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 235004us
I20260709 12:17:51.151293 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1743) took 2944 ms. Trace:
I20260709 12:17:50.950219 32653 rpcz_store.cc:276] 0709 12:17:49.401649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.407171 (+  5522us) service_pool.cc:224] Handling call
0709 12:17:49.487008 (+ 79837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.931009 (+444001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.915053 (+984044us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54004c27d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:50.915273 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":881788288}
I20260709 12:17:51.154438 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:51.156241 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:51.158128  1599 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.242247 32113 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:17:51.167363 31867 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 242) took 13916 ms (client timeout 15000 ms). Trace:
I20260709 12:17:51.170679  1669 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.173074  1670 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:51.177452  1704 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:51.022451  1804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.005s	sys 0.003s
I20260709 12:17:51.036964  1823 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.042240  1873 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.050539  1877 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:17:51.334631  1804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.005s	sys 0.004s
I20260709 12:17:51.334820 32658 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.038120  1915 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:51.187098 31971 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 165) took 4897 ms (client timeout 2277 ms). Trace:
W20260709 12:17:51.336712 31971 rpcz_store.cc:269] 0709 12:17:46.289617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.289739 (+   122us) service_pool.cc:224] Handling call
0709 12:17:51.186716 (+4896977us) spinlock_profiling.cc:243] Waited 4.89 s on lock 0x7b54000b5754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:51.186970 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11250165120}
I20260709 12:17:51.206044 31964 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.206604  1836 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:51.338743  1836 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.339715 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:51.340396  1660 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.341250  1836 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:51.206267  1914 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:51.341434 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:51.209405  1731 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:51.343043  1660 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:51.210251  1529 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.213301 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:51.348233 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:51.349546 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:51.215035 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:51.350773 32656 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:51.351713 32658 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.215343 32657 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:51.353736 31964 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:51.216404  1800 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:51.351713  1934 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 12:17:51.354779  1800 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:51.355942  1937 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:17:51.215363 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99359.8us
I20260709 12:17:51.224740  1904 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:51.359918  1917 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:51.245945 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:51.360631  1917 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:51.246062  1893 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:51.262818 32116 rpcz_store.cc:276] 0709 12:17:49.239306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.263344 (+ 24038us) service_pool.cc:224] Handling call
0709 12:17:51.148044 (+1884700us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400123754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1c9853 00007f229119760a 00007f228a71d7d4 00007f228a71d762 00007f228a71d691 00007f228a71d60c 00007f228a71c8c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:51.148328 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788763904}
I20260709 12:17:51.282160 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1762) took 1325 ms. Trace:
I20260709 12:17:51.362900 32117 rpcz_store.cc:276] 0709 12:17:49.940973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.140795 (+1199822us) service_pool.cc:224] Handling call
0709 12:17:51.141736 (+   941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.263768 (+122032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.266697 (+  2929us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:51.364074  1932 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:17:51.293931 31965 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:51.262195  1787 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.303542  1670 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.305399 31869 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 147) took 10400 ms. Trace:
I20260709 12:17:51.369367  1915 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:51.267612 31867 rpcz_store.cc:269] 0709 12:17:37.275267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.282741 (+  7474us) service_pool.cc:224] Handling call
0709 12:17:51.167029 (+13884288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:40.136539 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:40.167218 (+ 30679us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:40.167254 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:17:40.167272 (+    18us) tablet.cc:662] Decoding operations
0709 12:17:40.363768 (+196496us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:40.372177 (+  8409us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:40.374239 (+  2062us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:17:40.415687 (+ 41448us) tablet.cc:701] Row locks acquired
0709 12:17:40.415704 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:17:40.415913 (+   209us) write_op.cc:270] Start()
0709 12:17:40.416043 (+   130us) write_op.cc:276] Timestamp: P: 1783599460415844 usec, L: 0
0709 12:17:40.416065 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:17:40.416814 (+   749us) log.cc:844] Serialized 7677 byte log entry
0709 12:17:41.227962 (+811148us) op_driver.cc:464] REPLICATION: finished
0709 12:17:41.691338 (+463376us) write_op.cc:301] APPLY: starting
0709 12:17:41.691451 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:41.729762 (+ 38311us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:41.766835 (+ 37073us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:41.937323 (+170488us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:41.941586 (+  4263us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:17:41.941640 (+    54us) write_op.cc:312] APPLY: finished
0709 12:17:41.942248 (+   608us) log.cc:844] Serialized 132 byte log entry
0709 12:17:41.942450 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:41.942703 (+   253us) write_op.cc:454] Released schema lock
0709 12:17:41.942772 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:41.943254 (+   482us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:41.943285 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1993867,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":399123,"apply.run_cpu_time_us":7205,"apply.run_wall_time_us":296767,"num_ops":19,"prepare.queue_time_us":273180,"prepare.run_cpu_time_us":21978,"prepare.run_wall_time_us":430300,"raft.queue_time_us":287208,"raft.run_cpu_time_us":115809,"raft.run_wall_time_us":313524,"replication_time_us":811638,"thread_start_us":704906,"threads_started":4,"wal-append.queue_time_us":516004,"wal-append.run_cpu_time_us":16459,"wal-append.run_wall_time_us":2020138}]]}
W20260709 12:17:51.307379 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.008s
I20260709 12:17:51.310633  1704 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.311731  1669 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.315765  1747 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.319049 32660 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:51.320417 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:51.321971 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1752) took 1386 ms. Trace:
I20260709 12:17:51.330453  1599 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.344496  1823 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:51.263139 32118 rpcz_store.cc:276] 0709 12:17:48.206528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.163446 (+956918us) service_pool.cc:224] Handling call
0709 12:17:49.163974 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.165255 (+  1281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.165299 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.178856 (+ 13557us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.441316 (+262460us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:49.441372 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.441463 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.830212 (+388749us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:50.931292 (+1101080us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b24001f6478. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f228912109d 00007f228bd5310e 00007f228bd6a88c 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221
0709 12:17:51.150563 (+219271us) raft_consensus.cc:1685] finished
0709 12:17:51.150621 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.151015 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":641,"raft.queue_time_us":1397,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":9761,"spinlock_wait_cycles":9156224,"thread_start_us":221546,"threads_started":1,"wal-append.queue_time_us":427461}
W20260709 12:17:51.345494  1936 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.064s	user 0.003s	sys 0.000s
I20260709 12:17:51.334079  1877 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.393576  1704 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.393636 32659 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:51.396407  1599 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.401115 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.401501  1898 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.402344  1898 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:17:51.403069 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 157478us
I20260709 12:17:51.403281 32117 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:17:51.404951  1898 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:51.405700  1898 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:51.408242 31971 pending_rounds.cc:85] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 1
I20260709 12:17:51.408151 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.408665 31971 pending_rounds.cc:107] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:51.409086 31971 raft_consensus.cc:2889] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:51.409147 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.245280 31870 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 3 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:51.410673 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:51.412705 31870 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 2 to 4, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:51.369876  1915 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:51.413748  1898 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:51.414590 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46978.8us
I20260709 12:17:51.414991  1904 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:51.369594 31869 rpcz_store.cc:276] 0709 12:17:40.905492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.905842 (+   350us) service_pool.cc:224] Handling call
0709 12:17:51.305175 (+10399333us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:47.466200 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:47.497895 (+ 31695us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:47.497946 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:17:47.497979 (+    33us) tablet.cc:662] Decoding operations
0709 12:17:47.498959 (+   980us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:47.499043 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:47.499072 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:17:47.499815 (+   743us) tablet.cc:701] Row locks acquired
0709 12:17:47.499863 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:17:47.500346 (+   483us) write_op.cc:270] Start()
0709 12:17:47.500491 (+   145us) write_op.cc:276] Timestamp: P: 1783599467499969 usec, L: 0
0709 12:17:47.500510 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:17:47.619126 (+118616us) log.cc:844] Serialized 2789 byte log entry
0709 12:17:48.116885 (+497759us) op_driver.cc:464] REPLICATION: finished
0709 12:17:48.169076 (+ 52191us) write_op.cc:301] APPLY: starting
0709 12:17:48.169168 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:48.169378 (+   210us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:48.229129 (+ 59751us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:48.279646 (+ 50517us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:48.286523 (+  6877us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:17:48.292153 (+  5630us) write_op.cc:312] APPLY: finished
0709 12:17:48.343733 (+ 51580us) log.cc:844] Serialized 59 byte log entry
0709 12:17:48.396356 (+ 52623us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:48.412822 (+ 16466us) write_op.cc:454] Released schema lock
0709 12:17:48.412891 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:48.425269 (+ 12378us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:48.425305 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1707712,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":36027,"apply.run_cpu_time_us":4569,"apply.run_wall_time_us":244212,"num_ops":7,"prepare.queue_time_us":503075,"prepare.run_cpu_time_us":23411,"prepare.run_wall_time_us":300699,"raft.queue_time_us":238102,"raft.run_cpu_time_us":7836,"raft.run_wall_time_us":88258,"replication_time_us":595174,"thread_start_us":556258,"threads_started":4,"wal-append.queue_time_us":209737,"wal-append.run_cpu_time_us":31930,"wal-append.run_wall_time_us":1653589}]]}
I20260709 12:17:51.416371  1893 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:51.417439  1917 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:51.417822  1917 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:51.419068  1917 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:51.380730 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.008s	sys 0.008s
I20260709 12:17:51.408162 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:51.408577 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:51.415774  1898 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.369298  1670 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.420488  1917 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.353803  1816 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:51.414525  1915 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.376278  1800 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:51.420902 31965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1436) took 1385 ms (client timeout 1531 ms). Trace:
I20260709 12:17:51.382541 32112 rpcz_store.cc:276] 0709 12:17:49.935483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.937965 (+  2482us) service_pool.cc:224] Handling call
0709 12:17:51.321723 (+1383758us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400433fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:51.321879 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670734720}
W20260709 12:17:51.427870 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 659535us
I20260709 12:17:51.386874  1804 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:51.454249 31965 rpcz_store.cc:269] 0709 12:17:50.035134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.035591 (+   457us) service_pool.cc:224] Handling call
0709 12:17:51.420610 (+1385019us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400413a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:51.420810 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2437569408}
I20260709 12:17:51.423444  1779 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.362637  1529 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.461409 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:51.447501  1816 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:51.467474 31964 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:51.463428 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:51.476703 32044 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:51.420506  1859 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 12:17:51.488435 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 481507us
W20260709 12:17:51.497854 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.503305 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.510452 32046 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:51.512006  1816 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.513677  1816 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:51.514515  1599 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.497808  1912 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.532079  1747 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:51.494643 31870 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 6, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 6 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:51.546496 32652 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:51.497965  1933 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:17:51.562768 31870 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:51.565944 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:51.566746 32046 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.570379 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:51.571314  1886 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:51.571868 32112 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.573032  1859 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:51.561685 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:51.582332 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:51.583017 32044 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:51.583770 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.585088 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.585827 32113 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 1.
I20260709 12:17:51.586771 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.587282 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:51.587986 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.589442 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:51.594137 32660 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.596529 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:51.597337 31966 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:51.598158 32038 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:17:51.598549 32038 pending_rounds.cc:85] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 3
I20260709 12:17:51.598591 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:51.598853 32044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:56202
I20260709 12:17:51.599455 31972 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.599599 32044 raft_consensus.cc:606] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:51.600013 32044 raft_consensus.cc:614] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:51.601727 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:51.605063 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:51.598861 32038 pending_rounds.cc:107] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:17:51.605871 32040 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.606108 32038 raft_consensus.cc:2889] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:51.611030  1660 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:51.611500  1660 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:51.612660  1660 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.617250 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:51.617969 32112 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.605400 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:51.598593 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:51.493566  1779 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:51.620716 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 122388us
I20260709 12:17:51.625754 32040 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:51.637137 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:51.642504 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 121497us
I20260709 12:17:51.645910 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:51.599511 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:51.648190 32121 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.564884 31870 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 3 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), no longer has a pending config: VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2), VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:51.650444 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:51.652360 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:51.660691 32045 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:51.674599 32046 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:51.657963  1780 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:51.645524  1823 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:17:51.714389  1914 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:51.714782  1943 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:17:51.714890  1914 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:51.715425  1914 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.716552 31968 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.718336 31963 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:17:51.718744 31963 pending_rounds.cc:85] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 3
I20260709 12:17:51.719008 31963 pending_rounds.cc:107] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:17:51.719246 31963 raft_consensus.cc:2889] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:51.720410 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:51.721815 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:51.659524  1731 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:51.620497  1912 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:51.852377 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:51.808845  1713 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.629s	user 0.013s	sys 0.000s
I20260709 12:17:51.814149  1669 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:51.859093 32115 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:17:51.710572 31870 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 3, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 3 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:51.730841 32044 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:51.767227 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48090.6us
W20260709 12:17:51.730477 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:51.731297 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64807.4us
I20260709 12:17:51.821221 31972 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:17:51.756939 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.780555 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:51.738159 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:51.835507 32047 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:17:51.846961 32087 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.30.25.62:33695, user_credentials={real_user=slave}} blocked reactor thread for 43587us
I20260709 12:17:51.922551 32112 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:51.737016 31969 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.200331  1813 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.227543  1669 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:17:51.963658 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58379.1us
I20260709 12:17:51.734298  1779 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.244968 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:51.879222 31970 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:51.713078  1780 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:51.810251  1783 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.793s	user 0.013s	sys 0.003s
W20260709 12:17:51.815565  1944 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:17:51.992827 32044 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:52.081944  1816 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.000s
W20260709 12:17:52.081182  1915 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.004s
W20260709 12:17:52.080964  1917 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.002s
I20260709 12:17:52.229138 32042 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:51.739327 32040 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:17:52.028349 31870 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:51.737823  1921 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.223140  1529 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.239961 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.285219 32121 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:52.214928 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:52.248998 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.991068  1893 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.987030  1895 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:51.881155  1954 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:17:52.251853  1904 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:52.252002 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:52.295938 32044 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:52.252645  1780 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.253813 31935 leader_election.cc:400] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5.
I20260709 12:17:51.986279  1670 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.268s
I20260709 12:17:52.255087 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.256027 32116 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.888193  1731 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:52.259927 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.266897 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.214761  1660 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.279134  1779 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.282442  1915 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:51.895313 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:52.285568 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:52.249650 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.298570 31935 leader_election.cc:403] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:52.285717  1787 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.286464  1747 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:52.308914 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:52.318363 32115 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
I20260709 12:17:52.307354  1670 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:52.321486 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:52.299180  1895 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.323446 32121 pending_rounds.cc:85] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 2
I20260709 12:17:52.327179 32121 pending_rounds.cc:107] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:52.328222 32039 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.327495 32121 raft_consensus.cc:2889] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.329990  1906 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5.
I20260709 12:17:52.332554 32115 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:52.333457 31970 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:52.332086 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:52.307689  1779 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.320123 31870 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 3 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.296343  1921 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.324292  1917 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:52.302536  1893 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.330718  1731 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.315393  1787 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.345799 31870 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 2 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:52.311833  1904 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.350872 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.351518  1895 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:52.353444 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 99865.8us
I20260709 12:17:52.351643  1915 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.357591 31971 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:52.343300  1917 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:52.355994 32115 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:52.367195 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.372723 31970 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.333876  1747 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.376098 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:52.358703  1914 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.005s
I20260709 12:17:52.378456 32117 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:52.378276 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.377195  1958 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.382606  1958 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.385289  1958 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.387138 32120 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:52.389386 32013 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.364285  1917 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.376299  1816 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:52.401211 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:52.413506  1904 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.414386 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:52.418972  1957 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:52.419325  1893 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.423202  1787 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.425289 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:52.427341 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 116403us
I20260709 12:17:52.432366 32655 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:52.432796 32655 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:52.434582  1787 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.435186  1787 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.436342 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.436868 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.437613 32651 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:52.371598  1928 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:17:52.433701 32655 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:52.438956 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.436899 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57824.3us
I20260709 12:17:52.401841  1965 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:52.450176 31969 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:52.389540  1915 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:17:52.407992  1816 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.461153 32655 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.456892 32112 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:52.436424 32654 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:52.455202 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.420217 32656 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.394009  1914 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.443310  1787 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.468365 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.449411  1731 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.455163 31870 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 2 to 6, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 6 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:52.471134 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 192664us
I20260709 12:17:52.452565  1965 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.452066  1893 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.470295 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:52.464949  1928 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:52.466775 32654 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 5
I20260709 12:17:52.452108  1904 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.467345  1816 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.478494  1965 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.498382 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:52.483745  1928 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:52.500572  1914 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.501029  1928 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.490703  1906 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.502383  1928 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.502805  1906 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:52.503237  1906 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.507246  1780 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.507691  1904 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.509425  1780 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.509788  1904 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.511045 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:52.511961  1529 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.512395 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:52.512957 32041 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5.
I20260709 12:17:52.513269  1731 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:52.513716  1731 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:52.513995  1779 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.516256  1747 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.514098  1731 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.479499  1912 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.518865  1811 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:17:52.520790 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.520877  1811 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:52.521143 32656 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:52.521401  1811 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:52.471518  1915 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:52.523684  1915 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:52.523319  1811 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.524528 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:52.524796  1811 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.525439  1669 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:52.525851  1669 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:52.526232  1669 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.527908 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.528522 32112 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.490998 32654 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.6
I20260709 12:17:52.529135 32654 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Aborting config change with OpId 4.6: Aborted: Op aborted by new leader
I20260709 12:17:52.529592 32654 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.480243  1893 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.530941  1763 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:52.531333 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.531420 31870 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 2 to 3, term changed from 1 to 5, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:52.533633  1669 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.533216 31870 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 3 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 4 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.534859 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.535543 32117 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:17:52.530659 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.536541 32113 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:52.538496  1669 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.540711  1965 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.542963  1965 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.543125  1783 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:52.543740  1783 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.546406 32039 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:52.546825 32114 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:52.563860 32655 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:52.580363 32112 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:52.601058 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 63488.3us
I20260709 12:17:52.620499 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.621377 32112 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.553678  1529 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.553767  1763 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.639694 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 91027.6us
W20260709 12:17:52.641970 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 367444us
W20260709 12:17:52.639855 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 125732us
W20260709 12:17:52.639631 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 93869.5us
I20260709 12:17:52.645429 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.601315  1914 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.661511 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.664820 32116 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:52.640084  1823 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:52.659397  1943 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:17:52.642480 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.685765 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.686900 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:52.688040 32112 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:52.622763  1747 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:17:52.684787 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.658844  1912 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:52.667951  1968 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.646165  1965 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:17:52.693840 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.694438  1912 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.695962 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.698159 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:52.692540 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:52.691509 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.701717 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.708063 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.667181  1779 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:17:52.691298  1713 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.707105 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.719470 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.695760  1787 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:52.709503 32112 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:17:52.718144 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.695416  1731 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.696879  1904 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:52.675099  1763 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.723809 32117 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:52.696198  1914 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.734701 32118 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:52.735787 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 198) took 1176 ms. Trace:
I20260709 12:17:52.736157 32114 rpcz_store.cc:276] 0709 12:17:51.560806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.618832 (+ 58026us) service_pool.cc:224] Handling call
0709 12:17:52.735631 (+1116799us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b540012b6d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:52.735729 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983489920}
I20260709 12:17:52.736644  1914 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.737339  1804 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:52.737857  1804 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:52.738811  1804 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.696612  1898 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.740854  1898 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.741276 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.742169 32013 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:52.743011  1898 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:52.744032 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.745836  1933 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.729202 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57404.5us
I20260709 12:17:52.729454 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.730158  1964 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.730968  1779 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:52.732221  1787 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:52.722826 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:52.732468  1811 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.739248  1783 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.746717  1913 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:52.747509 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.747812 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1531) took 1152 ms. Trace:
I20260709 12:17:52.726858  1713 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.727640 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.749020  1914 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:52.802192 32120 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:17:52.755841 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.804239 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:17:52.800805 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.796720 32041 rpcz_store.cc:276] 0709 12:17:51.595736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.599117 (+  3381us) service_pool.cc:224] Handling call
0709 12:17:52.747638 (+1148521us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b5400217354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:52.747755 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1987281664}
I20260709 12:17:52.690985  1529 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.807729 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:52.722795  1928 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:52.749364  1670 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:52.757201 32116 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:52.805337 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:52.811785 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.793633  1974 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:52.733815  1816 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.793756  1779 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:52.863358 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.864872 32119 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:17:52.706583  1823 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.723192  1921 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:52.870702 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.718452  1966 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
W20260709 12:17:52.872771 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 165928us
I20260709 12:17:52.720690  1968 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.848170  1906 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.759684  1904 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.874823 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70667.1us
I20260709 12:17:52.847920  1928 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.804777  1913 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:52.884135 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.847070  1670 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:52.885501 32119 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:52.890269 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 218717us
I20260709 12:17:52.875907  1747 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:52.891081 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 83418.9us
I20260709 12:17:52.891520 32112 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:52.892848 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:52.893399 32113 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:52.889987 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 191929us
I20260709 12:17:52.875537  1974 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:52.880420  1978 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:52.803867  1933 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.883183  1904 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:52.886264  1928 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.899044 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:52.886989  1763 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.904476  1893 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:52.906809  1961 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:17:52.898069 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:52.907620  1917 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.907297  1779 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.889740  1811 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:17:52.905583 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:53.266883 32114 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:53.274961 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1536) took 1638 ms. Trace:
I20260709 12:17:53.275373 32038 rpcz_store.cc:276] 0709 12:17:51.636054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.641199 (+  5145us) service_pool.cc:224] Handling call
0709 12:17:51.665221 (+ 24022us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:51.734745 (+ 69524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:51.734777 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:51.737094 (+  2317us) log.cc:844] Serialized 52 byte log entry
0709 12:17:51.989848 (+252754us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b1800420dc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22900a358f 00007f22890e2571 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd
0709 12:17:52.034005 (+ 44157us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b1800420dc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9
0709 12:17:52.256590 (+222585us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b1800420dc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:17:52.458577 (+201987us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:52.458632 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.458719 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.274717 (+815998us) raft_consensus.cc:1685] finished
0709 12:17:53.274736 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:53.274885 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43453,"raft.queue_time_us":669,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":126429824,"thread_start_us":201774,"threads_started":1,"wal-append.queue_time_us":918078}
I20260709 12:17:52.910460  1914 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:53.102862 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:53.279652 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 379630us
W20260709 12:17:52.914669 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.225409 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.283250 32113 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
W20260709 12:17:53.190786 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 310728us
W20260709 12:17:53.284889 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 389016us
I20260709 12:17:52.952338 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:52.895190  1983 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:53.308010 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 422516us
W20260709 12:17:53.289304 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.905854  1921 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:53.264743  1988 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.358s	user 0.000s	sys 0.005s
W20260709 12:17:52.922259  1970 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 12:17:52.921957  1977 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:17:52.928114  1823 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:52.907768  1978 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:53.311877 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:53.321295  1978 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.934792  1974 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.327553  1959 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.328096  1959 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.915489  1713 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.319133  1921 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:52.921106  1787 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.307096  1958 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.930452  1816 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:52.905297 32651 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:52.914805  1581 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.932046  1906 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:52.924930  1981 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:53.316784 31936 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:52.911484  1747 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:52.940274  1933 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.292515 32117 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.926844  1783 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.929481  1731 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:53.334796 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 436748us
I20260709 12:17:53.344489 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:53.347267 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.383973 32112 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:53.348496 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:53.383252 32658 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:53.309751 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:53.388037 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 106532us
I20260709 12:17:53.333807  1921 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.346233 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:52.928344  1968 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.344731  1982 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:53.385281 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:53.354125  1983 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:52.920873  1915 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.330287  1959 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.353472  1958 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:53.399252 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:53.400791 32652 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:53.387552 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:53.404942 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.940145  1912 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:53.409436 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 100899us
I20260709 12:17:53.343472  1904 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.330891  1928 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:52.928786  1670 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.395589  1816 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:53.429131 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.404637  1804 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.404035  1968 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:53.411393  1958 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.405906  1529 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.415932  1921 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.402707  1906 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:53.400537  1581 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.397614  1783 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.455937  1971 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:53.455276  1986 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:53.544723 32010 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.997s (SENT)
W20260709 12:17:53.449247 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 121721us
I20260709 12:17:53.414674  1915 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.398823  1933 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:17:53.521133 32119 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:17:52.930896  1913 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.546669 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:17:53.570353 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 253596us
W20260709 12:17:53.547498 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 158843us
I20260709 12:17:53.574520 32655 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:53.408841  1914 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.529595  1921 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.581806 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.571916 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:53.404889  1981 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.520499  1968 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:53.447628  1816 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.527697  1813 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.577519 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.408119  1982 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.543841  1670 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:53.612192 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67515.5us
I20260709 12:17:53.584326 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1777) took 1278 ms. Trace:
I20260709 12:17:53.614465 32121 rpcz_store.cc:276] 0709 12:17:52.306169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.306431 (+   262us) service_pool.cc:224] Handling call
0709 12:17:52.316333 (+  9902us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:52.344464 (+ 28131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:52.344485 (+    21us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:52.369881 (+ 25396us) log.cc:844] Serialized 600 byte log entry
0709 12:17:52.619001 (+249120us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:52.619062 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.619156 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.584116 (+964960us) raft_consensus.cc:1685] finished
0709 12:17:53.584138 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:53.584270 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62254,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"thread_start_us":192909,"threads_started":1,"wal-append.queue_time_us":280600}
I20260709 12:17:53.492826  1804 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:53.585289 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 272082us
I20260709 12:17:53.617117 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:53.621539 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 309730us
W20260709 12:17:53.622897 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 175306us
I20260709 12:17:53.584944  1921 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.576793  1992 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:53.414721  1965 consensus_queue.cc:1175] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "38c927f0ee104b57972899f5b27aa99b" responder_term: 7 status { last_received { term: 6 index: 4 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:53.594394  1893 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:53.592633  1968 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.630546 32626 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:53.636075 32113 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:53.586287  1971 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:53.636958 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:53.631187 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:53.543852  1747 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.585862  1529 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 12:17:53.656920 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 347202us
I20260709 12:17:53.651171 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:53.661808 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:53.652038  1811 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:53.662873 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.663115  1811 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.663540 32114 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:53.652057 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:53.664410 32121 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.662441 32120 raft_consensus.cc:2376] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5. Re-sending same reply.
I20260709 12:17:53.618687  1813 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.670181 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:53.667861 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:53.668593 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:53.647588  1893 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:53.674835 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.671491  1986 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:53.678551  1996 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:53.678639  1986 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:53.679085  1996 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.679472  1996 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:53.651346  2003 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:53.675906 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 92837.3us
I20260709 12:17:53.672955 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:53.677021 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:53.676434 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 94362.2us
I20260709 12:17:53.681073  2004 raft_consensus.cc:2764] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:53.672755  1928 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.684515  2003 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.684948  1816 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.686607  1813 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.690774  1669 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.690641  1989 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:53.692085  1811 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.682936 32118 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:53.650240  2001 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:53.695685 32117 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:53.697363 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:53.674064  1978 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:53.709932  1898 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:53.710458 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:53.710884  1823 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:53.672670  2002 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:53.721433 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51322.6us
I20260709 12:17:53.681591  1912 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:53.723667 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:53.730167 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 76245.2us
W20260709 12:17:53.739112 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.741624 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.707638  1904 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.725479  1983 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:53.676662  2011 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.745332 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:53.733630 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.747641 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:53.749725 32114 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:53.751507 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.752108 32116 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:53.732579  1804 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.732304  2002 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.754437 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:53.740025  1823 raft_consensus.cc:1068] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Unable to promote non-voter ec7f26ea46944ea498ad253b1b1819b0: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:53.742578 32118 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:53.763435  1981 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.743094  2003 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.747970  2009 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.764825  2009 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.765286  2009 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.766394  2002 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:53.769132  1669 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:53.770430  2008 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 12:17:53.757371 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:53.771847  1999 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:17:53.772958 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:53.756717 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:53.679683  1986 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.773079 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:53.738706  1912 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:53.774355  2005 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:17:53.778504  1912 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.782532  1895 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.786346 32114 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:17:53.788511 32121 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:17:53.777905 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:53.750568  1983 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.759601  1965 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:53.802868 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.759207  1783 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.795056  1813 raft_consensus.cc:2764] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:53.795532  1670 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:53.798982  1983 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:53.722131  2001 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.798902  1904 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.825773 32120 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:53.795153  1669 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:53.784250  1895 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:17:53.804865  1581 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.803736  1893 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:53.839075  2006 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:17:53.840505  2001 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:53.841940  1968 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.744753  1971 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.847159 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:53.813470  1965 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:53.858454 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85485.5us
I20260709 12:17:53.834015  1904 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.862098 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.874164 32657 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:53.861279 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:53.853585  1813 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.827412  1958 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.865437  1913 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:53.874840  1904 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.883061  1913 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:17:53.890564 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 87201.7us
I20260709 12:17:53.891810 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:53.879853  2008 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:53.903152  2008 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:53.904084  2008 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.910444  1913 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:53.913735 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 330864us
I20260709 12:17:53.919010  1933 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.891278  1813 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.922688 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:53.965750 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 82001.7us
I20260709 12:17:53.891294 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:53.895224  1965 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:53.923120 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:53.976724  1914 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:53.976333  2002 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.964933 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:53.977336  1914 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.977993 32656 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:53.895958 32654 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:53.897219  1982 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:53.899765 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:53.902585  1917 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:53.981416 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 90194.2us
I20260709 12:17:53.913986  1912 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.950259 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 454) took 1402 ms. Trace:
I20260709 12:17:53.935679  1996 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:53.983640  2009 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.000105 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:54.000988 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:53.991178  1964 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:54.007220  2012 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.961596  1921 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:54.010488 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.014927 32655 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:54.016057  2009 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.040849 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:54.041437 32117 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:54.043529 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.041545 32120 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.051695 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:53.999239  1914 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.082808 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.083973 31967 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:54.084427 31971 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:54.088987 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1870936620000521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 12:17:54.089605 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0781936620001034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 12:17:54.090034 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0664936620000844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 12:17:54.090458 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0214936620000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260709 12:17:54.093197  1823 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:54.022990 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:54.041078  1813 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.041709  1968 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.180285 32653 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:54.183099 31972 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:54.101182 31965 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:54.187060 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:54.187399 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:54.192061  1904 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.195602  1904 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.059579  1964 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:54.198737 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.998896 32651 rpcz_store.cc:276] 0709 12:17:52.548110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.553854 (+  5744us) service_pool.cc:224] Handling call
0709 12:17:53.949981 (+1396127us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:53.950152 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484598528}
I20260709 12:17:54.204270 31970 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:17:54.208356 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 149635us
I20260709 12:17:54.210254 31969 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:54.213632 31972 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:54.214006 31972 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 5
I20260709 12:17:54.214224 31972 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.6
I20260709 12:17:54.226671  2033 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:54.230485  2026 raft_consensus.cc:2764] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:17:54.025050  2012 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.234140  1895 raft_consensus.cc:2764] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:17:54.234349 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48165.7us
I20260709 12:17:54.235594 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:54.236325 32115 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:54.236156 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:54.237498 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.243779 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:54.236053 31972 raft_consensus.cc:2917] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Aborting config change with OpId 4.6: Aborted: Op aborted by new leader
I20260709 12:17:54.208966  1813 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:54.250816  1813 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:54.251362  1813 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.235771  1912 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.243274  1995 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.258957  1964 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.324007 31972 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:54.325409 31971 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:54.325924 32120 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:54.327472 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:54.346941 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 96484.1us
I20260709 12:17:54.347358 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.348009 31969 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:54.337055 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:54.350250 31970 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:17:54.358525 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 123054us
I20260709 12:17:54.351171  2003 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:54.355830  1968 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:54.368450 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:54.354653  1921 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.353582  1933 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:54.372740 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.360381  1996 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.355123  2009 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:17:54.379515 31970 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:54.380919 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.355129  1971 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:17:54.383723 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:54.385252 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.378408  1968 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:54.343528  1983 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.346266  1995 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.393244 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:54.387050  2039 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:54.397324 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.400964 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.401926 32038 raft_consensus.cc:2368] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:17:54.408334 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.413506 32041 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:54.409747  2039 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:54.429278  1895 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:54.436427  1895 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:54.436830  1895 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:54.437222  2043 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:54.438258  2039 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.440630  1813 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.441776 31969 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:54.442453  1968 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.459327  1983 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.363500 32656 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:17:54.464723  2032 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:17:54.451771  1971 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:54.452972  2045 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 12:17:54.489140 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 80721us
I20260709 12:17:54.490087  2045 consensus_queue.cc:1243] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:54.471035  1971 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.510218  1995 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.514706  1813 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.515261  2000 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.515352  1813 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:54.515890  2000 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.516039  1813 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.516870 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.517654 32047 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:17:54.518455  1983 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:54.518638 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.519009  1983 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.520061 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.520646 32043 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:17:54.521060  1983 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.516206 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.521854 32046 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:17:54.518014  2000 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.521540  1895 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.523283  1813 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.525647  1895 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.527141 32041 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:54.527477 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.532500 32039 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.535617  1971 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:54.537338  2022 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.355s	user 0.003s	sys 0.000s
I20260709 12:17:54.537583  1813 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.528803 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.538653  2000 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.538903 32046 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:54.529458 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.539187  2000 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.539785 32043 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:54.530550 32047 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [3.2-7.4]
I20260709 12:17:54.531194 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.541291  2000 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:54.541576 32042 raft_consensus.cc:2368] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:17:54.544509 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.546763 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.547621 32047 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:54.551143 32039 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:54.492244  2043 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:54.662019 32043 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:54.660315 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:54.664844  2043 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.665330 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:54.672618 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.538117  1813 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.677332 31971 pending_rounds.cc:85] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 5
W20260709 12:17:54.679272 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 189279us
I20260709 12:17:54.679565 31971 pending_rounds.cc:107] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:17:54.679994 31971 raft_consensus.cc:2889] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:54.680421 32084 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5.
I20260709 12:17:54.688608  1992 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.689356  1992 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.690781 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.692241 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.693104 31963 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:54.694691 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.695611 31968 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:54.696470 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.697780 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.702183  1989 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.702695  1989 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:54.705170  1989 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:54.707178 32046 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:54.710168 32044 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:54.712426 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.713276 32045 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:54.715569 32044 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:54.715785 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.716964 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.717768 32041 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:54.710595 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1630) took 1129 ms. Trace:
I20260709 12:17:54.718631 32119 rpcz_store.cc:276] 0709 12:17:53.581464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.581601 (+   137us) service_pool.cc:224] Handling call
0709 12:17:54.710431 (+1128830us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54002b52d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:54.710541 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052565760}
I20260709 12:17:54.719174 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.720923 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.722793 32041 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:54.721414 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.721897 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.725610 32046 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:54.728052 32043 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:54.730427  2038 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:54.731369 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.731948  1971 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:54.732086 32043 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:54.734071 32039 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:54.737548 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.732610  1971 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:54.740630 32042 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:54.740641 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.738797 31972 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:54.734381  2033 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:54.748174 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.693979 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.746053 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:54.748821 31966 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:54.749063 31963 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:54.751348 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:54.757400 31967 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:17:54.759375 31968 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:54.757754 31967 pending_rounds.cc:85] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 2
I20260709 12:17:54.786876 32042 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:54.791682 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.787140 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.739151  2004 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:17:54.793340 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:55.015296 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:54.882731 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:54.878644 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.897423 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:54.793545 31964 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:17:54.882241 32119 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [6.2-8.3]
I20260709 12:17:54.837420 32043 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:54.896160 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 144848us
I20260709 12:17:55.246778 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.841737 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.241421 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:54.788426 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:55.248183 32117 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:55.251989 32045 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.755658  1983 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.838127 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.262727 32042 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:54.906648 31965 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:54.792377 31967 pending_rounds.cc:107] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:17:55.252852 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:55.264083 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.264773 31965 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:55.250576 31968 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:55.266474 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.267180 31968 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:55.267974 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.268481 31968 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:55.269235 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.270200 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.271004 31968 raft_consensus.cc:2376] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7. Re-sending same reply.
I20260709 12:17:55.271839 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.239480 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.272665 32044 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:17:55.240346 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.274801 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:55.264917 31967 raft_consensus.cc:2889] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:54.861879 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:54.861443 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.278292 32040 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:55.276227 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:54.838405  2000 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.280122 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:55.292235 32046 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:17:55.320036 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43885.3us
I20260709 12:17:54.841409  1779 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.896466  2036 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:55.281704  1983 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:54.790557  1971 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:55.260864  2056 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:54.840457  2012 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.332224  1983 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.332938  2012 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.333496  2012 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:55.306777 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.303416 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:55.302580 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.334887  1968 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:55.335397 32043 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:55.335503  1968 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:55.336323 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.337679  1968 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.313033 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:55.339221  1904 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.303640 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:55.302241 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.339792  1904 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:55.340590 31967 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:55.341547 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.273538 31963 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:55.342764 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:55.239897  2063 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.379s	user 0.000s	sys 0.001s
I20260709 12:17:55.317905 31968 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:55.349463 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.919826  1974 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:55.310642  1986 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:55.318454 32116 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:55.318924 32119 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:54.903620  1992 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.318990 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:54.757221  2039 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.541s
W20260709 12:17:55.097447  2051 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:17:54.898293  2060 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:17:54.758057  2009 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.555s
I20260709 12:17:54.878541  1713 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.324889 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:54.897639  2052 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.760946  1997 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.560s
I20260709 12:17:55.295567  1895 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:17:55.330830  2056 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:55.326627  2000 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:55.339255 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:55.343578 31963 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:55.343878 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:55.316952 32045 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:55.312834  2050 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:54.785684  2033 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:55.280740  2045 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
W20260709 12:17:55.670064 31935 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56088 dropped due to backpressure: service queue is full with 50 items
I20260709 12:17:55.437662 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:55.446246 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.800230 32039 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:55.780257 32047 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:55.448282 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:55.477222 32113 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:55.803383  2039 consensus_queue.cc:1243] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:55.447757 32657 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:55.597002 32652 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:55.354467 31967 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:17:55.355165 31968 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:17:55.396360 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45058us
I20260709 12:17:55.806533 31964 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 4.
W20260709 12:17:55.780045 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:55.795543 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.002s	sys 0.021s
W20260709 12:17:55.814357 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.002s	sys 0.021s
W20260709 12:17:55.797452 32624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.013s	sys 0.004s
W20260709 12:17:55.797686 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 434995us
W20260709 12:17:55.343364  2043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.009s	sys 0.019s
I20260709 12:17:55.798189 31965 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:55.802909 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.804699 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1564) took 1015 ms. Trace:
W20260709 12:17:55.779866 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.607862 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1467) took 1259 ms. Trace:
I20260709 12:17:55.338202  1968 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:55.813200 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.012s	sys 0.016s
W20260709 12:17:55.594130  2063 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:17:55.964705 31963 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:55.541087  1983 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:55.798905 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1560) took 1010 ms. Trace:
I20260709 12:17:55.598672  2053 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:55.976697 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:55.977103 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.354085  1747 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:55.891834 32624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.022s	sys 0.022s
I20260709 12:17:55.509333  1966 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.650643  2036 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:55.611373  1958 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:17:55.963850 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.013s	sys 0.017s
I20260709 12:17:56.010857 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:56.011420 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.011564 31967 raft_consensus.cc:2376] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7. Re-sending same reply.
I20260709 12:17:55.868527 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:55.812641 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:56.012421  2061 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.739s	user 0.002s	sys 0.000s
W20260709 12:17:55.596690 32653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 461) took 1681 ms (client timeout 2050 ms). Trace:
I20260709 12:17:55.603627  1813 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.965618 32040 rpcz_store.cc:276] 0709 12:17:54.789320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.322496 (+533176us) service_pool.cc:224] Handling call
0709 12:17:55.804600 (+482104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:55.932245 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.853s (SENT) [suppressed 25 similar messages]
W20260709 12:17:56.040472 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 108258us
I20260709 12:17:55.966123 31969 rpcz_store.cc:276] 0709 12:17:54.348783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.354699 (+  5916us) service_pool.cc:224] Handling call
0709 12:17:54.361571 (+  6872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:54.442149 (+ 80578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.607501 (+1165352us) spinlock_profiling.cc:243] Waited 80 ms on lock 0x7b54004556d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:55.607749 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184009600}
I20260709 12:17:55.798802 31935 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56088 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1780)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1781)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1774)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1796)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1782)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1788)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1752)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1789)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1786)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1473)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1474)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1478)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1749)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1481)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1798)
I20260709 12:17:55.970258 32045 rpcz_store.cc:276] 0709 12:17:54.788085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.788190 (+   105us) service_pool.cc:224] Handling call
0709 12:17:55.798845 (+1010655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.883673  1670 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.813033  2012 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.982817 31964 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:55.967806 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.049s	sys 0.048s
W20260709 12:17:56.043277 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.050s	sys 0.049s
I20260709 12:17:55.814771  1581 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:55.866214  2002 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:55.866571  1986 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.011122  2079 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:56.012678 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:56.013919 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 198) took 1337 ms. Trace:
I20260709 12:17:55.592933 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 467) took 1302 ms. Trace:
I20260709 12:17:55.814257  1713 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:55.866986  2004 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.127s
I20260709 12:17:56.012463 31968 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:55.883796  1992 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
W20260709 12:17:56.041314 32010 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.234s (SENT)
I20260709 12:17:56.075559 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:56.062680 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:56.045579  2063 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:17:56.042032 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:55.814574  1974 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:55.814733  2052 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.056664 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 221) took 1604 ms. Trace:
I20260709 12:17:56.020624  2073 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:56.049749 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 83729.8us
I20260709 12:17:56.055888 31971 rpcz_store.cc:276] 0709 12:17:54.676664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.676791 (+   127us) service_pool.cc:224] Handling call
0709 12:17:54.677126 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:54.680847 (+  3721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.680893 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:54.681599 (+   706us) log.cc:844] Serialized 52 byte log entry
0709 12:17:54.781525 (+ 99926us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:54.781577 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.781663 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.011738 (+1230075us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:56.013587 (+  1849us) raft_consensus.cc:1685] finished
0709 12:17:56.013608 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.013852 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1486,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":43119872,"thread_start_us":99444,"threads_started":1,"wal-append.queue_time_us":590895}
I20260709 12:17:56.035761  1966 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.066222 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:56.098623 32120 rpcz_store.cc:276] 0709 12:17:54.452220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.452374 (+   154us) service_pool.cc:224] Handling call
0709 12:17:54.457683 (+  5309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:54.458133 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.458165 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:54.458796 (+   631us) log.cc:844] Serialized 52 byte log entry
0709 12:17:55.239679 (+780883us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:55.441321 (+201642us) log.cc:844] Serialized 14 byte log entry
0709 12:17:55.441582 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.441678 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.056396 (+614718us) raft_consensus.cc:1685] finished
0709 12:17:56.056425 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.056590 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":747129,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":212831,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":146,"thread_start_us":512117,"threads_started":2,"wal-append.queue_time_us":448290}
I20260709 12:17:56.113081 32114 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:56.089309 32116 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:17:56.089418 31964 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:17:56.089517 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 276929us
I20260709 12:17:56.045014  1813 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:56.000268  2070 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:56.092489 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.093257 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:56.013187 32653 rpcz_store.cc:269] 0709 12:17:53.915750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.917940 (+  2190us) service_pool.cc:224] Handling call
0709 12:17:55.596324 (+1678384us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:55.596594 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102059264}
I20260709 12:17:56.013688  2043 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:17:56.014596  1747 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:55.969481  1968 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.097196 32121 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:55.976643  2053 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.104774 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:56.055966 31967 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.979295 32039 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:56.106544 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:56.106729 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 222156us
W20260709 12:17:56.108510 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66567.9us
I20260709 12:17:56.048554  2068 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.089558  1895 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.113983 32040 pending_rounds.cc:85] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 2
I20260709 12:17:56.235764 32040 pending_rounds.cc:107] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:56.236074 32040 raft_consensus.cc:2889] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:55.986075  1983 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:56.122252 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 132565us
I20260709 12:17:56.126566  2026 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.121140 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.090535 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 457) took 2772 ms. Trace:
I20260709 12:17:56.055493  1670 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.090608  2002 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:17:56.223893 32039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1557) took 1381 ms (client timeout 1618 ms). Trace:
I20260709 12:17:56.214903 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:56.224653 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.338066 31969 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:56.338230 31971 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:17:56.167373 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 126086us
I20260709 12:17:56.225179 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.224390 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:56.224745 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.340039 31964 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:56.340627 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1759) took 1104 ms. Trace:
I20260709 12:17:56.114791  1779 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.165665  2070 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.058104  1581 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.128110  1813 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:56.229777  2051 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.108s	user 0.002s	sys 0.004s
W20260709 12:17:56.325300 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 154667us
I20260709 12:17:56.325798  1992 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:17:56.217175  2088 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.182s
I20260709 12:17:56.127458  2057 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.342573  2071 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:56.336970 32039 rpcz_store.cc:269] 0709 12:17:54.842404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.314012 (+471608us) service_pool.cc:224] Handling call
0709 12:17:56.223826 (+909814us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.342329 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1771) took 1041 ms. Trace:
I20260709 12:17:56.405141 31969 rpcz_store.cc:276] 0709 12:17:55.300842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.129571 (+828729us) service_pool.cc:224] Handling call
0709 12:17:56.342250 (+212679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.338943  2026 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.341029 31964 rpcz_store.cc:276] 0709 12:17:55.236705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.129101 (+892396us) service_pool.cc:224] Handling call
0709 12:17:56.340551 (+211450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.185631  2069 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.354758 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:56.355049 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 129925us
I20260709 12:17:56.116704  2073 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.326710  2000 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:56.114786  2012 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.212s
W20260709 12:17:56.395526 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 171194us
W20260709 12:17:56.460075 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 343562us
I20260709 12:17:56.335477  2083 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.402465 32118 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:56.127779  2080 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.249s
I20260709 12:17:56.317559  1895 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.414597 31971 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:56.380854 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.415539  1713 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.130201  2036 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.060379 32656 rpcz_store.cc:276] 0709 12:17:54.242844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.245337 (+  2493us) service_pool.cc:224] Handling call
0709 12:17:54.325785 (+ 80448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:54.355008 (+ 29223us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:54.666141 (+311133us) log.cc:844] Serialized 14 byte log entry
0709 12:17:54.881296 (+215155us) log.cc:844] Serialized 14 byte log entry
0709 12:17:54.926299 (+ 45003us) log.cc:844] Serialized 14 byte log entry
0709 12:17:55.394216 (+467917us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:55.431063 (+ 36847us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:55.437486 (+  6423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.479043 (+ 41557us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.544827 (+ 65784us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":57098,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":112,"raft.queue_time_us":306260,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":22741,"thread_start_us":416509,"threads_started":2,"wal-append.queue_time_us":83130}
I20260709 12:17:56.360721  2085 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.458621 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.471298 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:56.458467 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.245400  2068 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.472195 32039 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:56.333287 31968 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:56.473033 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1795) took 1131 ms. Trace:
I20260709 12:17:56.460057 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.403581  2056 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:56.464180 32113 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:56.464704 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.476454 31964 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 2.
I20260709 12:17:56.476922 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1775) took 1175 ms. Trace:
I20260709 12:17:56.477211 31964 rpcz_store.cc:276] 0709 12:17:55.301807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.416263 (+1114456us) service_pool.cc:224] Handling call
0709 12:17:56.476863 (+ 60600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.332921 32654 rpcz_store.cc:276] 0709 12:17:53.313677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.316878 (+  3201us) service_pool.cc:224] Handling call
0709 12:17:53.397145 (+ 80267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.983692 (+586547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.038437 (+2054745us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b540053edd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:56.062489 (+ 24052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128850304}
I20260709 12:17:56.409868  2090 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:56.474018 31969 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:56.474884  1747 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.473310 31968 rpcz_store.cc:276] 0709 12:17:55.342091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.094631 (+752540us) service_pool.cc:224] Handling call
0709 12:17:56.472988 (+378357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.376618  1813 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.480172 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1782) took 1141 ms. Trace:
I20260709 12:17:56.489583 31969 rpcz_store.cc:276] 0709 12:17:55.338527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.414985 (+1076458us) service_pool.cc:224] Handling call
0709 12:17:56.480123 (+ 65138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.485394 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.469473  2076 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:56.490568 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4286940540000614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
W20260709 12:17:56.497539 32010 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.356s (SENT)
I20260709 12:17:56.495396 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:56.501544 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.502620 31971 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:56.503223 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1772) took 1202 ms. Trace:
I20260709 12:17:56.503729 31971 rpcz_store.cc:276] 0709 12:17:55.301111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.354476 (+1053365us) service_pool.cc:224] Handling call
0709 12:17:56.503167 (+148691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.504683 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:56.505426 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1766) took 1206 ms. Trace:
I20260709 12:17:56.508463 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:56.511270 32039 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:17:56.481544  2070 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.413430  2071 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.506731 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:56.503077 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:17:56.509087 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.481554  2080 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:56.490165  2098 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:56.497319  2076 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:56.515251 32010 connection.cc:256] RPC call timeout handler was delayed by -1.3675920240000323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 12:17:56.551936 32010 connection.cc:256] RPC call timeout handler was delayed by -1.2784920240001156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
W20260709 12:17:56.552389 32010 connection.cc:256] RPC call timeout handler was delayed by -1.2699061440000605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2842s for a response
W20260709 12:17:56.552765 32010 connection.cc:256] RPC call timeout handler was delayed by -1.229406143999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2887s for a response
W20260709 12:17:56.553205 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1268061440000565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30010000000000003s for a response
W20260709 12:17:56.553579 32010 connection.cc:256] RPC call timeout handler was delayed by -1.0993920240000534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
W20260709 12:17:56.553952 32010 connection.cc:256] RPC call timeout handler was delayed by -1.0350061440001355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3103s for a response
W20260709 12:17:56.554344 32010 connection.cc:256] RPC call timeout handler was delayed by -1.0340772190002099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 12:17:56.554724 32010 connection.cc:256] RPC call timeout handler was delayed by -1.0267073060001621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
I20260709 12:17:56.518227 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1760) took 1281 ms. Trace:
I20260709 12:17:56.555758 31964 rpcz_store.cc:276] 0709 12:17:55.237048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.483077 (+1246029us) service_pool.cc:224] Handling call
0709 12:17:56.518144 (+ 35067us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.494840  2056 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.514621 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.557320 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1747) took 1675 ms. Trace:
I20260709 12:17:56.557747 31969 rpcz_store.cc:276] 0709 12:17:54.882474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.501429 (+1618955us) service_pool.cc:224] Handling call
0709 12:17:56.557243 (+ 55814us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:56.516636  2077 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 12:17:56.539278  1974 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.539856  2099 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:56.540273  2092 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:56.496986  1747 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.513057 31971 rpcz_store.cc:276] 0709 12:17:55.299497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.504428 (+1204931us) service_pool.cc:224] Handling call
0709 12:17:56.505379 (+   951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.510000  1996 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.510229  1966 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.506836  2088 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:56.551434 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1788) took 1211 ms. Trace:
I20260709 12:17:56.510282  2039 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.538358  2080 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.466991  2083 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.483819  2090 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.505954  1713 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:17:56.533126  1992 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.519905  2000 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:56.588665 32045 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.012s	sys 0.026s
W20260709 12:17:56.658804 32045 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.014s	sys 0.032s
W20260709 12:17:56.645877 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.002s	sys 0.005s
I20260709 12:17:56.612121 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:56.659629 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.002s	sys 0.005s
I20260709 12:17:56.564764 32039 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:17:56.613250 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:56.643249 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58952.8us
W20260709 12:17:56.566152 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 271276us
I20260709 12:17:56.608886 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1556) took 1767 ms. Trace:
I20260709 12:17:56.614836 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.559437  2085 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:56.652665 31968 rpcz_store.cc:276] 0709 12:17:55.340316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.482157 (+1141841us) service_pool.cc:224] Handling call
0709 12:17:56.551372 (+ 69215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.667398 31963 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:56.667856 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1779) took 1343 ms. Trace:
I20260709 12:17:56.613997 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.659996 32044 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.659958 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.666827 31964 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:56.669411 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1761) took 1432 ms. Trace:
I20260709 12:17:56.589095 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:56.670277 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:56.565842  2074 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 12:17:56.552670  1995 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.671185 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1748) took 1709 ms. Trace:
I20260709 12:17:56.683043 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:56.683090 31969 rpcz_store.cc:276] 0709 12:17:54.962278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.558516 (+1596238us) service_pool.cc:224] Handling call
0709 12:17:56.671112 (+112596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.666059 32043 rpcz_store.cc:276] 0709 12:17:54.842021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.337317 (+495296us) service_pool.cc:224] Handling call
0709 12:17:55.337649 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:55.338038 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:55.338065 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:55.345116 (+  7051us) log.cc:844] Serialized 26 byte log entry
0709 12:17:55.785764 (+440648us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:55.785845 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.785948 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.608669 (+822721us) raft_consensus.cc:1685] finished
0709 12:17:56.608692 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.608821 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1380,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":70704,"threads_started":1,"wal-append.queue_time_us":702336}
I20260709 12:17:56.607015  1895 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:56.606045  2095 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.666957  2000 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:17:56.656471 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:56.690495  2095 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.671800  1581 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.657163  2080 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.692324  2000 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.669972 31971 rpcz_store.cc:276] 0709 12:17:55.237373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.659710 (+1422337us) service_pool.cc:224] Handling call
0709 12:17:56.669347 (+  9637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.668213 31963 rpcz_store.cc:276] 0709 12:17:55.324029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.612998 (+1288969us) service_pool.cc:224] Handling call
0709 12:17:56.667803 (+ 54805us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.565757  2099 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:56.611277  2068 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:17:56.568181  2092 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:56.644606  2071 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.656594  1986 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.687603 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.570559  2052 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.692673  2095 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.692912 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.693164  2094 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.693913 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.607969  1968 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.694558 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.790616 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.795562 31969 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:56.694469  1995 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:56.788168 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:56.611264  2026 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:56.794605 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1637) took 1870 ms. Trace:
I20260709 12:17:56.665107  2039 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:17:56.796208 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 113441us
I20260709 12:17:56.801692 31968 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:56.766669 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:56.663416  1983 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.756138 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:56.799435 32623 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.832s (SENT)
I20260709 12:17:56.799549 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.803706 32039 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:56.804128 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.805819 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.680593  2053 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:56.804641 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:56.801762 32115 rpcz_store.cc:276] 0709 12:17:54.924160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.924322 (+   162us) service_pool.cc:224] Handling call
0709 12:17:54.924782 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:55.239420 (+314638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:55.239449 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:55.239941 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:17:55.814847 (+574906us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:56.094152 (+279305us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.126435 (+ 32283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.126532 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.794347 (+667815us) raft_consensus.cc:1685] finished
0709 12:17:56.794375 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.794532 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20862,"prepare.queue_time_us":247848,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":25998,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":172,"thread_start_us":670089,"threads_started":2,"wal-append.queue_time_us":848406}
I20260709 12:17:56.791589 32656 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:56.789099  2093 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.810489 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:56.810066  2096 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:17:56.811906 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.801446  2094 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:56.687980  2076 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.665652  1992 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.788004  2083 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:56.788137  2090 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:56.919698 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:56.761021 32044 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:56.904896 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1560) took 1882 ms. Trace:
I20260709 12:17:56.933265 32039 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:56.695385 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.878857 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.934245 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1756) took 1698 ms. Trace:
I20260709 12:17:56.935376 32039 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:56.935680 31964 rpcz_store.cc:276] 0709 12:17:55.235670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.669639 (+1433969us) service_pool.cc:224] Handling call
0709 12:17:56.934191 (+264552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.934865 32043 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:56.699972 31967 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
W20260709 12:17:56.937202 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1774) took 1635 ms (client timeout 2020 ms). Trace:
I20260709 12:17:56.933564 32045 rpcz_store.cc:276] 0709 12:17:55.022106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.048877 (+1026771us) service_pool.cc:224] Handling call
0709 12:17:56.049161 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.049343 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:17:56.662423 (+613080us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.904194 (+241771us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.904429 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.904488 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:56.904553 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.904655 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.904821 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":354372,"prepare.run_cpu_time_us":7128,"prepare.run_wall_time_us":41616,"raft.queue_time_us":74791,"spinlock_wait_cycles":29818624,"thread_start_us":742401,"threads_started":2,"wal-append.queue_time_us":137779}
I20260709 12:17:56.812214 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.938218 31971 raft_consensus.cc:2376] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5. Re-sending same reply.
I20260709 12:17:56.936179 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1562) took 1135 ms. Trace:
I20260709 12:17:56.938989 32039 rpcz_store.cc:276] 0709 12:17:55.800546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.934043 (+1133497us) service_pool.cc:224] Handling call
0709 12:17:56.935215 (+  1172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.935961 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.936129 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.812551 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:56.851783 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85212.9us
I20260709 12:17:56.940757 31963 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
W20260709 12:17:56.904095 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53212.4us
I20260709 12:17:56.948364 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:56.949366  2102 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:56.942214 31969 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:56.950248 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:56.942369 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:56.942400 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 142995us
I20260709 12:17:56.850739 32654 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:56.961094 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1770) took 1660 ms. Trace:
I20260709 12:17:56.944983 32039 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:56.961668 31969 rpcz_store.cc:276] 0709 12:17:55.300582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.851276 (+1550694us) service_pool.cc:224] Handling call
0709 12:17:56.961021 (+109745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.807138  1968 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:56.878542  2075 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.008s	sys 0.005s
I20260709 12:17:56.963753 32010 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:17:56.964116 32010 leader_election.cc:403] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:56.965350 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:56.964104 32041 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:56.950239  2102 consensus_queue.cc:1243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:56.966488 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1563) took 1165 ms. Trace:
I20260709 12:17:56.966958 32041 rpcz_store.cc:276] 0709 12:17:55.800790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.935938 (+1135148us) service_pool.cc:224] Handling call
0709 12:17:56.963787 (+ 27849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.966221 (+  2434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.966433 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.967730  2102 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:17:56.968482  2102 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:56.968859  2102 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:56.969215  2102 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.959354  1780 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.971951 32043 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:56.972759 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1565) took 1171 ms. Trace:
I20260709 12:17:56.973253 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1752) took 1780 ms. Trace:
I20260709 12:17:56.973150 32043 rpcz_store.cc:276] 0709 12:17:55.801249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.939401 (+1138152us) service_pool.cc:224] Handling call
0709 12:17:56.960120 (+ 20719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.972387 (+ 12267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.972698 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.973758 31963 rpcz_store.cc:276] 0709 12:17:55.193384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.799435 (+1606051us) service_pool.cc:224] Handling call
0709 12:17:56.973194 (+173759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.938983 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1750) took 1904 ms. Trace:
I20260709 12:17:56.975145 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.975322 31971 rpcz_store.cc:276] 0709 12:17:55.034448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.804988 (+1770540us) service_pool.cc:224] Handling call
0709 12:17:56.938907 (+133919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.976423 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.977291 31971 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:56.979012 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:56.979808 31971 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:56.980512 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1789) took 1639 ms. Trace:
I20260709 12:17:56.980974 31971 rpcz_store.cc:276] 0709 12:17:55.340571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.978700 (+1638129us) service_pool.cc:224] Handling call
0709 12:17:56.980398 (+  1698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.960879 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:56.962522 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1566) took 1161 ms. Trace:
I20260709 12:17:56.984074 32039 rpcz_store.cc:276] 0709 12:17:55.801488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.939843 (+1138355us) service_pool.cc:224] Handling call
0709 12:17:56.944681 (+  4838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.962150 (+ 17469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.962464 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.975962 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1763) took 1738 ms. Trace:
I20260709 12:17:56.689563  2085 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:56.937642 31967 rpcz_store.cc:269] 0709 12:17:55.301569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.012355 (+710786us) service_pool.cc:224] Handling call
0709 12:17:56.937148 (+924793us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.986884 31964 rpcz_store.cc:276] 0709 12:17:55.238054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.936655 (+1698601us) service_pool.cc:224] Handling call
0709 12:17:56.975905 (+ 39250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.988042 32043 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:56.986945 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:56.989598 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.009189 32039 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:57.010551 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.010089 32044 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:17:56.950495  2012 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:56.850591  2073 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.012392 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.021097 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1786) took 1681 ms. Trace:
I20260709 12:17:57.021607 31964 rpcz_store.cc:276] 0709 12:17:55.339763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.012171 (+1672408us) service_pool.cc:224] Handling call
0709 12:17:57.021039 (+  8868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.911747  2080 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:57.014083 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48800us
I20260709 12:17:57.013643 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.015089 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.013593 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.016960 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1567) took 1215 ms. Trace:
I20260709 12:17:57.025991 31969 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.025921 32044 rpcz_store.cc:276] 0709 12:17:55.801731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.960835 (+1159104us) service_pool.cc:224] Handling call
0709 12:17:57.009796 (+ 48961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.016700 (+  6904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.016892 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.009546  2050 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:17:56.988480  1995 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.028942  2050 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:57.029314  2050 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:56.878135  2092 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:56.941608  2093 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.019374  2039 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:57.017036 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76026.4us
I20260709 12:17:57.017844 31967 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:56.986133  2076 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.018471  2114 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:57.018505 31971 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:56.659094  2088 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.047384 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:56.941885  1983 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.023067 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.023499  2066 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:57.020650  1997 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:57.042100 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.019759  2012 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:57.043321  2045 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.022375  1971 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.051251 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:57.052726 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:57.088743 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.089594 31969 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:57.089918 31967 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:17:57.090071 32116 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:57.080158 31964 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.075882 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:57.091418 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:57.045852  2114 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.075688 32117 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 9 index: 3. (index mismatch)
I20260709 12:17:57.086769 31935 leader_election.cc:400] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:17:57.050901  1933 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:57.062898 32113 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:57.097201 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1794) took 1755 ms. Trace:
I20260709 12:17:57.062006  1997 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.097631 31967 rpcz_store.cc:276] 0709 12:17:55.341824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.059302 (+1717478us) service_pool.cc:224] Handling call
0709 12:17:57.097135 (+ 37833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.059244  2066 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:57.076778  2000 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:57.087896  2012 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.106041 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.088109  2045 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.988756  2099 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.030082  2050 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.119259  2066 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.120150  2088 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.095525  2114 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.122556 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1568) took 1320 ms. Trace:
W20260709 12:17:57.133491 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 92267.9us
I20260709 12:17:57.120572  1996 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:57.148761  2066 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.122880  2045 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.139356 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.155479 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.102012  1997 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.106195  2000 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:57.158546  2000 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:57.160218  2054 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 2.369s	user 0.007s	sys 0.005s
I20260709 12:17:57.145509 32041 rpcz_store.cc:276] 0709 12:17:55.801956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.967727 (+1165771us) service_pool.cc:224] Handling call
0709 12:17:56.968032 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.968369 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:56.968672 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.088639 (+119967us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.122217 (+ 33578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.122270 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:57.122329 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.122391 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.122502 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45193,"prepare.queue_time_us":106850,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":62980,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":7399,"thread_start_us":109701,"threads_started":2,"wal-append.queue_time_us":46318}
I20260709 12:17:57.147276 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.166795  2099 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.168323 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.169453  2097 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:57.170176  2097 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.122376  2105 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:57.170912  2116 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:57.172544  2085 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.173382  2093 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.146356  2117 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:57.100066  1933 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.174783  2050 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.176885  2045 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.177716  1933 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.179286  1997 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:57.184535  1813 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.000s
I20260709 12:17:57.185371  1813 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.187453  2080 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.188714  1813 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.182184 31964 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:57.191184 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.192297 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1797) took 1767 ms. Trace:
I20260709 12:17:57.192869 31964 rpcz_store.cc:276] 0709 12:17:55.425100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.190679 (+1765579us) service_pool.cc:224] Handling call
0709 12:17:57.192223 (+  1544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.178710 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1471) took 2779 ms. Trace:
I20260709 12:17:57.179874  1992 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:57.180280 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74318.2us
I20260709 12:17:57.194450 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.182413 31967 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 2.
I20260709 12:17:57.195149 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:57.195927 32116 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:57.196046 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.196611 31964 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:57.204025 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:57.204945 32115 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:17:57.194299 31970 rpcz_store.cc:276] 0709 12:17:54.399686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.399844 (+   158us) service_pool.cc:224] Handling call
0709 12:17:54.412878 (+ 13034us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:54.450351 (+ 37473us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:54.450398 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:54.450966 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 12:17:55.248483 (+797517us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:55.248523 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.248587 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.159585 (+1910998us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:57.178396 (+ 18811us) raft_consensus.cc:1685] finished
0709 12:17:57.178428 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.178629 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19186,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3348,"spinlock_wait_cycles":3609472,"thread_start_us":285367,"threads_started":1,"wal-append.queue_time_us":289535}
I20260709 12:17:57.208647 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:57.209334 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 164092us
I20260709 12:17:57.188760  2045 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.188661  1997 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.253706 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.254436 31970 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:57.255867  2081 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.255998 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:57.256484  2081 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:57.257004 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43608us
I20260709 12:17:57.258488  1992 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:57.258912  1992 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:57.259367  1992 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.262645 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1564) took 1461 ms. Trace:
I20260709 12:17:57.263025 32045 rpcz_store.cc:276] 0709 12:17:55.801028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.938126 (+1137098us) service_pool.cc:224] Handling call
0709 12:17:56.960528 (+ 22402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.971763 (+ 11235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.971793 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:56.972325 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.099130 (+126805us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.099167 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.099240 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.262432 (+163192us) raft_consensus.cc:1685] finished
0709 12:17:57.262451 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.262582 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2489,"raft.queue_time_us":1077,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":21621,"thread_start_us":126446,"threads_started":1,"wal-append.queue_time_us":239134}
I20260709 12:17:57.257954 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:57.297442 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.303563 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45799.4us
I20260709 12:17:57.319249 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.321235 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.329547 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.330201 31970 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:57.320415  1813 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.334719 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:57.336639 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 168374us
I20260709 12:17:57.337491 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.330513  2099 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:17:57.342257 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.342904 31970 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:57.343772 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.344321 31970 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 2.
I20260709 12:17:57.348065 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.331218  2066 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.320155  2000 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.322461  1933 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:17:57.353116  2099 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:57.354592  2098 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.354719  2094 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.353175 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.341811  2093 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:57.361656 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.366395  2098 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:57.367383  2114 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.332976  2012 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.341922  2102 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.383422  2098 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.388128 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1769) took 2087 ms. Trace:
I20260709 12:17:57.393491 31970 rpcz_store.cc:276] 0709 12:17:55.300326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.361401 (+2061075us) service_pool.cc:224] Handling call
0709 12:17:57.388066 (+ 26665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.364564  2099 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:57.399252  2012 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.405279 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.407873 32045 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:57.406749 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.375319  2112 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:57.375847  2093 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:57.413440 32043 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:57.421765 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.424292 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.428880 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.429831 31935 leader_election.cc:400] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:17:57.430546 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.422441  2093 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.432942 32045 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:57.433172 31935 leader_election.cc:403] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:17:57.440214 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.440922 32045 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:57.443312 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.443998 32043 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:57.447108  2114 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.448275 31964 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:57.449007 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1474) took 2762 ms. Trace:
I20260709 12:17:57.449301 31964 rpcz_store.cc:276] 0709 12:17:54.687046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.447824 (+2760778us) service_pool.cc:224] Handling call
0709 12:17:57.448114 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.448733 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.448953 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.458747 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.436942  2066 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:17:57.445564  2039 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:57.474459 32045 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:57.476863 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1475) took 2768 ms. Trace:
I20260709 12:17:57.479215 31964 rpcz_store.cc:276] 0709 12:17:54.688247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.450117 (+2761870us) service_pool.cc:224] Handling call
0709 12:17:57.456096 (+  5979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.456436 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:17:57.456463 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.456506 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:57.456550 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.456606 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.456713 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":1417}
I20260709 12:17:57.479909 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.480019 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:57.481683 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1476) took 2791 ms. Trace:
I20260709 12:17:57.484392 31964 rpcz_store.cc:276] 0709 12:17:54.689882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.480627 (+2790745us) service_pool.cc:224] Handling call
0709 12:17:57.480893 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.481167 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.481194 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.481242 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.481299 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.481510 (+   211us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.481628 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.487911 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1478) took 2795 ms. Trace:
I20260709 12:17:57.470036  2098 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.488236 31964 rpcz_store.cc:276] 0709 12:17:54.692309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.487188 (+2794879us) service_pool.cc:224] Handling call
0709 12:17:57.487446 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.487603 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.487628 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.487667 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:57.487701 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.487751 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.487848 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.481146  2076 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.481083  2086 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:57.497159 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.497874  2076 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.500063  2076 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.504262 32045 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:57.503011  2000 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.504956  1933 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:57.505568  1933 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:57.505965  2102 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.506498  1933 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.508409  2098 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.455144  2099 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.522145 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:57.522954  2099 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.514439  2081 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.496842  2066 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:57.481422  2039 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.519222 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 469) took 2075 ms. Trace:
I20260709 12:17:57.519708 31963 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:57.523095 32656 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:17:57.523579  2014 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 3.783s	user 0.012s	sys 0.025s
I20260709 12:17:57.526345  2066 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:57.526824  2066 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.524417 32655 rpcz_store.cc:276] 0709 12:17:55.444049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.445872 (+  1823us) service_pool.cc:224] Handling call
0709 12:17:55.446223 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.128971 (+682748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.129006 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.129048 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:56.129085 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.519008 (+1389923us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b5400550ad4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:57.519052 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.519165 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569669504}
I20260709 12:17:57.527626 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.531610 32043 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:17:57.518970  1992 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.534628 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1632) took 3875 ms. Trace:
I20260709 12:17:57.534871 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.538033 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1473) took 2852 ms. Trace:
I20260709 12:17:57.538193 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:57.536724 31972 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1781) took 2207 ms (client timeout 1955 ms). Trace:
I20260709 12:17:57.537189 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1570) took 1011 ms. Trace:
I20260709 12:17:57.536098  2039 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.537578 32112 rpcz_store.cc:276] 0709 12:17:53.655411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.655532 (+   121us) service_pool.cc:224] Handling call
0709 12:17:53.655842 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:53.679115 (+ 23273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:53.679144 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:53.679697 (+   553us) log.cc:844] Serialized 52 byte log entry
0709 12:17:53.709854 (+ 30157us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:53.709897 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.709969 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.840911 (+130942us) spinlock_profiling.cc:243] Waited 8.67 ms on lock 0x7b48002301d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:17:55.619255 (+1778344us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b1800011980. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db
0709 12:17:57.523001 (+1903746us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:57.530400 (+  7399us) raft_consensus.cc:1685] finished
0709 12:17:57.530422 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.530565 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19192,"raft.queue_time_us":18828,"raft.run_cpu_time_us":3495,"raft.run_wall_time_us":159707,"spinlock_wait_cycles":40542976,"thread_start_us":29790,"threads_started":1,"wal-append.queue_time_us":46687}
I20260709 12:17:57.539126 32039 rpcz_store.cc:276] 0709 12:17:56.526042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.989678 (+463636us) service_pool.cc:224] Handling call
0709 12:17:57.008913 (+ 19235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.016032 (+  7119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.537005 (+520973us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b54000569d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:57.537135 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4632832}
W20260709 12:17:57.539029 31972 rpcz_store.cc:269] 0709 12:17:55.329216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.536531 (+2207315us) service_pool.cc:211] Skipping call since client already timed out
0709 12:17:57.536656 (+   125us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:17:57.538806 31970 rpcz_store.cc:276] 0709 12:17:54.685535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.409202 (+2723667us) service_pool.cc:224] Handling call
0709 12:17:57.418866 (+  9664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.419014 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:57.478986 (+ 59972us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.537590 (+ 58604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.537654 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.537729 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.537819 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.537964 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":92854,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":109536,"threads_started":2}
I20260709 12:17:57.541944  1983 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:57.542114  2000 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:57.543980  1983 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:57.544353  1983 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:57.541585 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:57.547923 31970 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:57.548514 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1749) took 2520 ms. Trace:
I20260709 12:17:57.548818 31970 rpcz_store.cc:276] 0709 12:17:55.028404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.546753 (+2518349us) service_pool.cc:224] Handling call
0709 12:17:57.547762 (+  1009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.548301 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.548465 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.550061 31970 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:57.550599 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1765) took 2251 ms. Trace:
I20260709 12:17:57.550909 31970 rpcz_store.cc:276] 0709 12:17:55.299159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.549521 (+2250362us) service_pool.cc:224] Handling call
0709 12:17:57.549768 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.550388 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.550552 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:57.553074  2113 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.463s	user 0.003s	sys 0.001s
I20260709 12:17:57.554494 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1559) took 2627 ms. Trace:
I20260709 12:17:57.554999 31972 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:17:57.555569 31972 pending_rounds.cc:85] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 4
I20260709 12:17:57.556044 31972 pending_rounds.cc:107] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:17:57.556380 31972 raft_consensus.cc:2889] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.554833 32040 rpcz_store.cc:276] 0709 12:17:54.926697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.046430 (+1119733us) service_pool.cc:224] Handling call
0709 12:17:56.113773 (+ 67343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:56.791758 (+677985us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b18004215a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1c7bfb 00007f228c1d264b
0709 12:17:56.796450 (+  4692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.796477 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:56.803997 (+  7520us) log.cc:844] Serialized 52 byte log entry
0709 12:17:57.007622 (+203625us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:57.007673 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.007765 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.552466 (+544701us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:17:57.554275 (+  1809us) raft_consensus.cc:1685] finished
0709 12:17:57.554294 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.554440 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3255,"raft.queue_time_us":129060,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":11633,"spinlock_wait_cycles":11929088,"thread_start_us":276840,"threads_started":2,"wal-append.queue_time_us":159627}
I20260709 12:17:57.556799  1904 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.557088 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1745) took 2763 ms. Trace:
I20260709 12:17:57.546120  2105 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:57.557634  2105 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.557468 31972 rpcz_store.cc:276] 0709 12:17:54.794077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.541125 (+2747048us) service_pool.cc:224] Handling call
0709 12:17:57.554636 (+ 13511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.556851 (+  2215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.557038 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.558182  2105 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:57.542397  2086 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:17:57.559774 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 468) took 2116 ms. Trace:
I20260709 12:17:57.560148 32652 rpcz_store.cc:276] 0709 12:17:55.443713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.443859 (+   146us) service_pool.cc:224] Handling call
0709 12:17:55.596551 (+152692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.812980 (+216429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.559601 (+1746621us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540063e550. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:57.559721 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350178560}
W20260709 12:17:57.561344 32047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1542) took 2843 ms (client timeout 1756 ms). Trace:
W20260709 12:17:57.561733 32047 rpcz_store.cc:269] 0709 12:17:54.702633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.705844 (+  3211us) service_pool.cc:224] Handling call
0709 12:17:57.545450 (+2839606us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400056ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:57.545546 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443430016}
I20260709 12:17:57.552650  1983 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.567414 31972 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:57.567373  1983 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.568207 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1481) took 2241 ms. Trace:
I20260709 12:17:57.568576 31972 rpcz_store.cc:276] 0709 12:17:55.326422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.566468 (+2240046us) service_pool.cc:224] Handling call
0709 12:17:57.567060 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.567947 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.568148 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.569979 31972 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:57.572064  1670 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.573025 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1561) took 1772 ms. Trace:
W20260709 12:17:57.574061 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.574942 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.577848 32085 connection.cc:256] RPC call timeout handler was delayed by -1.1592930639999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22860000000000003s for a response
I20260709 12:17:57.579879  2105 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.582044  2105 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.584242 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.592265 32657 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:17:57.578248 32085 connection.cc:256] RPC call timeout handler was delayed by -1.1142590169999949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 12:17:57.594825 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1482) took 2266 ms. Trace:
I20260709 12:17:57.595255 31972 rpcz_store.cc:276] 0709 12:17:55.328720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.569356 (+2240636us) service_pool.cc:224] Handling call
0709 12:17:57.569659 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.570296 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.594766 (+ 24470us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.584969 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.596946 32659 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:57.573396 32038 rpcz_store.cc:276] 0709 12:17:55.800299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.086475 (+286176us) service_pool.cc:224] Handling call
0709 12:17:56.097650 (+ 11175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.097844 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:56.326826 (+228982us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.572177 (+1245351us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b1800421540. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a
0709 12:17:57.572564 (+   387us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.572729 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.572772 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.572817 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.572875 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.572972 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":380737,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":69,"raft.queue_time_us":1147,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":127,"spinlock_wait_cycles":5346176,"thread_start_us":377288,"threads_started":2,"wal-append.queue_time_us":317194}
I20260709 12:17:57.585620 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.586730 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:57.587406 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.588481  2095 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.589043  2086 consensus_queue.cc:1243] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:17:57.589860 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48355.1us
I20260709 12:17:57.603684 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.585358 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.606500 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.608254 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.600000 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.601670 31964 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:57.602348 32652 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:17:57.609117 32047 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:57.610908 31972 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:17:57.611313 31972 pending_rounds.cc:85] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 3
I20260709 12:17:57.611594 31972 pending_rounds.cc:107] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:17:57.611889 31972 raft_consensus.cc:2889] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.612150 32085 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.614362  2081 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:57.614764  2081 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:57.615089  2004 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:57.615106  2081 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.615439 31971 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:57.615713  2004 raft_consensus.cc:1068] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Unable to promote non-voter ec7f26ea46944ea498ad253b1b1819b0: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:57.616050 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1785) took 2276 ms. Trace:
I20260709 12:17:57.616380 31971 rpcz_store.cc:276] 0709 12:17:55.339457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.614835 (+2275378us) service_pool.cc:224] Handling call
0709 12:17:57.615153 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.615819 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.616000 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.616719 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:57.617928 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:57.623195 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.624682 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:57.625818 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.532444  2093 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.621111 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.632907 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.633702 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1483) took 2299 ms. Trace:
I20260709 12:17:57.634325 31972 rpcz_store.cc:276] 0709 12:17:55.334405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.596009 (+2261604us) service_pool.cc:224] Handling call
0709 12:17:57.610500 (+ 14491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.633473 (+ 22973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.633648 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.613075 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:57.635004 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.635635 32047 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:17:57.633924 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.638250 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.610239  2086 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:57.652132  2086 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:57.652558  1813 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:57.652756  2086 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.653040  1813 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:57.644153  2114 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.656011  2114 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.656495  2114 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.656991 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.657588 31968 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.650823 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1773) took 2349 ms. Trace:
I20260709 12:17:57.619654 32656 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:57.632474  2105 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:57.658877 31966 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:17:57.639492  1933 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:57.664249 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.658793 31967 rpcz_store.cc:276] 0709 12:17:55.301346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.155206 (+1853860us) service_pool.cc:224] Handling call
0709 12:17:57.650634 (+495428us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54000b6154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:57.650763 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59851520}
I20260709 12:17:57.665001 31969 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:57.665868  1995 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:57.665977 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.632721  2095 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:57.666453  1995 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:57.666062 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.666941  1995 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.667531 31972 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:57.639634  2079 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.667802 31969 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.618388  1983 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.669445 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.670068  1983 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.670478  1983 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.672363 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.674178 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:57.639631  2076 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:57.672556 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.677357 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1753) took 2430 ms (client timeout 1514 ms). Trace:
I20260709 12:17:57.679981 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.680437 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.680797 31967 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:57.681842 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.682664  2088 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:57.644546  2004 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:57.683779 31967 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 9 index: 3. (term mismatch)
W20260709 12:17:57.677770 31966 rpcz_store.cc:269] 0709 12:17:55.234253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.274338 (+ 40085us) service_pool.cc:224] Handling call
0709 12:17:57.658580 (+2384242us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b54001e7c28. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fac5f4 00007f2291195227 00007f2291194ab8 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6
0709 12:17:57.664370 (+  5790us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3205888}
I20260709 12:17:57.684146 31967 pending_rounds.cc:85] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 2
I20260709 12:17:57.684389 31967 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 8.3
I20260709 12:17:57.684623 31967 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.676635 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.685590 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1801) took 1562 ms. Trace:
I20260709 12:17:57.685900 31967 rpcz_store.cc:276] 0709 12:17:56.122983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.681863 (+1558880us) service_pool.cc:224] Handling call
0709 12:17:57.682155 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.685292 (+  3137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.685466 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.674863 32653 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.690007 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 555) took 1027 ms. Trace:
I20260709 12:17:57.689481 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.691128  2088 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.696499  2009 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:57.696612  2088 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.696974  2009 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:57.697151  2088 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.697461  2009 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:57.653987  1813 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.644428  2050 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:17:57.658119  2081 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.677210  2076 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.713097 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.722894 32047 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:17:57.691679 32651 rpcz_store.cc:276] 0709 12:17:56.662668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.662807 (+   139us) service_pool.cc:224] Handling call
0709 12:17:56.683192 (+ 20385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.078752 (+395560us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:57.326750 (+247998us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.433712 (+106962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.433783 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:57.433852 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.682039 (+248187us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b18002bc120. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419
0709 12:17:57.689723 (+  7684us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400592254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:57.689789 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.689953 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17169,"prepare.queue_time_us":39850,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":97,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":2212,"spinlock_wait_cycles":907861632,"thread_start_us":327625,"threads_started":2,"wal-append.queue_time_us":150783}
I20260709 12:17:57.719440 31966 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:57.725417 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1484) took 1184 ms. Trace:
I20260709 12:17:57.725848 31966 rpcz_store.cc:276] 0709 12:17:56.540988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.695485 (+1154497us) service_pool.cc:224] Handling call
0709 12:17:57.719264 (+ 23779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.725063 (+  5799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.725332 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.687652  2004 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:17:57.728621 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 63845.6us
I20260709 12:17:57.730907 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:57.727392  2076 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:57.735095 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.736769 32047 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:57.739526 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:57.727573  2004 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.712255  1971 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 12:17:57.775388 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.781126 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.735563  1813 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.730207  2079 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 12:17:57.784039 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53196.3us
I20260709 12:17:57.787855  2093 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:17:57.788368  2133 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:17:57.789307  2134 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:17:57.790798  1983 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:57.792440 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56649.7us
W20260709 12:17:57.793740 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.794080 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.750187  1992 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.755795  1898 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:57.800848 31969 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:57.779671  2050 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.779472  2039 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:17:57.812842 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73450.4us
I20260709 12:17:57.758662 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1480) took 2432 ms. Trace:
I20260709 12:17:57.810515 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.813326 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.759001  2105 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:57.814345 31970 rpcz_store.cc:276] 0709 12:17:55.326002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.551570 (+2225568us) service_pool.cc:224] Handling call
0709 12:17:57.551813 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.552004 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:57.590796 (+ 38792us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.660967 (+ 70171us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.758340 (+ 97373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.758390 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:57.758438 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.758497 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.758608 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39492,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":1910,"raft.queue_time_us":27547,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":10858,"spinlock_wait_cycles":2285440,"thread_start_us":166750,"threads_started":2,"wal-append.queue_time_us":107843}
I20260709 12:17:57.816439 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.818580 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.819849 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.822233 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.824927 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.819201 31970 raft_consensus.cc:2376] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:17:57.825515 31969 raft_consensus.cc:2376] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:17:57.827044 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:57.828100 32040 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:57.827327 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.830914 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:57.813037  2039 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.833016 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:57.834692  2136 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:17:57.830124  1670 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:57.842700  1823 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:57.842963  1670 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:57.843320  1823 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:57.843711  2081 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.844065  1670 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.844257  1823 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.846429  2039 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.852558 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.849100 31968 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:57.851040 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.852448 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1492) took 1280 ms. Trace:
I20260709 12:17:57.854964 31963 rpcz_store.cc:276] 0709 12:17:56.571587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.975095 (+403508us) service_pool.cc:224] Handling call
0709 12:17:57.852222 (+877127us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54000b6654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:57.852376 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136944000}
I20260709 12:17:57.856088 31971 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:57.828691  1996 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:57.855084 32660 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:17:57.866833 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.855814 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.868431  1996 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:57.868671  2099 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.868976  1996 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.869220  2099 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.871685  2099 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:57.872429 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.873178 32653 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:17:57.875823 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 552) took 1301 ms. Trace:
I20260709 12:17:57.876555 32653 rpcz_store.cc:276] 0709 12:17:56.574863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.658180 (+ 83317us) service_pool.cc:224] Handling call
0709 12:17:57.674021 (+1015841us) spinlock_profiling.cc:243] Waited 61.5 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aae9c 00007f22890aaa5b 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:17:57.875757 (+201736us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141446528}
I20260709 12:17:57.875278 31969 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:17:57.878489 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1487) took 1336 ms. Trace:
I20260709 12:17:57.878942 31969 rpcz_store.cc:276] 0709 12:17:56.541807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.851525 (+1309718us) service_pool.cc:224] Handling call
0709 12:17:57.875043 (+ 23518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.878196 (+  3153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.878438 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.880280 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.881074 31969 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.881409 31963 pending_rounds.cc:85] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 2
I20260709 12:17:57.884411 31963 pending_rounds.cc:107] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:17:57.884858 31963 raft_consensus.cc:2889] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.886124 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1489) took 1343 ms. Trace:
I20260709 12:17:57.873186 32040 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 5: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:57.887192 31968 rpcz_store.cc:276] 0709 12:17:56.542300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.862092 (+1319792us) service_pool.cc:224] Handling call
0709 12:17:57.884889 (+ 22797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.885332 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.885406 (+    74us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.885521 (+   115us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.885596 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.885729 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.885988 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.883463 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.894729 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.896041 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:57.896643 32652 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.832126  2037 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:17:57.897823  2004 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.898662 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:57.898494  2066 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.900460  2004 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:57.900537  2092 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.900729  2105 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:57.900568 32040 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:57.903851  2132 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.216s	user 0.000s	sys 0.002s
I20260709 12:17:57.904469  2067 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:57.973584 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.975097 32039 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:57.905459 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:17:57.922188 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 71514.1us
I20260709 12:17:57.833330  1995 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:57.947119 31965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1767) took 2647 ms (client timeout 1938 ms). Trace:
W20260709 12:17:57.980103 31965 rpcz_store.cc:269] 0709 12:17:55.299821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.924896 (+2625075us) service_pool.cc:211] Skipping call since client already timed out
0709 12:17:57.947037 (+ 22141us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:17:57.958036 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.008847 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.010444 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.010602 32658 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:58.012198 32044 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:58.013813 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.014415 32044 raft_consensus.cc:2376] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7. Re-sending same reply.
I20260709 12:17:58.013378  2088 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.019941 32044 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:58.020308  2088 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.021564 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.022683 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.020923  2039 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.023272 32042 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:58.024487 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.025032 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:58.030830  2080 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.031229  2080 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.021886 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:57.971390 31969 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:57.947330 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:57.978325 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.979277 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:57.981683 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.981822 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.983479  1933 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:57.983956 31997 heartbeater.cc:646] Failed to heartbeat to 127.30.25.62:33695 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.30.25.62:33695 timed out after 15.000s (SENT)
I20260709 12:17:57.984835  1995 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:57.986070  1996 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.986717 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:57.978790 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:58.065807 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 87097.6us
I20260709 12:17:57.989750 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.990533 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:57.992060  2114 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.994078  2086 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:57.994596 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:58.078799 31965 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:57.999118 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:17:58.080441 32010 leader_election.cc:336] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.918s (SENT)
I20260709 12:17:58.080870 31968 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:17:58.081261 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:57.999817  2066 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.001304  2094 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:58.087625 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 140389us
W20260709 12:17:58.087971 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66171.9us
I20260709 12:17:58.075050  2095 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:58.057241 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 204) took 3321 ms. Trace:
I20260709 12:17:58.090058 31964 rpcz_store.cc:276] 0709 12:17:54.736108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.493464 (+2757356us) service_pool.cc:224] Handling call
0709 12:17:57.493722 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.609099 (+115377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.056917 (+447818us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400002154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:58.057167 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247880576}
I20260709 12:17:58.092046 31964 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:58.060752 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:58.094139 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 217) took 1077 ms. Trace:
I20260709 12:17:58.094602 31967 rpcz_store.cc:276] 0709 12:17:57.016860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.093184 (+1076324us) service_pool.cc:224] Handling call
0709 12:17:58.093523 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.093716 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.093755 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:58.093810 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:58.093868 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.093943 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.094074 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.307629 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:57.989009 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1636) took 3065 ms (client timeout 3368 ms). Trace:
W20260709 12:17:58.096637 32121 rpcz_store.cc:269] 0709 12:17:54.913711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.918419 (+  4708us) service_pool.cc:224] Handling call
0709 12:17:57.978496 (+3060077us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54003e4ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:57.978609 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944576896}
I20260709 12:17:58.067611 32038 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
W20260709 12:17:58.053308 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.076052 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:58.100152 31970 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:17:58.033336 32660 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:58.046926 32039 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:58.044329 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.061450  1933 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.977294 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:58.081866 32655 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:58.083792 32120 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:58.082783 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.004622 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1798) took 2563 ms. Trace:
I20260709 12:17:58.108481 31971 rpcz_store.cc:276] 0709 12:17:55.440971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.617189 (+2176218us) service_pool.cc:224] Handling call
0709 12:17:57.852372 (+235183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.861769 (+  9397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.004391 (+142622us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54000b6650. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:58.004551 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540172160}
I20260709 12:17:58.105412 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 209) took 1190 ms. Trace:
I20260709 12:17:58.107523 31968 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:58.111577 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 208) took 1241 ms. Trace:
I20260709 12:17:58.111805 31967 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:58.111286 31964 rpcz_store.cc:276] 0709 12:17:56.915045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.091356 (+1176311us) service_pool.cc:224] Handling call
0709 12:17:58.091754 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.105081 (+ 13327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.105343 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:58.081481  2045 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:58.119872 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.081777  2144 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:58.111969 31968 rpcz_store.cc:276] 0709 12:17:56.870375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.081471 (+1211096us) service_pool.cc:224] Handling call
0709 12:17:58.107214 (+ 25743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.111335 (+  4121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.111525 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:58.118402 31971 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:58.116290 31964 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:58.112341 31967 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:58.131227 32655 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:17:58.122047 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77824.2us
I20260709 12:17:58.132982 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.134155 32652 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:58.087929  2073 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.122680  1780 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:17:58.087706  2094 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:58.135066 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:58.138864 31968 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:58.115733  2039 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.155117 31965 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:58.127799 31967 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.143280 31971 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:58.075093  2092 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:17:58.145670 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.124245  2045 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.135460  1971 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:58.136067  2095 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.139375 31964 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:58.150048 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:58.175050 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.120700  2142 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:17:58.032071  2080 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.149457 31968 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:58.166594 32039 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:58.115785  1904 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:17:58.119503  2098 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.146742  2144 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.149895 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:58.115919  2069 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:17:58.173272 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1488) took 1631 ms. Trace:
W20260709 12:17:58.174324 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 215640us
I20260709 12:17:58.178198 31967 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:58.180886 31968 pending_rounds.cc:85] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 1
W20260709 12:17:58.167660 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 78979.1us
I20260709 12:17:58.188127 31964 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:58.148361  2073 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.208789 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:58.194722 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:58.204150 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.205440 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.213954 32039 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:58.208465 31968 pending_rounds.cc:107] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:17:58.209219 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.214951 31968 raft_consensus.cc:2889] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:58.172842  1823 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:58.206506 31963 rpcz_store.cc:276] 0709 12:17:56.542050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.861615 (+1319565us) service_pool.cc:224] Handling call
0709 12:17:57.881177 (+ 19562us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:57.885573 (+  4396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.885609 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:57.886147 (+   538us) log.cc:844] Serialized 52 byte log entry
0709 12:17:58.172814 (+286667us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:58.172871 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.172968 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.173027 (+    59us) raft_consensus.cc:1685] finished
0709 12:17:58.173045 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.173200 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4373,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":41969,"thread_start_us":286189,"threads_started":1,"wal-append.queue_time_us":280288}
I20260709 12:17:58.218600 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.218814 31964 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:58.220952 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.172866  1971 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.223953 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.225416 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.227998  2045 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.229162  2045 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:58.230686 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:17:58.230913  2145 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:17:58.231765  2088 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:58.233291  2155 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.234104  2090 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:58.234426 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.234941  1996 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:58.234997 32039 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:17:58.201905  2066 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.237046  2066 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:58.237418  2066 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.239434  2039 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:58.239756 31967 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:58.240378 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1499) took 1180 ms. Trace:
I20260709 12:17:58.240666 31967 rpcz_store.cc:276] 0709 12:17:57.060004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.095766 (+1035762us) service_pool.cc:224] Handling call
0709 12:17:58.096179 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.240123 (+143944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.240317 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:58.240821  2079 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:58.205521  2067 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:58.241849  2079 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:58.242226  2079 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.179831  2076 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.243340  2081 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:58.243487  2155 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:58.243927  2081 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.243993  2155 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:58.244534  2155 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.244974  2081 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.246613  1971 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:58.236388  2009 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.002s
I20260709 12:17:58.247889  1983 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.222169 32654 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9. Re-sending same reply.
I20260709 12:17:58.290360 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.291267 31964 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.243007  2079 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.242245  2067 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.246662  2144 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.250412  2146 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.256477 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1485) took 1715 ms. Trace:
I20260709 12:17:58.257562 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:58.260627 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.268018 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.296263 31972 rpcz_store.cc:276] 0709 12:17:56.541283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.714812 (+1173529us) service_pool.cc:224] Handling call
0709 12:17:57.730212 (+ 15400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.730562 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.730592 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.731015 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.810794 (+ 79779us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.810833 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.810896 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.256274 (+445378us) raft_consensus.cc:1685] finished
0709 12:17:58.256293 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.256417 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":1637,"thread_start_us":52541,"threads_started":1,"wal-append.queue_time_us":215640}
I20260709 12:17:58.268708 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.257530 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:17:58.270507  2039 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:58.270589  1823 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.271157 32047 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [5.3-7.4]
W20260709 12:17:58.272979  2162 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 12:17:58.284905 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 206) took 1535 ms. Trace:
I20260709 12:17:58.273243  2073 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.319792 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:17:58.299099 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:58.298591 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.255414  2080 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.304944 31965 rpcz_store.cc:276] 0709 12:17:56.749566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.079457 (+1329891us) service_pool.cc:224] Handling call
0709 12:17:58.108069 (+ 28612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.155668 (+ 47599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.284606 (+128938us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b54001e96d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:58.284827 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107758336}
W20260709 12:17:58.319985 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62522.9us
I20260709 12:17:58.327109 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:58.293673  2079 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:58.321362 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.322365 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.320497 32046 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9. Re-sending same reply.
I20260709 12:17:58.328667 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:58.302774 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.320441 31964 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:58.329684 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.338481 32047 raft_consensus.cc:2376] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8. Re-sending same reply.
I20260709 12:17:58.338905 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.339660 31963 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:58.342949 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 260643us
I20260709 12:17:58.302013  2146 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:58.300289  2142 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.344762 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.352144 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.347471 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.348177 31964 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:58.348141 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.349126 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.350558 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.354804 32121 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:58.355382 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.355948 32045 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.356312 31964 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:58.350633 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.361274 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.327358  2114 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:58.335251  2090 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:58.332485  2152 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:58.296624  2157 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:17:58.292759  2009 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.255084  2094 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.397850  2094 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:58.373374 32121 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:58.365226 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:58.401811 32038 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:58.373777 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.404044 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1500) took 1341 ms. Trace:
I20260709 12:17:58.352488  2146 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.404515 31970 rpcz_store.cc:276] 0709 12:17:57.062915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.100981 (+1038066us) service_pool.cc:224] Handling call
0709 12:17:58.101355 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.110414 (+  9059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.110481 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:58.127169 (+ 16688us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.329682 (+202513us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:58.329717 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.329777 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.403796 (+ 74019us) raft_consensus.cc:1685] finished
0709 12:17:58.403818 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.403980 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4673,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5054,"thread_start_us":149898,"threads_started":1,"wal-append.queue_time_us":205760}
I20260709 12:17:58.361765  2142 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.406658 32121 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:58.407049 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.407270  1898 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.373427  2152 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:58.407799 32112 raft_consensus.cc:2368] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:17:58.407954  2152 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:58.409716 32118 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:58.410786  2004 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.411473  2004 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.412928  1995 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:58.416605 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:58.418712 32114 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:58.413635  1995 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:58.432955 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.437994  2157 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.438118 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.375502 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.439078 31965 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.439036 31970 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.375841 32039 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:17:58.335474  2092 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:17:58.441789  2092 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:58.442296  2092 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:58.443529  2092 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.354722  2088 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.390919  2086 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.453809 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1815) took 1420 ms. Trace:
I20260709 12:17:58.454366 32120 rpcz_store.cc:276] 0709 12:17:57.033456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.034092 (+   636us) service_pool.cc:224] Handling call
0709 12:17:57.978754 (+944662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.106903 (+128149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.275679 (+168776us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54003e4ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:58.453544 (+177865us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b54003e4ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:58.453728 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2365399040}
I20260709 12:17:58.392549  1997 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:58.459120  1997 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.460587  1997 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.393334  1823 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.360994  1670 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.438556  2157 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.463038  2157 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.464196  2157 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.393644  2076 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.476413  2009 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.374603 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:58.480751 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.480806  2009 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEARNER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:58.488627  2076 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:17:58.489813 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 147604us
I20260709 12:17:58.491902  2156 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:58.493229  1996 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:58.494035  1996 consensus_queue.cc:1243] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:58.502198  2105 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.504424  2066 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.505704 31964 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:58.506153 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.508737  2004 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.510558  2155 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.512924  2155 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.407905  1898 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:58.514716 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 425158us
I20260709 12:17:58.516695 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:58.517357 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 142850us
I20260709 12:17:58.519193 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.521307 32114 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:58.524580 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1486) took 1983 ms. Trace:
I20260709 12:17:58.526479 31970 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:17:58.343756 32010 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.955s (SENT)
I20260709 12:17:58.526530 32045 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:58.544924 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:58.545131 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:58.541316 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 239864us
I20260709 12:17:58.525113 31966 rpcz_store.cc:276] 0709 12:17:56.541565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.738109 (+1196544us) service_pool.cc:224] Handling call
0709 12:17:57.764228 (+ 26119us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:57.824822 (+ 60594us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:58.152479 (+327657us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.501199 (+348720us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:58.501809 (+   610us) log.cc:844] Serialized 52 byte log entry
0709 12:17:58.501994 (+   185us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:58.502033 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.504066 (+  2033us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.523447 (+ 19381us) raft_consensus.cc:1685] finished
0709 12:17:58.523476 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.524499 (+  1023us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":199107,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":45057,"raft.run_cpu_time_us":497,"raft.run_wall_time_us":34774,"thread_start_us":657255,"threads_started":3,"wal-append.queue_time_us":260931}
I20260709 12:17:58.545327 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.554036 32114 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:17:58.565809 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 221022us
I20260709 12:17:58.569195 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:58.571174 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52042.5us
I20260709 12:17:58.572619 31965 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:58.572798 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.573516 31970 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:58.573850 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.574169 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:58.573884 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.575507 32652 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:58.576310 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:58.577281 32118 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:58.567804 31869 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 3 to 5, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:58.581688 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.585148  2090 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:58.543759  2092 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:58.582253 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:58.558290  2105 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:58.579082 31869 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 2 to 5, term changed from 1 to 5, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:58.594604 32652 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:58.600301  2067 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:58.600601  2164 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:17:58.601291  2067 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:58.595121 31869 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 3 to 6, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 6 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:58.602371  1670 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:17:58.602818  2164 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:58.603426  1670 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.603820  1670 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:58.602834 31869 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 3 to 6, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 6 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:58.604799  1670 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:58.606591 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90826.1us
I20260709 12:17:58.609221 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.612764 32112 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:58.608845 31869 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 6, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 6 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:17:58.613934 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 60226.7us
I20260709 12:17:58.615274 32120 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [6.2-8.3]
I20260709 12:17:58.607826 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:17:58.616107 31870 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 228) took 16607 ms (client timeout 15000 ms). Trace:
I20260709 12:17:58.616926 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:58.623355 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 279612us
I20260709 12:17:58.574682  2050 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:17:58.637059 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.628628 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.606200  1670 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.571748  1996 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 12:17:58.617960  1933 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.504s	user 0.015s	sys 0.000s
W20260709 12:17:58.617489 31870 rpcz_store.cc:269] 0709 12:17:42.008442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.014897 (+  6455us) service_pool.cc:224] Handling call
0709 12:17:58.616037 (+16601140us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:53.173713 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:53.315069 (+141356us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:53.352297 (+ 37228us) write_op.cc:435] Acquired schema lock
0709 12:17:53.352312 (+    15us) tablet.cc:662] Decoding operations
0709 12:17:53.613886 (+261574us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:53.613978 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:53.614008 (+    30us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:17:53.615210 (+  1202us) tablet.cc:701] Row locks acquired
0709 12:17:53.615259 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:17:53.615438 (+   179us) write_op.cc:270] Start()
0709 12:17:53.615586 (+   148us) write_op.cc:276] Timestamp: P: 1783599473615368 usec, L: 0
0709 12:17:53.615611 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:17:53.664619 (+ 49008us) log.cc:844] Serialized 4850 byte log entry
0709 12:17:54.382017 (+717398us) op_driver.cc:464] REPLICATION: finished
0709 12:17:54.486695 (+104678us) write_op.cc:301] APPLY: starting
0709 12:17:54.488672 (+  1977us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:54.492521 (+  3849us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:54.492697 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:54.523446 (+ 30749us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:54.523875 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:17:54.523931 (+    56us) write_op.cc:312] APPLY: finished
0709 12:17:54.524319 (+   388us) log.cc:844] Serialized 89 byte log entry
0709 12:17:54.524498 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:54.524643 (+   145us) write_op.cc:454] Released schema lock
0709 12:17:54.524706 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:54.525164 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:54.525191 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36693,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":92909,"apply.run_cpu_time_us":4451,"apply.run_wall_time_us":38473,"num_ops":12,"prepare.queue_time_us":175513,"prepare.run_cpu_time_us":22532,"prepare.run_wall_time_us":892214,"raft.queue_time_us":441694,"raft.run_cpu_time_us":14514,"raft.run_wall_time_us":113159,"replication_time_us":752177,"spinlock_wait_cycles":2304,"thread_start_us":677416,"threads_started":4,"wal-append.queue_time_us":122789,"wal-append.run_cpu_time_us":20594,"wal-append.run_wall_time_us":1708144}]]}
I20260709 12:17:58.638406  1933 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:58.637667 32010 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.185s (SENT)
I20260709 12:17:58.640229 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:58.622933 31869 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 6 to 7, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:58.644591 31869 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:58.650681 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.589877  2039 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:17:58.593173  2114 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:17:58.680822 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 90150.9us
I20260709 12:17:58.628814 32656 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:58.620757  2105 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:58.680886 31869 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:58.640342  2098 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:17:58.646883 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.633361  2069 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.683835 31972 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 12:17:58.683745 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 101552us
I20260709 12:17:58.628290  1904 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:58.636163  2009 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:58.686547 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50925.3us
I20260709 12:17:58.696596 31869 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:58.681151  2142 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.646070 32653 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:58.702433 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:58.707480  2090 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:17:58.699314 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81545.3us
I20260709 12:17:58.681793  1995 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.686941  2066 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.710350 31869 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 5, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:58.738461 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.717404 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.684608  1971 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:58.706221  2069 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:58.744037 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.744014 31869 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 6 to 7, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:17:58.749492 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.749869 31869 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 5, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:17:58.751909 31869 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 3 to 4, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:58.753959 31869 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 3, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 3 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:58.709463  2050 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:58.682997  2073 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:17:58.711447  2009 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:17:58.759897 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:58.769723 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:58.785987 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.786540 32045 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.727455  1780 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.791971 32116 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:17:58.781410 31963 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:17:58.804200 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 44447.2us
I20260709 12:17:58.804353 32039 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:58.698740  2081 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:17:58.738794  2155 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:58.812469 32045 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:58.757047  2069 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.758285  2050 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:58.748716  2142 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:17:58.800989 31869 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 3 to 2, term changed from 5 to 7, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from VOTER to NON_VOTER, now has a pending config: VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2), VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:58.761978  1997 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:58.840523 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 202858us
I20260709 12:17:58.737808  1904 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
W20260709 12:17:58.808938  2045 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.003s
W20260709 12:17:58.833493  2174 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:17:58.854010 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:58.862538 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.865228 32114 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:58.865402 32120 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:58.858403 32115 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6.
I20260709 12:17:58.849006 31869 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 6, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 6 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:58.845242  2173 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:17:58.761559  2098 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:17:58.869443 31869 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 1 to 4, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:17:58.756381  2146 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:58.848091 32010 leader_election.cc:336] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.842s (SENT)
I20260709 12:17:58.819547  2066 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:58.873289 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:58.845077  2159 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:58.847630  2073 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.877101 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:58.876556 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:58.882709 32115 raft_consensus.cc:2410] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:58.834583  1992 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:17:58.854563  2185 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.870396  1904 consensus_queue.cc:1243] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:58.761888  2105 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.865459 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:58.840346  2155 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.833489  2158 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 12:17:58.877059  2159 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:58.880007  2073 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:58.845655  2167 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.265s
I20260709 12:17:58.854923  2142 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:58.949319 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 101247us
W20260709 12:17:58.954571 32010 connection.cc:256] RPC call timeout handler was delayed by -2.3255390839999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3345s for a response
I20260709 12:17:58.898046  2185 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.898484  2181 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:58.956300 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:58.961347  2170 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:17:58.885102  2009 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:58.966516  1933 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:58.979992  1971 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:58.980464  2066 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.981562  1992 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:58.997058 32010 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 3.032s (SENT)
I20260709 12:17:59.025790 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.027310  2158 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.007515 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.028439 32119 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:17:59.027477 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.028633 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43996.7us
I20260709 12:17:59.000276 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:59.030402 32010 connection.cc:256] RPC call timeout handler was delayed by -1.808496229000184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.40590000000000004s for a response
W20260709 12:17:59.031055 32010 connection.cc:256] RPC call timeout handler was delayed by -1.6031006890000299s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3195s for a response
I20260709 12:17:59.033285 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.035866 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:58.981429  2159 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.034122  2158 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.041774  2167 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.043226 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.043629  2142 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:59.046803 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.047550  1898 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:17:59.091682 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48550us
I20260709 12:17:59.048682  2146 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:58.988418  2185 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:59.051025 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.100282 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.101205 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:59.054677  2179 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:59.055262  2081 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.057431 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:58.938799  2050 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:59.061125  2152 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
W20260709 12:17:59.022111  2173 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:17:59.061213 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.096596  2009 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.065789 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:59.119535 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53880.8us
I20260709 12:17:59.068856 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:59.121582 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:59.122171 32039 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.123060 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:59.123624 32047 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.124351 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:59.125059 32045 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:59.037768  1992 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.093042 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.125569 32658 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 1.
I20260709 12:17:59.094830 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:17:59.080539  2094 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.682s	user 0.014s	sys 0.002s
W20260709 12:17:59.065105  2189 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:17:59.103148 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:59.089710  2181 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.086817  2167 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.108114 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:17:59.153205 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45153.3us
I20260709 12:17:59.114678  2146 raft_consensus.cc:2764] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:17:59.116393 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.155709  2146 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:59.156510  2146 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:59.156823  2146 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:59.157785  2184 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:59.157625  2146 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.022632  2098 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:59.170300 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 467943us
I20260709 12:17:59.170534 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.099882  2182 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:59.158200  2184 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:59.103294  2159 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.140681  1986 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:59.141319  2079 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.805s	user 0.010s	sys 0.004s
W20260709 12:17:59.141693  2191 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:17:59.114493  2081 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.143162  2158 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:59.088969  1971 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:17:59.144506  1992 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:59.145983  2050 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.087863  2073 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.114190  2179 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.126734  2152 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:59.116802  2080 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.785s	user 0.006s	sys 0.011s
I20260709 12:17:59.148731 31971 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:17:59.154953 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:59.064170 31967 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:59.034669  2066 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.162281 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:59.163166 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70186.7us
I20260709 12:17:59.222994 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.164554 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:59.224148 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 59667.2us
I20260709 12:17:59.171209 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:59.176888 32118 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:59.179114 32659 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:59.179610 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 102815us
I20260709 12:17:59.228755 31965 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:59.232661 32116 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:59.136020 32039 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:17:59.233762 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:59.184140 32047 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:17:59.164011  2167 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.244280  2184 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:59.191134 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.163872  2181 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.191890  1997 raft_consensus.cc:1064] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:17:59.168911  2094 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:17:59.193897 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.177831  2144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.001s	sys 0.008s
I20260709 12:17:59.205767 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:59.177911  2076 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.004s	sys 0.005s
I20260709 12:17:59.179280  2182 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:59.219887 31971 pending_rounds.cc:85] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 3
I20260709 12:17:59.265206 31971 pending_rounds.cc:107] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:59.265540 31971 raft_consensus.cc:2889] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:59.264407 32121 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:17:59.220546 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 65659.3us
I20260709 12:17:59.217377  2158 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.178898  2098 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.250213 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:17:59.260558  2144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.002s	sys 0.008s
I20260709 12:17:59.251024 31967 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:59.211783  2146 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.251662 32119 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
W20260709 12:17:59.254702 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 83559.1us
I20260709 12:17:59.286933 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.250224  2079 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:59.258126 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.209378  1986 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:59.266923 32044 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.100s	sys 0.108s
I20260709 12:17:59.268489 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.250955 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.253700  2094 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:59.261689  2092 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.000s
I20260709 12:17:59.293433 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.312045 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.269840  2182 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:59.250059  2067 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:17:59.252298  1670 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:17:59.301071 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.294201  1898 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:59.301383 32044 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.100s	sys 0.108s
W20260709 12:17:59.285226  2088 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.826s	user 0.017s	sys 0.000s
W20260709 12:17:59.282935  2180 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.631s	user 0.002s	sys 0.003s
I20260709 12:17:59.266472  2080 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.263819  2073 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:59.313337 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:59.254891 32112 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:17:59.356329 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:17:59.317399 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 94464.3us
W20260709 12:17:59.264505  2076 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.004s	sys 0.005s
I20260709 12:17:59.324930  2144 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.325196  2158 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:59.288932 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1807) took 1393 ms. Trace:
I20260709 12:17:59.342684 32660 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:59.312273  2094 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.361959 31968 rpcz_store.cc:276] 0709 12:17:57.895942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.180345 (+284403us) service_pool.cc:224] Handling call
0709 12:17:58.180695 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:58.301043 (+120348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.301088 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:58.458408 (+157320us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:17:58.459080 (+   672us) log.cc:844] Serialized 52 byte log entry
0709 12:17:58.567619 (+108539us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:58.587894 (+ 20275us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.009813 (+421919us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b1800354c40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:17:59.010093 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.263361 (+253268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.263762 (+   401us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.264015 (+   253us) raft_consensus.cc:1685] finished
0709 12:17:59.264130 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.288857 (+ 24727us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":411586,"prepare.queue_time_us":454150,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":11934,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":21179,"spinlock_wait_cycles":46729472,"thread_start_us":360697,"threads_started":2,"wal-append.queue_time_us":147340}
W20260709 12:17:59.343802  2195 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:17:59.344241  1986 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:59.345453 32120 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:59.348969 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1594) took 1322 ms. Trace:
I20260709 12:17:59.289995  2079 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:17:59.357125 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43858.8us
I20260709 12:17:59.357345 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.349349  2205 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.365157 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.392537 32114 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:17:59.341380  1971 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:59.367581  2144 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:59.373127  2069 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.001s
I20260709 12:17:59.355997  2088 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.370585 32044 rpcz_store.cc:276] 0709 12:17:58.026744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.029007 (+  2263us) service_pool.cc:224] Handling call
0709 12:17:58.029315 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.328177 (+298862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.328206 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:58.335715 (+  7509us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.589370 (+253655us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:58.632274 (+ 42904us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.348578 (+716304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.348673 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.348732 (+    59us) raft_consensus.cc:1685] finished
0709 12:17:59.348752 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.348909 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77766,"prepare.queue_time_us":42720,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":275597,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":5826,"thread_start_us":1131632,"threads_started":3,"wal-append.queue_time_us":361339}
I20260709 12:17:59.375068  2211 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:59.375272  1996 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.406749 32120 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:59.376699 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.374195  2201 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.356369  2080 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:17:59.360208  2092 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:17:59.375334  2158 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.375787  2152 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:59.406198  2181 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:59.394208 32114 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:59.405608  1992 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.412791 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.415189  2210 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:59.407615  2202 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:17:59.418046 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.419723 31968 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:59.412819 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1806) took 1517 ms. Trace:
I20260709 12:17:59.452479 31971 rpcz_store.cc:276] 0709 12:17:57.895690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.156513 (+260823us) service_pool.cc:224] Handling call
0709 12:17:58.182700 (+ 26187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.283002 (+1100302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.412594 (+129592us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:17:59.412764 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220880640}
I20260709 12:17:59.420781 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.425899 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.454100 31970 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:17:59.427456 31964 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:59.391260  2079 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:59.363628 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72522.5us
I20260709 12:17:59.435889 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.457507 31967 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:17:59.390760  2081 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:59.448910 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:17:59.473737 31966 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.449210 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:17:59.474787 32657 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:17:59.404253  2066 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:59.438645  2164 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:17:59.456435 32119 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:59.405388  2098 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:59.476770  2164 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.477252  2098 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:59.477651  2212 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:59.478670  2208 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.479053  2164 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:59.479221  2208 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.481443  2208 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:59.408727  2185 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.402170  2069 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.474865 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.476027  2158 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.476308 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.430269  1971 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:59.657982 32047 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:59.648095  2192 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:17:59.611784 31971 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:59.653538 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.668227 31971 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:59.559139  2194 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.639776  2197 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:17:59.586098 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.525775 31963 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:17:59.670889 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.491672 31966 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:17:59.647715 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.669571 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:59.672403 31965 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:17:59.436422  2080 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:59.673517 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 405056us
I20260709 12:17:59.559398  1823 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.685994 32047 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:59.653313 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:17:59.686391 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.686746 31967 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:17:59.667128 31968 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:17:59.652809 31964 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:59.688596 32047 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [3.2-8.4]
I20260709 12:17:59.671518 31971 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.707295 31971 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.707593 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:59.708789 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:17:59.655224 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 420438us
I20260709 12:17:59.710086 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:17:59.484814 32115 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:59.711617 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:59.712772 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:59.711699 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.714599 32046 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:59.714953 32046 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:59.715570 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:17:59.715534 31938 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:17:59.666585 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.676386  2190 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:59.677199  2181 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:17:59.681397 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.681468 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.612002  2202 consensus_queue.cc:1243] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:17:59.685137 32039 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:59.726300 32039 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:59.407524  1997 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.718559 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:59.728243 32042 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:17:59.669436 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 268564us
I20260709 12:17:59.613379  2167 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.723939 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:59.733834 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1671) took 1023 ms. Trace:
I20260709 12:17:59.734253 32118 rpcz_store.cc:276] 0709 12:17:58.710719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.710976 (+   257us) service_pool.cc:224] Handling call
0709 12:17:59.733663 (+1022687us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54002b52d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:17:59.733778 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005761408}
I20260709 12:17:59.723201 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.736815 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.724644 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.727249 32039 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.739880 32039 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.715668 32046 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.743167 32046 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.724727 32044 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:59.749033 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:59.751418 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.752261 32118 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:17:59.744518 32626 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.484462  2185 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:59.729238 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:17:59.719127  2066 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:59.483258  2144 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:17:59.674259  2081 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.670531  2179 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:59.724419  2211 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.741437  2086 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.755046 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:59.719210  2208 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.757015  2092 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.760149 32043 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:17:59.760684 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.763607 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.669553  2197 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:59.788575 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57277us
I20260709 12:17:59.762825 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.789341 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.790284 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.612823  2094 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.801584 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:17:59.760694  2220 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:59.763849 32039 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:17:59.806231 32039 pending_rounds.cc:85] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 3
I20260709 12:17:59.806527 32039 pending_rounds.cc:107] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:17:59.806767 32039 raft_consensus.cc:2889] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:59.771104  2066 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:17:59.728973  2067 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.774053  2086 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.732743 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:59.721781 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:17:59.728892  2205 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:17:59.763881  1670 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.747283  2152 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:59.729728 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.822324 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.746872  2105 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.776093 32046 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:59.774235 32652 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:17:59.475469  2210 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.718209  1823 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:17:59.834966  2214 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:17:59.762030 31971 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:59.669476  2194 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.802683 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:17:59.837136 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47882.6us
I20260709 12:17:59.837795 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.764792 32038 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:17:59.803488 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.800980 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.733042  2206 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.815455 32118 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:17:59.841523 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 126058us
I20260709 12:17:59.842643 32118 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:59.842959 31938 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6.
I20260709 12:17:59.843241 31938 leader_election.cc:403] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:59.843760 32118 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:59.836321 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 100160us
I20260709 12:17:59.772694  2081 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.848865 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.850327 32119 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:59.898417 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49620.6us
I20260709 12:17:59.846597 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:17:59.851046 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 138329us
I20260709 12:17:59.900364 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:59.853225 32118 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:17:59.816365  2086 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.814049  2211 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.804669  2202 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 12:17:59.897342 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50005.7us
W20260709 12:17:59.899793 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.910105 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.913935 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.899998  2188 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:59.854564  2067 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:59.919317  2188 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:17:59.913832  2211 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.919634  2213 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:59.809551  2220 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:17:59.800947  1971 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:17:59.934612 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 88290.8us
W20260709 12:17:59.935104 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 97384us
I20260709 12:17:59.936254 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.900444  2221 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.909394  2105 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6.
I20260709 12:17:59.844424  1670 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:17:59.945652 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:17:59.913625  2206 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.954864 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:17:59.912398  2090 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:17:59.934465  2102 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:17:59.964005 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:17:59.930299  2213 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:17:59.941497  2152 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.933787  2210 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:59.941777  2190 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.941120  2222 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:00.072851 32112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1669) took 1436 ms (client timeout 1524 ms). Trace:
W20260709 12:18:00.063243 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 162940us
I20260709 12:17:59.952159  2181 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:00.081702 32112 rpcz_store.cc:269] 0709 12:17:58.636671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.636807 (+   136us) service_pool.cc:224] Handling call
0709 12:18:00.072696 (+1435889us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54001482d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:00.072793 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310445952}
W20260709 12:18:00.066253 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 111500us
I20260709 12:18:00.089002 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:17:59.957379  2208 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:59.949682  2069 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:17:59.942078  2223 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:59.949573  1823 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:17:59.958513  2218 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.055415  2155 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:00.107314 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 172117us
I20260709 12:18:00.054557  2221 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:00.122445 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.154031 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65103.9us
I20260709 12:18:00.072315  2178 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:00.156827 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.157112 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.071933  2231 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:00.066211  2105 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:00.086061  2181 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:00.109870  2224 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:18:00.068181  2102 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.072336  2216 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:00.130949 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47943.5us
I20260709 12:18:00.228726 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.230304 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.078511  2188 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:00.225878 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.106994  2223 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:00.215049  2189 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:18:00.216784  2233 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:18:00.215059  1813 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 12:18:00.219434  2219 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
W20260709 12:18:00.216051  2237 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 12:18:00.218124  2229 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.271s	user 0.000s	sys 0.000s
W20260709 12:18:00.217442  2225 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:18:00.255410 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 228) took 1274 ms. Trace:
I20260709 12:18:00.106300  2222 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.255803 32116 rpcz_store.cc:276] 0709 12:17:58.981130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.340154 (+359024us) service_pool.cc:224] Handling call
0709 12:17:59.340538 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.346734 (+  6196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.346785 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:59.359853 (+ 13068us) log.cc:844] Serialized 26 byte log entry
0709 12:17:59.755823 (+395970us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:59.755861 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.755923 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.255208 (+499285us) raft_consensus.cc:1685] finished
0709 12:18:00.255225 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.255349 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5697,"raft.queue_time_us":8728,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":5350,"thread_start_us":368569,"threads_started":1,"wal-append.queue_time_us":449849}
I20260709 12:18:00.257447  2222 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.108624  2239 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:00.258180  2239 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:00.214828 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57774.9us
I20260709 12:18:00.258701  2239 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.162283  2178 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:00.260711  2178 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.258212  2206 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:00.099823  2073 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.738s	user 0.009s	sys 0.001s
I20260709 12:18:00.262276  2206 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:00.164691  2155 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:00.079483  2152 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.262955  2155 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:00.162933  1780 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260709 12:18:00.104616  2009 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.977s	user 0.010s	sys 0.002s
I20260709 12:18:00.265553  2223 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:00.268266  2050 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.049s	user 0.010s	sys 0.001s
I20260709 12:18:00.268136 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.268720  1996 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.268950  2238 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:00.268967 32118 raft_consensus.cc:2368] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:18:00.270737 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:00.272083 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:00.272815 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:00.273516 32116 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:00.273636 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.274369 32655 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:00.274545  2218 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:00.275156  2218 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:00.251109  2235 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:18:00.215063  2230 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:00.278338 31938 leader_election.cc:400] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7.
I20260709 12:18:00.278755  2220 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.090483  2202 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:18:00.109807  1904 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:00.281188 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:00.282280 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:00.306632 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.269196  1996 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.275660  2218 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:00.297060 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.265033  2152 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.262795  2206 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:00.054934  2066 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.163288  2092 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.308596 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:00.310953 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 361206us
I20260709 12:18:00.262081  2178 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.153667  2208 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:00.311072  2067 raft_consensus.cc:670] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } [suppressed 1 similar messages]
I20260709 12:18:00.215247  2193 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:00.166369  2167 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.280543  2181 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.316798  2188 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.221282  2184 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:00.223848  2194 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.325623 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.174273  2094 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:00.316036 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.226647  2231 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.333638 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:00.173647  2146 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.144s
I20260709 12:18:00.302584  2090 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.305271  1904 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:00.315124  2230 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.403239 31968 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:00.340983  1670 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:00.405500 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.312649  2156 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.232908  2105 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.343881 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.337293  2193 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:00.347436  2171 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.460s	user 0.005s	sys 0.038s
I20260709 12:18:00.354815  2208 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.356803  2223 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:00.360083 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.363494 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:00.364218 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.427712 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.428228 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.428539 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.429888 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:00.336448  1933 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.432221 32118 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:00.344434  2197 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.365640  2167 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.422379 32653 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:00.366554  2177 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:00.366243  2206 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:00.366882 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1666) took 1753 ms (client timeout 2117 ms). Trace:
W20260709 12:18:00.464752 32121 rpcz_store.cc:269] 0709 12:17:58.613020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.613195 (+   175us) service_pool.cc:224] Handling call
0709 12:18:00.366732 (+1753537us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400139054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:00.366828 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357474048}
I20260709 12:18:00.466162 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.370949  1904 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.373018  2155 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:00.373955  2182 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.025s	user 0.012s	sys 0.000s
I20260709 12:18:00.335340  2092 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.343758  2081 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.374897  2086 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.375231  2102 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.376616 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.376638 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1676) took 1488 ms. Trace:
W20260709 12:18:00.377907  2159 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.173s	user 0.010s	sys 0.000s
I20260709 12:18:00.358954  2090 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.382772  1780 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:18:00.386739  2080 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 12:18:00.389535  2158 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.390452  2210 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.393227 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 227) took 1165 ms. Trace:
I20260709 12:18:00.393325 32047 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:00.393361 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:00.394287  2211 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.511857 32045 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:00.395735 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.395936 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:00.514658 32039 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.511142 31972 rpcz_store.cc:276] 0709 12:17:59.227474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.251471 (+ 23997us) service_pool.cc:224] Handling call
0709 12:17:59.265634 (+ 14163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.326173 (+ 60539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.326364 (+   191us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:59.326919 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:18:00.392711 (+1065792us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b180010c660. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1
0709 12:18:00.392851 (+   140us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:00.392897 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.392960 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.393000 (+    40us) raft_consensus.cc:1685] finished
0709 12:18:00.393012 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.393179 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100045,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":30795,"spinlock_wait_cycles":18634496,"thread_start_us":131317,"threads_started":2,"wal-append.queue_time_us":156673,"wal-append.run_cpu_time_us":1451,"wal-append.run_wall_time_us":1015683}
I20260709 12:18:00.396914 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.397261 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.398025 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 568) took 1845 ms. Trace:
I20260709 12:18:00.400808  1996 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.402390  2216 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:00.407500 32652 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:00.409778 32659 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:00.408332 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 178127us
W20260709 12:18:00.414825 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.421021 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.423846 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.430616 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:00.431764 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68338.7us
W20260709 12:18:00.346449  2079 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.002s
I20260709 12:18:00.435617 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.518131 32660 rpcz_store.cc:276] 0709 12:17:58.552338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.552448 (+   110us) service_pool.cc:224] Handling call
0709 12:17:58.552752 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.363891 (+811139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.397818 (+1033927us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b5400526cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:00.397968 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815775104}
I20260709 12:18:00.441677 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.442293 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1513) took 2560 ms. Trace:
I20260709 12:18:00.527274 32658 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:00.527370 31969 rpcz_store.cc:276] 0709 12:17:57.882052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.144711 (+262659us) service_pool.cc:224] Handling call
0709 12:17:58.145127 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.162969 (+ 17842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.163005 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:58.163486 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.628601 (+465115us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:58.628648 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.628715 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.326821 (+1698106us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:00.441946 (+115125us) raft_consensus.cc:1685] finished
0709 12:18:00.441980 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.442226 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3219,"raft.queue_time_us":934,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":11715,"spinlock_wait_cycles":8729472,"thread_start_us":361923,"threads_started":1,"wal-append.queue_time_us":553423}
I20260709 12:18:00.443914  2223 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.450506  2193 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.451238  2239 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.453070 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.440969  2177 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:00.456688  2142 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:00.458127  1997 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:00.416429  2156 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.506407 32115 rpcz_store.cc:276] 0709 12:17:58.888048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.888687 (+   639us) service_pool.cc:224] Handling call
0709 12:17:58.892338 (+  3651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.711092 (+818754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.376394 (+665302us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b5400138154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:00.376575 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361975552}
I20260709 12:18:00.395113 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.545926 32659 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:00.546494 32046 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:00.499078  2182 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:00.521977 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.482055  2218 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.498893  2181 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.523672 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.527061 32660 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:00.531098 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.538556 32039 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:00.540303 32045 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:00.510696  2220 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.525847  2167 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.522775 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.523125 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:00.499037  1904 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:00.553083 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:00.555516 32652 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:00.557602 32658 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:00.559538 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.560698 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.515319  2080 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.561153 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:00.560142 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.563573 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:00.564401 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.564738  2159 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:00.565409 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.566108 32046 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:00.566787 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:00.571990 32652 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:00.572707 32043 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:00.573395  2226 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.511s	user 0.000s	sys 0.001s
I20260709 12:18:00.574374  2182 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.561368  2102 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:18:00.576997  2080 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.577574 32117 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:00.579577  2092 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.535954  1996 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:00.553730  2155 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.562952  2066 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:00.585415  2156 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.517686  2158 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.587460  2090 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.588615  2086 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.589202  2218 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:00.632737 32042 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:00.558127  2206 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:00.535382  2079 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.650507  2079 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.594985  2177 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.651332  2177 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.595517  2181 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.595896  2105 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.597376  2239 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.562095  2167 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.545739  2142 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.655668 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:00.598834  2205 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.656769  2167 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.599694 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:00.519461  2210 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.602645 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:00.605118 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.605463 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.602540 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:00.672013 31964 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:00.606138 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.607782 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.673234 31967 raft_consensus.cc:2368] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7.
I20260709 12:18:00.674099 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.608036 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.608114 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.609302 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.675061 31971 raft_consensus.cc:2368] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8.
I20260709 12:18:00.675356 31966 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:18:00.611351 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:00.676462 32041 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:00.612795 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.623098 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.606879  2208 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.624488 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:00.681748 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 239) took 1542 ms. Trace:
I20260709 12:18:00.644309 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.682412  2208 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.646713 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:00.684434 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81962.6us
I20260709 12:18:00.684578 31968 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.682189 32114 rpcz_store.cc:276] 0709 12:17:59.139811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.443494 (+303683us) service_pool.cc:224] Handling call
0709 12:17:59.444062 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.445698 (+  1636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.446134 (+   436us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:59.676007 (+229873us) log.cc:844] Serialized 26 byte log entry
0709 12:17:59.787061 (+111054us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:59.787117 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.787204 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.572898 (+785694us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:00.678827 (+105929us) raft_consensus.cc:1685] finished
0709 12:18:00.678858 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.681682 (+  2824us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45438,"raft.queue_time_us":184,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":11418,"spinlock_wait_cycles":32724352,"thread_start_us":110547,"threads_started":1,"wal-append.queue_time_us":238536}
I20260709 12:18:00.664889 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.686831 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:00.631711  2066 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:00.677860 31965 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:00.700922 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.701440 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.676211 32040 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:00.677861 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.702509 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:00.702858 31969 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:18:00.671378 32046 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:00.690996 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:00.703866 32038 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:00.692739 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:00.704459 31963 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:18:00.704823 32044 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.705042 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:00.658242 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 91098.4us
I20260709 12:18:00.691599 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.707085 31966 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:00.704825 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.656819  2142 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.709183 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.675792  2069 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.716187 32116 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:00.711028 31971 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:00.657101  2205 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.710031 32041 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:00.718283 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.705956  2239 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:00.718992  2210 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.719293  2239 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:00.719800  2239 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.720438 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:00.720647 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.721163 32114 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:00.721211 32653 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:00.709285  2105 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:00.722326 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.722527 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:00.724436 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.712028 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.676354  2206 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:00.719133 31966 raft_consensus.cc:2410] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:18:00.710079 32010 leader_election.cc:400] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7.
I20260709 12:18:00.725663 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:00.727279  2142 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:00.796476 32044 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:00.731408 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.797309 31971 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:18:00.731895  1933 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:00.798672  1933 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.732836  2215 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:00.715678  2086 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:18:00.715111 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.688349  2158 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.735046  2197 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:00.737006  2242 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:00.738981 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.740711 32653 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:00.786829 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:00.742822  2066 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.704373  2159 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:00.727407 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:00.790613 31968 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:00.835254  2205 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:00.836242 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.810400 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 181932us
I20260709 12:18:00.837551 31964 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 12:18:00.837637 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 110317us
I20260709 12:18:00.792276 31963 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: [5.3-7.4]
W20260709 12:18:00.794400 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84509.6us
I20260709 12:18:00.792760  2069 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:00.795075  2208 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.795428  2202 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:00.796288 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 230) took 1327 ms. Trace:
I20260709 12:18:00.700717  1992 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:00.796638 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69525.8us
I20260709 12:18:00.796060  2206 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.786049  2184 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.801694 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:00.843933  2159 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:00.868686 32116 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:00.849843  2069 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.837671  2211 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:00.853956 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:00.841799  2142 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:00.852314 31970 rpcz_store.cc:276] 0709 12:17:59.468601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.707433 (+238832us) service_pool.cc:224] Handling call
0709 12:17:59.707803 (+   370us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:59.708219 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.708249 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:59.708992 (+   743us) log.cc:844] Serialized 78 byte log entry
0709 12:18:00.359301 (+650309us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b1800293b60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:18:00.538649 (+179348us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:00.558483 (+ 19834us) log.cc:844] Serialized 14 byte log entry
0709 12:18:00.558816 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:18:00.562447 (+  3631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.562526 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.796065 (+233539us) raft_consensus.cc:1685] finished
0709 12:18:00.796088 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.796232 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41068,"prepare.queue_time_us":41272,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":18997,"raft.queue_time_us":47745,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":24002,"spinlock_wait_cycles":15157632,"thread_start_us":179139,"threads_started":1,"wal-append.queue_time_us":906659}
I20260709 12:18:00.848724 32010 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
W20260709 12:18:00.860597 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58976us
I20260709 12:18:00.879124 32010 leader_election.cc:400] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8.
I20260709 12:18:00.874936  2069 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:00.866489  2184 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.880514  1933 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:00.880592 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:00.881541  1971 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:00.882009  1971 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:00.870565  2181 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:00.884006 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.884627 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:00.885414 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:00.850801  2079 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:18:00.804380  2242 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:00.853396  1996 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:18:00.870301  2105 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:00.882926  1971 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.871806 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:00.904139 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.904354 32115 raft_consensus.cc:2410] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:00.802779  2215 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:00.880157 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:00.910071 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:00.902196 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.913209 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.902779  2188 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:18:00.873971  2211 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:00.916375 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:00.916566  2215 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.917223 32659 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 3: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:00.917001  2211 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.918546  2211 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:00.921514 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:00.882230  2184 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:00.896203  2164 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:00.898738  2194 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:00.928151  2098 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:00.928921  2188 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:00.920069  2205 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.922839  2167 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.872723  2202 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:18:00.924610 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:00.929189  2213 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:00.953682 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.959215 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:00.931521  2210 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:00.899287  2181 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:00.961591 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:00.925997  2105 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:00.946676  2188 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:01.010653 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.015195 31963 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:01.019404 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.023839 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:00.948582  2194 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:01.000705  2249 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:18:01.010228 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48741.6us
I20260709 12:18:01.039861 31963 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:00.948704  2164 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:01.007197 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48066.3us
I20260709 12:18:01.000689  2184 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.958849  2215 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.959600  2098 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:01.054859 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:01.060130 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 233) took 2036 ms. Trace:
I20260709 12:18:01.060613 32117 rpcz_store.cc:276] 0709 12:17:59.023309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.357477 (+334168us) service_pool.cc:224] Handling call
0709 12:17:59.369045 (+ 11568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.609175 (+1240130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.059907 (+450732us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54004349d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:01.060074 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2778784128}
I20260709 12:18:01.008121  2246 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.052552 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:01.066058 32045 raft_consensus.cc:2376] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:18:01.000890  1933 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:01.000787  2210 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:01.056274 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.053761 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 167787us
I20260709 12:18:00.958596  2197 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.063019 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:01.069620 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:01.013587  2220 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:01.075093 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:01.033968  1670 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.069847 31971 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:01.073961 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:01.076669 31964 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:01.053054  2213 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.947948  2242 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.007383  2239 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.081645 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:00.948031  2159 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.101094 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.091857  2188 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.092994  2193 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.092782  2080 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:01.093454  2050 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.089061  1996 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:01.099436 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:01.116472 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.064110  2142 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:01.074116  2210 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.114957 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.129168 31965 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:01.117076 31971 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:01.131662 31971 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:01.063730  2215 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.077943  1670 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.077888  2079 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:18:01.161789 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.077386  2246 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.117575  2050 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:01.074949  1933 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.081238  2197 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.130151  2218 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:01.185774 31968 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
W20260709 12:18:01.185142 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.189339 31936 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.777s (SENT) [suppressed 47 similar messages]
W20260709 12:18:01.183327 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 128548us
I20260709 12:18:01.208455 31969 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:01.209555 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:01.141605  1670 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.226867  2102 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:01.141009  2188 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.143579  2273 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.110797  2105 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.189376  2050 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:01.244689 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 145306us
I20260709 12:18:01.245445 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:01.168108  1996 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:18:01.133302  2098 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:01.254714 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.232385  2267 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:01.240149  2273 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:01.237919  2066 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:01.169729  2164 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.229954  2102 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.171350  2194 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:01.294553 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64158.6us
I20260709 12:18:01.271317  2273 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.310250 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:01.288483  2181 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.285549  2267 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:01.294154  2277 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.290949  2279 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:01.315999 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:01.318837  2102 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.318995  1933 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.253631  2142 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.307197  2280 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:01.333993  2239 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.289340  2066 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.342653 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 482) took 1578 ms. Trace:
I20260709 12:18:01.343215 32656 rpcz_store.cc:276] 0709 12:17:59.763895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.767441 (+  3546us) service_pool.cc:224] Handling call
0709 12:17:59.805227 (+ 37786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.262662 (+457435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.262712 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:00.262784 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:00.262835 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.338657 (+1075822us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400550ad4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:01.338724 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.342568 (+  3844us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844425600}
I20260709 12:18:01.315315  2218 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:01.335543  2266 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:18:01.325876  2277 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:01.354600  2268 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:18:01.319764  2197 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.320533  2279 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.384177  2285 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:01.388017  2283 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:18:01.389570  2196 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.390139  2196 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.371361  2282 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:01.378242  2279 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.410583  2218 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.402456  2267 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.411710  2210 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.399798  2282 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.413347  2282 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.408231  2221 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:01.419037  2269 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.313s	user 0.002s	sys 0.002s
I20260709 12:18:01.418709  2280 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:01.419868  2274 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 12:18:01.448621 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.463639 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 499) took 1005 ms. Trace:
I20260709 12:18:01.484549  2105 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:01.486047  2105 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.498082 31971 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:01.531095  2283 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.532346  2283 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.509007  2159 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.534405  2221 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:01.534850  2159 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.535791  2283 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:01.536044  2221 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.537573  2221 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.537671  2159 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.550554  2211 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:01.551594  2211 consensus_queue.cc:1243] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:01.552219 31966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:01.553125 31966 raft_consensus.cc:606] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:01.553542 31966 raft_consensus.cc:614] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:01.557178 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 269) took 1160 ms. Trace:
I20260709 12:18:01.557626 32038 rpcz_store.cc:276] 0709 12:18:00.396813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.396994 (+   181us) service_pool.cc:224] Handling call
0709 12:18:01.556726 (+1159732us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400033ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:01.556931 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427636608}
W20260709 12:18:01.558893 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.559976 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.414695  2285 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:01.568320 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 268) took 1171 ms. Trace:
I20260709 12:18:01.568780 32040 rpcz_store.cc:276] 0709 12:18:00.396543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.396662 (+   119us) service_pool.cc:224] Handling call
0709 12:18:01.567943 (+1171281us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54000d0f54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:01.568123 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365165824}
I20260709 12:18:01.569569  2283 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.570231  2210 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.566934  2246 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.465093 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.525748  2105 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.552232  2211 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:01.586988 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.594954 31967 raft_consensus.cc:2410] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:01.586810  2211 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:01.603227  2211 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.605064  2211 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:01.605803 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.608258  1997 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.609004  1997 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.570569  2283 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:01.612377  2167 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.007s
I20260709 12:18:01.612517  2283 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.614872 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.616500 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:01.617591 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.618938  2066 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.618892  2079 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:01.619932  2164 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.620527  2215 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.621225  1971 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.565059  2196 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:01.578994  2159 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.626083  2210 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.621626 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.631561 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:01.586448  2069 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.004s
I20260709 12:18:01.641124  2167 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:01.420511  2280 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.648981 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:01.643576 31967 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:01.656266 31969 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:01.665805 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.663424 31968 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:01.634405  2164 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.675926 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:01.676095 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:01.677153  2167 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:01.677888  2167 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.680066  2167 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.681320  2201 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.681844  2086 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.682598  2205 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:01.683372  2205 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.685168 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:01.672201  2079 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.688189 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:01.682037  2201 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.688331  2079 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.684358  1904 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.679979  2164 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:01.688642 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:01.692227 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45376.4us
W20260709 12:18:01.670712 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 420733us
I20260709 12:18:01.640646  2069 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.699625 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.681380 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:01.712514 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 436446us
I20260709 12:18:01.718907 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.698058  1904 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.721206  1997 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:01.722352 32112 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:01.723309 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1815) took 1119 ms. Trace:
I20260709 12:18:01.723836 31972 rpcz_store.cc:276] 0709 12:18:00.604114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.604224 (+   110us) service_pool.cc:224] Handling call
0709 12:18:01.722837 (+1118613us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400312c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:01.723070 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824727296}
I20260709 12:18:01.726497  2086 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.729226  2086 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.713095  2210 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.730465 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:01.731719  2098 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:01.674893  2159 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:01.732722  2196 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:01.750417 32656 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:01.750550 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:01.750968 32039 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:01.751430 31970 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.751381  2285 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:01.753050 32657 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:01.754258 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.754742 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:01.755610 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67587.4us
I20260709 12:18:01.758342 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:01.464437 32660 rpcz_store.cc:276] 0709 12:18:00.457932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.551717 (+ 93785us) service_pool.cc:224] Handling call
0709 12:18:00.551991 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.552190 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:00.958938 (+406748us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.419336 (+460398us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:01.462060 (+ 42724us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.462617 (+   557us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:01.462709 (+    92us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:01.462868 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.463027 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.463348 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":323911,"prepare.queue_time_us":356315,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":95,"raft.queue_time_us":147699,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":23067,"spinlock_wait_cycles":17469824,"thread_start_us":586338,"threads_started":3,"wal-append.queue_time_us":201046}
I20260709 12:18:01.760353  2164 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:01.760773 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.760941 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:01.761818  2277 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.742813  2205 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.762221  2201 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:01.763226 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:01.763809  2155 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:01.764268  1997 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:01.765848 32120 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:18:01.765906 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.691350  1971 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:01.778528 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 162392us
I20260709 12:18:01.773279 31967 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:01.747463  2167 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:01.767206  2196 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.780704 32120 pending_rounds.cc:85] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 3
I20260709 12:18:01.803511 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:01.796171 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:01.799154 32120 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 6.4
W20260709 12:18:01.961374 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 308351us
I20260709 12:18:02.038687 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:02.899936 32120 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:01.842761 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.803387 32118 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:02.908433 32651 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
W20260709 12:18:01.953898 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.987421 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.951124 32115 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
W20260709 12:18:01.886693 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:02.102751 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.963961 32652 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.066999 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:02.324409 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 266679us
I20260709 12:18:01.953472 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:02.961095 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1703) took 1198 ms. Trace:
I20260709 12:18:01.846261 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:02.961692 32118 rpcz_store.cc:276] 0709 12:18:01.757009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.760368 (+  3359us) service_pool.cc:224] Handling call
0709 12:18:02.955301 (+1194933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.103191 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:01.884554 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:02.871119 32041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.007s	sys 0.023s
W20260709 12:18:02.964393 32041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.010s	sys 0.026s
I20260709 12:18:02.964395 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 577) took 1189 ms. Trace:
W20260709 12:18:01.987002  2310 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:18:02.966850 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1707) took 1076 ms. Trace:
W20260709 12:18:01.963917  2279 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.005s
W20260709 12:18:02.409053  2282 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.000s
I20260709 12:18:02.594777  2205 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:02.244863  2193 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.126s	user 0.010s	sys 0.001s
I20260709 12:18:02.967532 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1705) took 1200 ms. Trace:
W20260709 12:18:02.972048  2285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.011s	sys 0.007s
I20260709 12:18:01.842434 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:01.803699 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:01.952672 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:01.807862  2196 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.804136  2155 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:02.966046 31972 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:02.980156 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1538) took 1233 ms. Trace:
I20260709 12:18:02.980516 31972 rpcz_store.cc:276] 0709 12:18:01.752224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.771433 (+ 19209us) service_pool.cc:224] Handling call
0709 12:18:02.980104 (+1208671us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.965080 32651 rpcz_store.cc:276] 0709 12:18:01.771354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.778036 (+  6682us) service_pool.cc:224] Handling call
0709 12:18:02.960257 (+1182221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:02.814777  2188 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.578s	user 0.010s	sys 0.011s
I20260709 12:18:01.804740 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:02.986990 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 578) took 1204 ms. Trace:
I20260709 12:18:02.139684  2069 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:02.063694  1670 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.824s	user 0.010s	sys 0.005s
W20260709 12:18:01.887005  2184 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.008s
I20260709 12:18:02.991629 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 251) took 1058 ms. Trace:
I20260709 12:18:02.102451  2301 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 1.137s
I20260709 12:18:02.137737  2280 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:02.971789 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1618) took 1085 ms. Trace:
I20260709 12:18:01.981700  2216 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:02.969849 32116 rpcz_store.cc:276] 0709 12:18:01.890699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.890832 (+   133us) service_pool.cc:224] Handling call
0709 12:18:02.966775 (+1075943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.995033 32119 rpcz_store.cc:276] 0709 12:18:01.929805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.952252 (+ 22447us) service_pool.cc:224] Handling call
0709 12:18:02.988369 (+1036117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:02.824579  2273 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.507s	user 0.014s	sys 0.000s
I20260709 12:18:02.112792  2167 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:02.976459 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1702) took 1216 ms. Trace:
W20260709 12:18:02.972594  2142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.014s	sys 0.012s
I20260709 12:18:02.977089 32114 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:02.979854  2155 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:02.977375 32047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.005s	sys 0.009s
W20260709 12:18:02.647475  2050 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.989s	user 0.009s	sys 0.005s
W20260709 12:18:02.975059  2285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.011s	sys 0.007s
W20260709 12:18:03.011173 32047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.005s	sys 0.009s
I20260709 12:18:02.988785 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:02.990026  1971 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:02.990701 32121 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:02.987509 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:03.002161 32120 rpcz_store.cc:276] 0709 12:18:01.756543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.764710 (+  8167us) service_pool.cc:224] Handling call
0709 12:18:01.765173 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.971161 (+1205988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.973258 (+  2097us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.987818 32659 rpcz_store.cc:276] 0709 12:18:01.782337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.803292 (+ 20955us) service_pool.cc:224] Handling call
0709 12:18:02.986946 (+1183654us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.990847  2205 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:02.999228 32046 rpcz_store.cc:276] 0709 12:18:01.886149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.890804 (+  4655us) service_pool.cc:224] Handling call
0709 12:18:02.971728 (+1080924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.000945 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:02.976557  2215 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:02.975549 32117 rpcz_store.cc:276] 0709 12:18:01.766891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.771955 (+  5064us) service_pool.cc:224] Handling call
0709 12:18:02.967462 (+1195507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.999913  2184 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.010443 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1706) took 1223 ms. Trace:
I20260709 12:18:02.999578 31971 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:18:03.016721 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.016885 32114 rpcz_store.cc:276] 0709 12:18:01.788267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.795442 (+  7175us) service_pool.cc:224] Handling call
0709 12:18:03.010376 (+1214934us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.021924 32038 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:03.022176  2215 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.017673 31967 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.023097  2167 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.023658  2184 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.024289  2184 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.017112 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1539) took 1229 ms. Trace:
I20260709 12:18:03.024585  2215 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.022521 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:02.976404  2279 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.026249 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:03.022114 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 303622us
I20260709 12:18:03.002995  2159 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:03.011662  2221 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.472s	user 0.011s	sys 0.004s
I20260709 12:18:03.025271 31971 rpcz_store.cc:276] 0709 12:18:01.789871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.798773 (+  8902us) service_pool.cc:224] Handling call
0709 12:18:03.017049 (+1218276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.978411  2282 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:03.011247  2142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.015s	sys 0.013s
I20260709 12:18:03.029700 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 256) took 1076 ms. Trace:
I20260709 12:18:03.041394 32116 rpcz_store.cc:276] 0709 12:18:01.952826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.000770 (+1047944us) service_pool.cc:224] Handling call
0709 12:18:03.029645 (+ 28875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.041992 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 255) took 1089 ms. Trace:
I20260709 12:18:03.045562 32117 rpcz_store.cc:276] 0709 12:18:01.952531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.016824 (+1064293us) service_pool.cc:224] Handling call
0709 12:18:03.041934 (+ 25110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.045753  2069 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:03.048666 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:03.001529  1670 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.050690 32112 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:03.028646 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 580) took 1224 ms. Trace:
I20260709 12:18:03.010830  2216 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.051016 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 254) took 1098 ms. Trace:
I20260709 12:18:03.053401 32118 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:03.055932 32119 rpcz_store.cc:276] 0709 12:18:01.952241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.007473 (+1055232us) service_pool.cc:224] Handling call
0709 12:18:03.050630 (+ 43157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.018282  2273 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.027140  2050 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.035981  2215 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:03.060583 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 253) took 1104 ms. Trace:
I20260709 12:18:03.062541 32118 rpcz_store.cc:276] 0709 12:18:01.951854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.985659 (+1033805us) service_pool.cc:224] Handling call
0709 12:18:03.055842 (+ 70183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.063045 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1614) took 1305 ms. Trace:
I20260709 12:18:03.052145  2167 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.054663  2282 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.070060 32116 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.069413 32117 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:03.038870  2280 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.070950 32113 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.052716 32654 rpcz_store.cc:276] 0709 12:18:01.804445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.804817 (+   372us) service_pool.cc:224] Handling call
0709 12:18:03.027313 (+1222496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.064363  2081 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:03.065676 32047 rpcz_store.cc:276] 0709 12:18:01.752621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.756303 (+  3682us) service_pool.cc:224] Handling call
0709 12:18:01.756723 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.757129 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:01.949804 (+192675us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.057344 (+1107540us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.057390 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:03.057419 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.057493 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.057602 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26,"prepare.queue_time_us":205911,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":37555,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":1076107,"threads_started":3,"wal-append.queue_time_us":954767}
I20260709 12:18:03.075471  2205 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.079615  1670 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.078948  2081 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:03.087877  2312 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:18:03.076526  2215 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:03.089147  2314 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:18:03.083877  2184 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:03.095656 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.096779  2300 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.452s	user 0.004s	sys 0.044s
W20260709 12:18:03.099377 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.103914  2279 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:03.106091  2317 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:18:03.110210 32113 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:03.111341 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 234) took 2105 ms. Trace:
I20260709 12:18:03.111838 31970 rpcz_store.cc:276] 0709 12:18:01.009022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.009145 (+   123us) service_pool.cc:224] Handling call
0709 12:18:01.750932 (+741787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.786337 (+ 35405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.111041 (+1324704us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400312c50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.111251 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667796352}
W20260709 12:18:03.112890 32113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1695) took 1448 ms (client timeout 1689 ms). Trace:
W20260709 12:18:03.113413 32113 rpcz_store.cc:269] 0709 12:18:01.664577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.666479 (+  1902us) service_pool.cc:224] Handling call
0709 12:18:03.112579 (+1446100us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54001234d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.112808 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3206026112}
I20260709 12:18:03.092427  2282 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:03.118651  2213 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.974s	user 0.008s	sys 0.009s
I20260709 12:18:03.128729  1670 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.129438  1670 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.093765  2081 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.132539 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.133226  2285 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:03.133555  2220 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.133617  2081 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.137352 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 486) took 2758 ms. Trace:
I20260709 12:18:03.133941 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.136489  2050 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:03.135119  2156 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:03.134641 31965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1835) took 2518 ms (client timeout 2197 ms). Trace:
I20260709 12:18:03.146128 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:03.143836 32117 raft_consensus.cc:2410] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:18:03.146529 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.080714  2155 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:03.147166 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1852) took 2625 ms (client timeout 1995 ms). Trace:
W20260709 12:18:03.147228 31965 rpcz_store.cc:269] 0709 12:18:00.616245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.701171 (+ 84926us) service_pool.cc:224] Handling call
0709 12:18:03.134205 (+2433034us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400413a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.134447 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981591424}
I20260709 12:18:03.143441 32657 rpcz_store.cc:276] 0709 12:18:00.379364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.379511 (+   147us) service_pool.cc:224] Handling call
0709 12:18:00.379793 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.795094 (+1415301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.136952 (+1341858us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:03.137250 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3157157248}
I20260709 12:18:03.115866  2282 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:03.162067 32121 rpcz_store.cc:269] 0709 12:18:00.521760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.522452 (+   692us) service_pool.cc:224] Handling call
0709 12:18:03.146892 (+2624440us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54002f9c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.147054 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5579531136}
I20260709 12:18:03.142765  1670 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.144361  2210 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.142272  2142 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.159358  2273 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:03.172358 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 103713us
I20260709 12:18:03.144361  2280 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:03.173990 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 202343us
I20260709 12:18:03.145751  2215 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.147006  2156 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:03.179919 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.181994 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.38398e+06us
I20260709 12:18:03.134533  1971 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:03.168748  2282 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.191834 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:03.195431 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.179399  1670 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.197351  1670 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.143639  2220 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:03.210071 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.210608 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.194509  2216 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.214040 32117 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:03.216071 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 1.26226e+06us
I20260709 12:18:03.216958  2242 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.217718  2156 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.231494  2282 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.237016  2282 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:03.179199  2285 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.210651  2323 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:03.208408  1971 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.243384  2325 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:03.243438  2215 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.244355  2155 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.257686 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:03.266330  1971 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:03.259730 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:03.267694 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43657.1us
I20260709 12:18:03.207398  2142 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.271486  2215 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.272586  2321 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.275767  2216 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:03.278240 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 252) took 1338 ms. Trace:
I20260709 12:18:03.278265 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 258) took 1146 ms. Trace:
I20260709 12:18:03.278805 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.283676 32116 rpcz_store.cc:276] 0709 12:18:02.131660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.047347 (+915687us) service_pool.cc:224] Handling call
0709 12:18:03.061803 (+ 14456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.076710 (+ 14907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.277484 (+200774us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b5400147dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:03.277643 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18559872}
I20260709 12:18:03.268639  2156 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.283490 32010 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:18:03.288888 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:03.277175  2280 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.306483 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:03.307087 32116 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.308179 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:03.319870  2337 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.094s	user 0.002s	sys 0.000s
I20260709 12:18:03.283576 32114 rpcz_store.cc:276] 0709 12:18:01.930195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.021943 (+1091748us) service_pool.cc:224] Handling call
0709 12:18:03.022331 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.022545 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:03.206204 (+183659us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.264622 (+ 58418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.264688 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:03.268240 (+  3552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.268505 (+   265us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.268626 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35112,"prepare.queue_time_us":192353,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":39353,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":3954,"thread_start_us":195680,"threads_started":2,"wal-append.queue_time_us":12978}
I20260709 12:18:03.325841 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:03.283522  2216 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:03.327004 32117 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:03.304857  2321 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.304651  2282 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:03.280582  2323 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.330003  2324 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:03.334046  2216 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.342803 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:03.353029 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 141724us
I20260709 12:18:03.352054 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 257) took 1327 ms. Trace:
I20260709 12:18:03.358678 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:03.362382  2285 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.362771  2321 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:03.363422  2285 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:03.363937  2220 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.363824 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.364756  1904 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.365685 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.366196 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1837) took 2323 ms. Trace:
I20260709 12:18:03.366430  2330 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.366976  2330 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.366649 31963 rpcz_store.cc:276] 0709 12:18:01.043046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.052475 (+  9429us) service_pool.cc:224] Handling call
0709 12:18:01.101970 (+ 49495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.102421 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.102474 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:01.103039 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.635585 (+532546us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:01.668381 (+ 32796us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.693492 (+ 25111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.704764 (+ 11272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:03.096061 (+1391297us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:03.135607 (+ 39546us) raft_consensus.cc:1685] finished
0709 12:18:03.135735 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.365940 (+230205us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b5400069850. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.366042 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29388,"prepare.queue_time_us":13339,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":31327,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":23198,"spinlock_wait_cycles":129004544,"thread_start_us":387423,"threads_started":1,"wal-append.queue_time_us":515510}
I20260709 12:18:03.369066  2330 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.377851 32045 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:03.379540  2142 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:03.379588 32045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 276) took 1693 ms (client timeout 1752 ms). Trace:
I20260709 12:18:03.380584  2276 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:03.380398 32045 rpcz_store.cc:269] 0709 12:18:01.686109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.688226 (+  2117us) service_pool.cc:224] Handling call
0709 12:18:03.379402 (+1691176us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54001941d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.379532 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3849647488}
I20260709 12:18:03.381323  2276 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.381887  2276 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.363845  2285 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:03.382956  2285 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.386446  2331 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:03.387115  2331 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.387621  2331 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.390352  2322 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:03.391204  2322 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.393847  2322 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.394446  2342 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.395078  2342 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.397449  2342 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.398598 32114 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:03.399729 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.400929 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.401544 32114 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
W20260709 12:18:03.410526 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59291.6us
I20260709 12:18:03.411317 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.366722  1670 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.337815  2280 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.418658 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:03.341293  2218 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:18:03.419737  1670 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:03.420151  1670 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:03.420392  2280 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:03.423725  1670 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.425239  1670 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.425335  2323 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.332716  2325 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.426599  2215 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.429047  2325 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.347523  2324 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.429728  2324 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.434527  2326 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:03.435214  2273 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:03.435580  2326 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:03.436019  2326 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:03.437187  2326 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.439828  2326 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.442641  2346 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.443470  2330 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.443356  2346 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.444044  2330 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:03.448944 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.449122 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1616) took 1695 ms. Trace:
I20260709 12:18:03.450693  2319 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:03.451522 32651 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:03.451293  2319 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.452850 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:03.453364 32659 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:03.453563 32045 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.453874  2319 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.454669  2320 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.454764 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:03.455324  2320 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.456089 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:03.456676 32656 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:03.347141  2282 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.438625 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:03.440418  2279 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:03.480027  2279 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.482363 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 504) took 1837 ms. Trace:
I20260709 12:18:03.482784 32652 rpcz_store.cc:276] 0709 12:18:01.645079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.647813 (+  2734us) service_pool.cc:224] Handling call
0709 12:18:01.648289 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.958928 (+1310639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.482082 (+523154us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400154d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:03.482255 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493284608}
I20260709 12:18:03.438407  2335 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.465977 32047 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:03.485630 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1698) took 1742 ms. Trace:
I20260709 12:18:03.485808  2335 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.466615  2327 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.467079 32046 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:03.467439 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:03.467566  2343 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.467985  2323 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.354121 32120 rpcz_store.cc:276] 0709 12:18:02.019209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.035590 (+1016381us) service_pool.cc:224] Handling call
0709 12:18:03.036006 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.067738 (+ 31732us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:03.281186 (+213448us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.346525 (+ 65339us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.346576 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:03.346634 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.346692 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.346796 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":147513,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":4618,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6023,"thread_start_us":233751,"threads_started":2}
I20260709 12:18:03.472893 32042 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:03.473852 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1699) took 1720 ms. Trace:
I20260709 12:18:03.473923 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:03.474766 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 263122us
W20260709 12:18:03.476800 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58248.4us
I20260709 12:18:03.477263 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1603) took 2917 ms. Trace:
I20260709 12:18:03.462671 32657 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:18:03.479001 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.447186  2346 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.430554 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:03.486274 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.486976 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:03.680316 32047 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.543224 32116 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:03.577579 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.449652 32041 rpcz_store.cc:276] 0709 12:18:01.753489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.758955 (+  5466us) service_pool.cc:224] Handling call
0709 12:18:01.759402 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.759800 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:01.789657 (+ 29857us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.190225 (+1400568us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.448450 (+258225us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b1800421660. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:03.448665 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.448734 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:03.448796 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.448893 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.449050 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13563,"prepare.queue_time_us":204701,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":122,"raft.queue_time_us":318413,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":247,"spinlock_wait_cycles":11435776,"thread_start_us":1325314,"threads_started":3,"wal-append.queue_time_us":1088579}
I20260709 12:18:03.451370  2069 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.512645 32045 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:18:03.678762 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.679417 32039 rpcz_store.cc:276] 0709 12:18:00.559455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.559567 (+   112us) service_pool.cc:224] Handling call
0709 12:18:01.562505 (+1002938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.778014 (+215509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.475645 (+1697631us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400033ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:03.477035 (+  1390us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400033ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.477167 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720447104}
I20260709 12:18:03.486057 32112 rpcz_store.cc:276] 0709 12:18:01.743355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.743520 (+   165us) service_pool.cc:224] Handling call
0709 12:18:01.764028 (+ 20508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.052415 (+1288387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.485399 (+432984us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000807d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:03.485567 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2958559616}
I20260709 12:18:03.686137  2069 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.686540 32658 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:03.686496  2327 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.688755  2069 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.689909  2327 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.690174 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:03.487088 32038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 275) took 1804 ms (client timeout 1818 ms). Trace:
W20260709 12:18:03.691457 32038 rpcz_store.cc:269] 0709 12:18:01.682288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.684842 (+  2554us) service_pool.cc:224] Handling call
0709 12:18:03.486941 (+1802099us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.487037 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3063674240}
I20260709 12:18:03.446148  2330 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.680516 32115 rpcz_store.cc:276] 0709 12:18:01.753665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.753787 (+   122us) service_pool.cc:224] Handling call
0709 12:18:03.473680 (+1719893us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002b57d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:03.473795 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415527168}
I20260709 12:18:03.697649 32044 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:18:03.619459 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.701229 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.692173 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.685988  2213 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:03.722530 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.487465  2279 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:03.727461 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:03.678335  2320 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.678428 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:03.728156 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:03.746912 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 316508us
I20260709 12:18:03.620626 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:03.709160 32116 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:03.710466 32121 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:03.750612 32120 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:03.704005 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:03.752383 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.755268  2282 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.727566  2323 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.756172 31867 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 6, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 6 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:03.757987  2323 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.728605 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:03.758526  2267 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.118s	user 0.041s	sys 0.059s
W20260709 12:18:03.760015  2307 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.533s	user 0.004s	sys 0.003s
W20260709 12:18:03.759999  2267 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.175s	user 0.041s	sys 0.059s
W20260709 12:18:03.760666 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.761195 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.761896 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 274) took 2080 ms. Trace:
I20260709 12:18:03.759109 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:03.755988  2282 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.759325 32039 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:18:03.755189 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46192.1us
I20260709 12:18:03.759299 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:03.775413 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:03.778445 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.778940 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.779119 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.727228 32043 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:03.780733 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:03.780892 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56494.6us
I20260709 12:18:03.748190 32115 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:18:03.781466 32045 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:03.783396 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:03.784209 32041 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:03.785362 32044 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:03.762280 32040 rpcz_store.cc:276] 0709 12:18:01.681923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.682480 (+   557us) service_pool.cc:224] Handling call
0709 12:18:01.683036 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.683615 (+   579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.683644 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:01.698531 (+ 14887us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.787273 (+ 88742us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:02.338130 (+550857us) log.cc:844] Serialized 14 byte log entry
0709 12:18:02.344800 (+  6670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.344911 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:03.465489 (+1120578us) spinlock_profiling.cc:243] Waited 66.2 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:03.761642 (+296153us) raft_consensus.cc:1685] finished
0709 12:18:03.761723 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.761842 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":224,"prepare.queue_time_us":176146,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":1256,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":14526,"spinlock_wait_cycles":152154496,"thread_start_us":449582,"threads_started":2,"wal-append.queue_time_us":251833}
I20260709 12:18:03.785357 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.785534 32658 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:03.795161 32043 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:03.795814 32039 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.796298  2320 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.796936 32038 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:03.798974 32116 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:18:03.799362 32116 pending_rounds.cc:85] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 2
I20260709 12:18:03.799595 32116 pending_rounds.cc:107] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:18:03.799844 32116 raft_consensus.cc:2889] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:03.781232 31938 connection.cc:256] RPC call timeout handler was delayed by -1.5973542550000275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
I20260709 12:18:03.814199 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.802040 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:03.743073  2346 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.764415  2218 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:03.815155 32114 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:03.782869  2282 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.825079 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:03.702248  2343 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:03.828287 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 124376us
I20260709 12:18:03.829330 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.829783  2267 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:03.830262 32115 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 2: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:03.830292  2267 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.830952  2267 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:03.834604 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.836040 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.837266 32113 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:03.837313 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.837656  2321 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:03.800525  2285 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.838642  2321 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:03.838866 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.839973 32121 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:03.840530  2343 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.839041  2321 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:03.841230 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 139335us
I20260709 12:18:03.841797  2321 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.842602 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:03.843537  2321 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.843999 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:03.844832  2327 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:03.845644  2327 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:03.845664  2081 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.000s
I20260709 12:18:03.847227 32118 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:03.844136  2267 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.848179 32047 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:03.851297 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.851459 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.851966 31967 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:18:03.852203 32117 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:03.853202 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.853842 31970 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:18:03.854059  2156 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.000s
W20260709 12:18:03.855512 31938 connection.cc:256] RPC call timeout handler was delayed by -1.5244542549999096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260709 12:18:03.855728  2267 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.849275 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:03.856047  2350 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.822099  2218 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:03.856950  2320 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:03.857565 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55588.6us
I20260709 12:18:03.857473  2350 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.858700 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.859953 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:03.822245 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:03.860596  2155 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.000s
I20260709 12:18:03.860682 32120 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:03.861073  2350 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.824587  2050 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.861990 31963 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:03.862290 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:03.862879 32658 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:03.861409  2346 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.857455  2216 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:03.864233 31938 connection.cc:256] RPC call timeout handler was delayed by -1.5109542549998878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
I20260709 12:18:03.863847 32119 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:03.847918  2276 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:03.869450 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.869889 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:03.857679  2218 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:03.823211 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 131754us
I20260709 12:18:03.866799 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.871899 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:03.872856  2340 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:03.873260 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:03.874926 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 111196us
I20260709 12:18:03.876672 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:03.817324  2323 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.877765 32658 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:18:03.871964 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:03.883211 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:03.885144  2216 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.885869  2267 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:03.886464  2267 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:03.887290 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 209029us
I20260709 12:18:03.887456  2267 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.888372 31965 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:03.889076  2142 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:03.890878 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.890765  2330 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.891753  2330 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:03.890283 31938 connection.cc:256] RPC call timeout handler was delayed by -1.4002542549999362s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 12:18:03.893003 31971 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:03.893769  2330 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.894198 32039 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.894284  2330 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:03.894930  2330 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.896046 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.896899  2330 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.897102  2343 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.898213 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:03.898694 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.899385 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.899887  2346 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.900000 31964 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:03.900761 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.901496 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.897750 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:03.903215 31970 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:18:03.904038  2285 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.904412 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:03.905027 31971 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.897704  2343 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.908033  2343 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.908829 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:03.909552 31965 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.916040 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:03.917438  2330 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.917557 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:03.919629  2350 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.919656 31970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:03.920463  2352 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:03.920622  2330 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:03.921121  2352 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:18:03.922202  2330 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.922648  2330 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:03.924073  2352 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:03.863047  2335 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:03.933756  2216 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:03.933775 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64462.9us
W20260709 12:18:03.883320 31936 leader_election.cc:336] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.777s (SENT)
I20260709 12:18:03.926419 31970 raft_consensus.cc:606] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:03.935559 31970 raft_consensus.cc:614] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:03.932082 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55508.8us
I20260709 12:18:03.936691 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:03.933138  2323 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:03.937604 32039 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:03.938383 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:03.940567 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 347394us
I20260709 12:18:03.932188 31965 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:03.931850 31971 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:03.933097  2213 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:03.933223 32656 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:03.863915  2282 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.931264 32038 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:03.943459 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:03.944108 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:03.949288 32039 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:03.935134  2216 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:03.939168 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:03.945071  2280 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:18:03.951337 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.950299 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:03.953131 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.954679 32119 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:03.942209  2285 raft_consensus.cc:2764] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:18:03.957144 32115 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:03.958773 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:03.967391 32038 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.959095 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:03.947001  2213 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:03.959458 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.959677  2335 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.941960  2322 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:03.960057 31968 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:18:03.977134 31968 pending_rounds.cc:85] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 3
I20260709 12:18:03.977402 31968 pending_rounds.cc:107] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:18:03.977639 31968 raft_consensus.cc:2889] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:03.960204  2323 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:03.949527  2196 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.949s	user 0.009s	sys 0.002s
I20260709 12:18:03.976125 32118 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:03.976554  2213 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.958060  2352 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:03.979951  2347 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.180s	user 0.004s	sys 0.000s
I20260709 12:18:03.987764 31867 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:03.983326  1670 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 12:18:03.985975 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:03.981240  2331 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 12:18:03.991140 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:03.995416 32039 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:03.976871  2335 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:03.980300  2322 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.002430 32660 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:03.985975 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:03.994863  2343 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.003767  2331 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:03.993181 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:04.015254 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 131935us
W20260709 12:18:04.019606 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67133.6us
I20260709 12:18:04.020925 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.021785 32039 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:03.986657  2285 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:04.027168 31970 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.027372 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.006650  2335 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.018213  2320 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.029268 31867 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 3 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:04.006513  2322 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.031452 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.030092 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:03.983309  2196 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.037616 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:04.055083 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:04.038587  2350 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.002776  2330 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.039039  2346 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:04.056980  2350 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.039710  2344 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:04.040699  2331 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:04.041538 32038 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:04.043428  2321 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:04.043908 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:04.044633 31970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:04.046633  2276 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.052693  2282 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.019186  2142 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:04.064401 31970 raft_consensus.cc:606] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:04.064930 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.079332 31970 raft_consensus.cc:614] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:04.079509 32039 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:18:04.033356  2216 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.067574 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:04.058264  2350 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.087747 31867 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:18:04.083865  2282 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.091287 31970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:04.089627 31867 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 5, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 5 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:04.094753  2321 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.095543  2321 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.095927  2321 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:04.096905  2321 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.061725  2342 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:04.053586 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:04.078401 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 92499.5us
I20260709 12:18:04.083845  2196 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:04.085884 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:04.021525  2343 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.134670 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.099871 32652 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.098230  2321 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.079775  2320 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.081243  2346 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.075552 31971 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:04.165671 31970 raft_consensus.cc:606] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:04.166074 31970 raft_consensus.cc:614] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:04.159623 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.102895  2342 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.133612 32653 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:04.173614 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 167690us
I20260709 12:18:04.174327 31963 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 12:18:04.179739 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.180852 32039 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:04.184087 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 129066us
I20260709 12:18:04.182232 31867 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:04.185865 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 142084us
I20260709 12:18:04.166587  2320 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.170539  2346 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:04.175455  2342 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:04.176273  2279 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:04.201813 31963 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.205374 32039 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:18:04.208832 31867 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 7, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:04.207554 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:04.210332 31867 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 3 to 5, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:04.191656  2320 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.203442  2346 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.219882 31963 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.206489  2279 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:04.225924 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 232788us
I20260709 12:18:04.225997  2344 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:04.230533  2344 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:04.235321 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:04.236696 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.237424 31963 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:04.243158 32039 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:18:04.245224 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.228633 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.295955 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:04.313695 31867 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 2, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 2 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:04.315652 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.316480 32046 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:04.228128  2050 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:18:04.234031  2365 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.322736 31867 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:04.323479 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:04.231932  2362 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:04.334422 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.246261  2142 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:18:04.337172 31867 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 5, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 5 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:04.245523  2218 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:04.350919  2357 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 12:18:04.351960 31869 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 242) took 11963 ms (client timeout 15000 ms). Trace:
I20260709 12:18:04.352901 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.353730 32041 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.354522  2081 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:04.355034 32038 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:04.355418 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.356247 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.356968 32042 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:04.328975  2365 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:04.357476  2365 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.357829 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.326345  2322 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:04.352378 31869 rpcz_store.cc:269] 0709 12:17:52.251585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.251977 (+   392us) service_pool.cc:224] Handling call
0709 12:18:04.208608 (+11956631us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:59.249783 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:59.386561 (+136778us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:59.386597 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:17:59.386613 (+    16us) tablet.cc:662] Decoding operations
0709 12:17:59.705610 (+318997us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:59.705691 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:59.705732 (+    41us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:18:00.056049 (+350317us) tablet.cc:701] Row locks acquired
0709 12:18:00.056081 (+    32us) write_op.cc:260] PREPARE: finished
0709 12:18:00.079042 (+ 22961us) write_op.cc:270] Start()
0709 12:18:00.099258 (+ 20216us) write_op.cc:276] Timestamp: P: 1783599480073081 usec, L: 0
0709 12:18:00.099303 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 12:18:00.347029 (+247726us) log.cc:844] Serialized 6964 byte log entry
0709 12:18:00.681657 (+334628us) op_driver.cc:464] REPLICATION: finished
0709 12:18:00.800293 (+118636us) write_op.cc:301] APPLY: starting
0709 12:18:00.816150 (+ 15857us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:00.836817 (+ 20667us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:00.851062 (+ 14245us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:00.929136 (+ 78074us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:00.929448 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:18:00.929498 (+    50us) write_op.cc:312] APPLY: finished
0709 12:18:00.929856 (+   358us) log.cc:844] Serialized 119 byte log entry
0709 12:18:00.930057 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:00.930235 (+   178us) write_op.cc:454] Released schema lock
0709 12:18:00.930298 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:01.000748 (+ 70450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:01.001827 (+  1079us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":60685,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":31898,"apply.run_cpu_time_us":5461,"apply.run_wall_time_us":152014,"num_ops":17,"prepare.queue_time_us":181258,"prepare.run_cpu_time_us":26585,"prepare.run_wall_time_us":1246663,"raft.queue_time_us":87549,"raft.run_cpu_time_us":8431,"raft.run_wall_time_us":31540,"replication_time_us":577533,"thread_start_us":333714,"threads_started":4,"wal-append.queue_time_us":194541,"wal-append.run_cpu_time_us":15699,"wal-append.run_wall_time_us":1519440}]]}
I20260709 12:18:04.360190  2322 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.354871 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.354601  2343 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:04.337409  2368 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:18:04.361757 32047 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:04.362135 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.372130 32039 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:04.371933 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:04.374537 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:04.375392 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 267874us
I20260709 12:18:04.340756  2326 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.379809  2213 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.370759 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.388712 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.390074 32657 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.355062  2081 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.343080  2363 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.392056 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.395998 32042 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:04.393138 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:04.384654  2324 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.954s	user 0.012s	sys 0.000s
I20260709 12:18:04.403645  2324 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.405833  2324 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:04.407047  2364 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:04.408692 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.409371 32043 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.410616 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:04.412317 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:04.413228 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.416625  2344 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.417285  2218 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.380942  2267 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.417935  2344 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.402606 32654 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:04.418236  2267 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:04.398940  2081 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:04.422349 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.440084 32045 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:04.422639 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.441156 32041 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:04.423475 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.423339  2365 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.384495  2326 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:04.370913  2335 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.401717  2050 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:04.425112 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.382570  2362 raft_consensus.cc:2764] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:04.431574 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.431672 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.463801 32043 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.458501 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:04.438474 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.431768 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.465288 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:04.443768 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.466365 32042 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:04.441983 32040 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.467360 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:04.467942 32119 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.333681  2279 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.452095 32038 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:04.456128 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.470906 32047 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.432126 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.443040  2267 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.473049 32039 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:04.444936  2142 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.463037 32044 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:04.438097  2363 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:04.473963 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 178140us
I20260709 12:18:04.442883  2327 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:04.475485 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 89301.2us
I20260709 12:18:04.473239 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.483970 32045 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:04.484860 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:04.485435 32118 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.490049 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:04.492079 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:04.493364  2365 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.457018  2050 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:04.507738 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60452.3us
W20260709 12:18:04.509037 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 115959us
I20260709 12:18:04.494108 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:04.517307 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:04.515164 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.470932  2279 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.522223 32115 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:04.523828 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.523998 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.524574 32044 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:18:04.525835 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.526448 31963 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.528129 32115 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 12:18:04.478566  2142 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.528615 32115 pending_rounds.cc:85] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 3
I20260709 12:18:04.530934 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.565073 32115 pending_rounds.cc:107] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:18:04.575903 32115 raft_consensus.cc:2889] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:04.567101 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.568049 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.568745 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.577031 31972 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.574501 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:04.578673 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.579972 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.530045 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.531373 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:04.575405 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.495085  2367 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.520411  2050 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.591104 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.591557 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.591759 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.605221 31971 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.494568  2369 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:04.580719 32115 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:04.523478  2370 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.603672 31965 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.603777 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.606422 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.503615  2213 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.523518  2196 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:04.616629 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.494179  2365 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
W20260709 12:18:04.617115 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85846.4us
I20260709 12:18:04.620257 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.566893  2142 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.621682 32115 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:04.494799  2343 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 12:18:04.506163  2342 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.527225  2282 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
W20260709 12:18:04.615118  2350 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.003s
I20260709 12:18:04.623457 32120 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.590941  2367 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.564231  2327 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:04.625770 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.641736 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:04.642640 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:04.606671  2376 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.646818 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:04.653404 32660 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:04.614910  2369 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:04.643579 32119 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:04.619294  2370 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:04.660318 31965 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:04.680718 31971 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:04.682530 32112 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:04.650820  2367 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.683904 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:04.685456  2331 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:04.685945  2331 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:04.686789  2331 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.687945  2335 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:04.689198  2373 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:18:04.690475  2327 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:04.690977 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.691946  2370 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.691674  2327 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.708048  2342 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:04.709203  2342 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:04.712945  2370 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.655246  2350 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.634672  2346 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.736984  2350 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:04.737555  2346 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:04.714823  2196 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:04.744793 32653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 579) took 2940 ms (client timeout 1711 ms). Trace:
I20260709 12:18:04.744817  2370 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:04.718719 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.724388  2050 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.725986 32116 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:18:04.762867 31970 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:04.755502 32658 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:04.766196 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 200742us
W20260709 12:18:04.767488 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.783911 32121 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:04.792879 32118 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:04.675163  2369 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:04.764986  2321 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.002s
W20260709 12:18:04.799842 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.800396 31972 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
W20260709 12:18:04.801807 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.763384  2196 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:04.811090 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.813143 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.813439  2330 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:04.635438  2326 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:04.829993 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.827235 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.830852 32658 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:04.798695 32654 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:04.840128 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.842713 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.843917 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.844599 32042 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:04.845268 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.845755 32042 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:04.745428 32653 rpcz_store.cc:269] 0709 12:18:01.804058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.804269 (+   211us) service_pool.cc:224] Handling call
0709 12:18:04.744476 (+2940207us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540066e9d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:04.744669 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499836672}
I20260709 12:18:04.845759 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.822988  2346 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.853225 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.856110 32039 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:04.853484 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.853816 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.857348 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.857715 32045 raft_consensus.cc:2376] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8. Re-sending same reply.
I20260709 12:18:04.827232  2370 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.855022 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.861230 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.861783 31971 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:04.858237 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.862974 32047 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:04.860507 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.860664 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.863909 31972 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:04.865612 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:04.866227 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 354435us
I20260709 12:18:04.866837 32046 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:04.867307 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 387104us
I20260709 12:18:04.870568 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.870638 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.871304 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:04.884346 31965 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:04.871591 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.830200  2196 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.885308 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.886997 31969 raft_consensus.cc:2368] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
W20260709 12:18:04.920862 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 280282us
I20260709 12:18:04.940716 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.941011 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:04.941342 31965 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:04.871548 32043 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:04.941829 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.885236 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.942390 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:04.942616 32041 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:04.888689 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.943756 31971 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:04.944341 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:04.945019  2323 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:04.945537 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:04.946480  2375 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:04.946574  2390 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:04.947434  2390 consensus_queue.cc:1243] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:18:04.948020  2375 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:04.943282 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:04.938443 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:04.952647 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 585) took 1207 ms. Trace:
I20260709 12:18:04.885859 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.953398 31972 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:04.953152 32657 rpcz_store.cc:276] 0709 12:18:03.745179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.745316 (+   137us) service_pool.cc:224] Handling call
0709 12:18:03.783681 (+ 38365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.394304 (+610623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.952453 (+558149us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400592254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:04.952595 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376122112}
I20260709 12:18:04.959297 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:04.962811 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.963382 32042 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:18:04.933665  2384 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:04.963476 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:04.952479 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:04.975672 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:04.970711 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:04.975813 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:04.970721  2395 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:18:04.974489 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:04.977463 32656 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:04.978602 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:04.979606  2378 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:04.980333  2378 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.980849  2378 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:04.981001  2335 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:04.981366  2335 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:04.981773  2378 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.983278 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.982191  2335 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.984740  2335 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.985080 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:04.985705 32040 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:04.986750 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:04.951758  2320 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.000s
I20260709 12:18:04.988013  2320 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:04.988382 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.987684 32041 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:04.993978 32656 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:04.998661  2392 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:04.999150  2392 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.999513  2392 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.000200 31971 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:04.885462  2321 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.116s
I20260709 12:18:04.970948  2213 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.999675 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:04.862444  2346 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.005757 32042 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:05.007109 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:04.999451 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.008397 32039 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:05.015277 32656 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:05.018049 31963 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:05.002869  2213 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.029146 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:05.021955 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50573.5us
I20260709 12:18:05.035188 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.035339 32047 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.035804 32038 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.868249  2370 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.039196 32655 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:05.039644 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61082.5us
I20260709 12:18:05.040319 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:05.042673 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.941461  2081 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:04.974457  2323 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.990247  2320 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:05.006001  2381 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.999156  2050 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:05.051287 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 87860.7us
I20260709 12:18:05.056670 32115 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:05.034915  2213 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.057406  2218 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:05.058405  2369 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.059306  2324 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:05.075721 32653 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:05.084188  2324 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:05.045737  2365 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.083539  2369 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:05.085129  2324 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:05.086855  2388 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:05.092650 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:05.093087 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:05.093389 32121 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:05.094928 32121 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:05.098950 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.101274  2081 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:05.102479 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:05.103248 32115 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:05.104068 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 83655.3us
I20260709 12:18:05.105587 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:05.109254 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:05.110579 32047 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:05.111593 32038 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:05.112555 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.113299 32038 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:05.114229  2344 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:05.115406  2344 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:05.115868  2344 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:05.116025  2390 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:05.116428  2390 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:05.116796  2344 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.117190  2390 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.118458  2390 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.118712  2330 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:18:05.120589 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85928.6us
I20260709 12:18:05.121799 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.122699  2391 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.123123  2391 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.123436  2391 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.127306 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:05.127534  2267 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.000s
I20260709 12:18:05.127900  2367 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.127975  2267 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.007439  2382 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:18:05.055106  2323 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.145437 32115 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:05.146710 32120 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:05.077659  2279 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:05.148623 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 174649us
I20260709 12:18:05.150267 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.151278 32038 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:05.152406 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.132057  2320 raft_consensus.cc:2764] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:05.136840 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.162108 32047 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:05.061322  2381 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:05.146111  2384 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:05.152643 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.138388 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:05.160782 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 209481us
I20260709 12:18:05.162981 32113 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:05.102670  2374 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:05.160378 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1553) took 1152 ms. Trace:
I20260709 12:18:05.131899  2370 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.139305  2365 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:05.130281  2367 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.161213  2350 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.166790 32038 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:05.147356  2323 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:05.168445 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 192826us
I20260709 12:18:05.183254 32047 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:18:05.149886  2369 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.168907 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:05.178546 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.160948  2395 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.167640 31968 rpcz_store.cc:276] 0709 12:18:03.999836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.999989 (+   153us) service_pool.cc:224] Handling call
0709 12:18:04.003627 (+  3638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.003871 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:04.295307 (+291436us) log.cc:844] Serialized 14 byte log entry
0709 12:18:04.506011 (+210704us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b180041a520. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:05.040188 (+534177us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.151957 (+111769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.152004 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:05.152045 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.152107 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.152201 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111474,"prepare.queue_time_us":309897,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":9781,"raft.queue_time_us":190744,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":7594,"spinlock_wait_cycles":12457728,"thread_start_us":764450,"threads_started":3,"wal-append.queue_time_us":508201}
I20260709 12:18:05.158874 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:05.157343  2391 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.157692 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:05.166018  2381 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.202979 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:05.164296  2142 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 12:18:05.175529  2365 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.217165  2365 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.191704  2218 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.181154  2350 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.191591  2369 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.197330  2404 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:05.204051 32116 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:05.186281  2330 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:05.183872  2370 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.161849  2276 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:18:05.217847 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.219748 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.219187 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:05.274240 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55144.8us
W20260709 12:18:05.275081 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55424.3us
I20260709 12:18:05.222900 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:05.218645  2346 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.225143  2404 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.226181  2391 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.285491 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.285768 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:05.286072 32043 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.227367  2369 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:05.223323  2350 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:05.230983 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 78418.6us
I20260709 12:18:05.231590  2382 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.233328 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1556) took 1139 ms. Trace:
I20260709 12:18:05.302974 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.221339  2387 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:18:05.219892  2320 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.237270 32654 raft_consensus.cc:2368] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4.
I20260709 12:18:05.237462  2050 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.238282  2196 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.201931  2335 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.240041 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:05.247908  2326 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.249496 32660 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:05.217980  2142 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.251988 32658 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:05.252177  2381 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.253989  2279 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.253891  2392 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.254431 32046 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:05.271165 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:05.275745 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57996.3us
I20260709 12:18:05.220513 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:05.278040 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:05.278651 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.279049 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.284215 32114 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:05.269767 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:05.286819 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:05.299692 32038 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:05.299999 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.302186 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.304906 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:05.284633  2362 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.272326  2395 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.245685  2213 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.310091 32045 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:05.302862 31966 rpcz_store.cc:276] 0709 12:18:04.094388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.094520 (+   132us) service_pool.cc:224] Handling call
0709 12:18:04.094900 (+   380us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:04.214773 (+119873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.214810 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:04.245708 (+ 30898us) log.cc:844] Serialized 78 byte log entry
0709 12:18:04.917175 (+671467us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b1800048ca0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:18:05.050291 (+133116us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:05.182434 (+132143us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.189427 (+  6993us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.192567 (+  3140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.192643 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.233112 (+ 40469us) raft_consensus.cc:1685] finished
0709 12:18:05.233131 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.233270 (+   139us) inbound_call.cc:177] Queueing success response
0709 12:18:05.243626 (+ 10356us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b1800048ca0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1
0709 12:18:05.245404 (+  1778us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b1800048ca0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
Metrics: {"mutex_wait_us":2771,"prepare.queue_time_us":17496,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":216528,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":24136,"spinlock_wait_cycles":55024256,"thread_start_us":373723,"threads_started":3,"wal-append.queue_time_us":839635}
I20260709 12:18:05.285422  2322 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:05.258173  2365 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.314419 32044 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:05.315907 32041 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:05.363085 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:05.363399 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.363721 31968 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:18:05.284046  2394 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:18:05.363906 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:05.364631 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:05.365157 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:05.365880 31966 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:05.342816 32042 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:05.365990  2335 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.366989 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.367075  2142 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:05.359611  2352 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:05.368220  2352 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:05.343020 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81749.2us
W20260709 12:18:05.343945 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64975.3us
I20260709 12:18:05.369125  2352 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.370415  2403 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.370736 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.371125  2403 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.372040 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.373190 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.306028  2396 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:18:05.375296  2381 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:05.342242  2327 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:18:05.376965 32119 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:18:05.377034  2327 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:05.377234  2392 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.366194 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:05.363832 32113 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:05.363417 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:05.380086 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.369881 32040 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.348412 31963 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:05.351259 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:05.356247 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51404.9us
I20260709 12:18:05.285434  2404 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.312435  2050 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:05.357856 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:05.359058 31965 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 8. Ops: []
I20260709 12:18:05.361838 32039 raft_consensus.cc:2368] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8.
I20260709 12:18:05.388825 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:05.364051 32041 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:05.342126 32047 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:05.373288  2403 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.286510  2389 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.365794 31967 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:18:05.321835 32084 connection.cc:256] RPC call timeout handler was delayed by -1.7163325780002197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 12:18:05.386005 31969 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:05.386862 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:05.391984 32084 connection.cc:256] RPC call timeout handler was delayed by -1.7118325780002124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 12:18:05.392581 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6605325780001294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 12:18:05.392982 32084 connection.cc:256] RPC call timeout handler was delayed by -1.5417325780001647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:18:05.393293 32084 connection.cc:256] RPC call timeout handler was delayed by -1.5273325780001414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 12:18:05.393607 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4733325780002815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
W20260709 12:18:05.393903 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4076325780001753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 12:18:05.394196 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4022325780001665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 12:18:05.394584 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4013325780001651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 12:18:05.394883 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3005325780002295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
I20260709 12:18:05.395408 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:05.396014 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:05.401316 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.402451 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:05.403537 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:05.404165 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.404572 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.404632 32046 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:05.405875 31936 leader_election.cc:400] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4.
I20260709 12:18:05.406252 31936 leader_election.cc:403] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:18:05.408789 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:05.357750  2279 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.389318 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:05.355050  2395 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.411775 31970 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:05.412849 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:05.413718 31964 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:05.416160 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:05.342540 32653 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:18:05.411582 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:05.420289 31965 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:05.377672  2327 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:05.359894  2322 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.362023  2344 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.432984 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:05.435817 32658 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:05.410476  2389 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.400318  2391 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.308238  2382 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.385617  2267 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:05.318173  2320 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:05.444577 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.426016  2375 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:05.457233 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61900us
I20260709 12:18:05.400106  2342 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.458151 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:05.448699 31972 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:05.419667  2390 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.419355  2069 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:05.391382  2335 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:05.421667  2321 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.419660  2367 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:05.456007 31965 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:18:05.432087  2369 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.435441  2322 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.434929  2378 raft_consensus.cc:2764] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:05.442059  2389 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.421252  2370 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:18:05.451655  2330 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.450968  2391 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.461162  2342 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.465095  2335 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.420260  2346 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:18:05.452836  2218 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:05.457706  2375 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:05.506826 32038 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:05.531852 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.532514 32038 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:05.510882 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:05.535090 32044 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.390340  2392 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:18:05.498190  2394 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:05.449465  2381 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:05.448212  2387 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:18:05.488416  2367 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:05.490788  2344 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4.
W20260709 12:18:05.537325 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 179525us
I20260709 12:18:05.523962  2335 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:05.543326 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 123290us
I20260709 12:18:05.514742  2391 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.516166  2342 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.553275 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:05.572203 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:05.574883 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.543442  2394 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.539999  2375 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.501132  2321 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.576575 32653 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:05.577534  2324 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.593966  2324 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:05.595170  2324 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.596691  2324 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:05.600497  2410 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:18:05.608379  2327 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.609965  2327 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.573913 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.542217  2330 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:05.624863 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51026.8us
I20260709 12:18:05.575964  2403 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:05.482995  2050 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.589946  2394 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.554486  2367 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.590613 32038 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:18:05.552865  2218 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:05.614905 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53752.9us
W20260709 12:18:05.691012 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.498752 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 111556us
W20260709 12:18:05.621593 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 231712us
I20260709 12:18:05.584224 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:05.546962  2416 raft_consensus.cc:2764] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:18:05.696350 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.683162 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.670480 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 283673us
W20260709 12:18:05.688527 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 272452us
I20260709 12:18:05.699307 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:05.627177  2396 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:05.667274  2370 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.653978  2403 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:05.730090  2069 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.730873  2069 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:05.730966  2423 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:05.732221  2069 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.734263 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:05.737160 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:05.738755 32117 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:05.733505  2069 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.739167 32117 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:05.740110 32117 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.742156 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:05.741231 32117 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.676659  2365 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:05.759713 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.621062  2404 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.706233  2378 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.685844  2346 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.737874 32121 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.755332 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:05.783119 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 157286us
I20260709 12:18:05.784013 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:05.627616  2417 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:05.777834 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 193698us
I20260709 12:18:05.784947 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:05.785598 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:05.786375 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.787004 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:05.770994  2323 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.000s
I20260709 12:18:05.685582  2381 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:18:05.783411 32116 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.762429 32653 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:05.725831  2370 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:05.782013 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.746781  2396 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:05.793031 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.796036 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.770078  2342 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.761360  2331 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.802987 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:05.804548 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:05.787781  2320 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:05.817619 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:05.819834 32043 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.000s
I20260709 12:18:05.828258 32043 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:05.830401 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.773484  2282 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:18:05.797348  2380 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:05.835042 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:05.836575 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 137817us
I20260709 12:18:05.838747 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:05.840494 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44534.4us
I20260709 12:18:05.841259 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.798705  2142 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.798028  2415 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.811635  2362 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.806162  2417 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:05.834791  2196 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.788739  2378 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.810782  2376 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.876245 31964 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:18:05.770550  2330 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:05.902946 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 211114us
W20260709 12:18:05.922015 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48811.4us
I20260709 12:18:05.787792  2346 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.797879  2323 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:05.871506  2416 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:05.969718 32116 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:05.836354  2352 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:06.003173 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.840870  2156 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.844704  2331 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:05.845309  2415 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:05.921923  2365 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.004890  2434 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:06.007128  2420 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:18:05.920157  2320 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:06.021701 32044 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:05.978066  2142 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:18:06.033142 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 198134us
I20260709 12:18:06.002985  2416 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.842782  2218 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:05.840814  2403 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.022511  2415 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.060132 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:05.976110  2392 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 12:18:06.064847 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 260380us
I20260709 12:18:06.069875 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:06.070248 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:06.070574 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 92346.8us
W20260709 12:18:06.039497  2389 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.517s	user 0.003s	sys 0.006s
I20260709 12:18:05.926631  2362 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.027712  2321 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:06.064486  2424 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.075920 32656 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:06.068379  2323 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:06.079553  2350 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.771s	user 0.006s	sys 0.003s
I20260709 12:18:06.074759  2381 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:18:06.031010  2156 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.154873  2438 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:06.082297  2437 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:06.162580  2437 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.162915  2342 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.086023  2444 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:06.163662  2142 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:06.038874  2320 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.029858  2382 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:06.085451  2326 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.765s	user 0.010s	sys 0.000s
I20260709 12:18:06.062840  2416 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:06.150075 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62366.7us
I20260709 12:18:06.067018  2352 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.030162  2365 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.087244  2389 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.162985  2437 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:05.971328  2370 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.174023 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:06.156190 32121 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
W20260709 12:18:06.172366 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.112699  2081 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:06.142802 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 65476.4us
W20260709 12:18:06.178889 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.058954  2396 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:06.176721  2422 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.384s	user 0.001s	sys 0.000s
I20260709 12:18:06.180847  2438 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:06.181949  2438 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.154134  2196 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:18:06.186460  2352 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:06.187126  2335 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.000s
I20260709 12:18:06.187310  2352 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.187570  2335 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:06.189646  2322 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.000s
I20260709 12:18:06.189734  2335 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:06.161163  2442 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:18:06.160256  2413 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.160485  2282 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:06.160194  2156 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.160380  2427 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:06.158774  2213 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.216041 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:06.216255 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 375030us
I20260709 12:18:06.226950 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:06.176769  2320 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.228125 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:06.194739  2392 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.179447  2424 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:06.157119  2350 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.218770 31967 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.231920 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:06.247587 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 408956us
I20260709 12:18:06.190183  2322 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:06.241240  2395 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.806s	user 0.010s	sys 0.001s
I20260709 12:18:06.250419 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.248734 32121 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:06.253190 32045 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:06.253329 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:06.255892 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:06.254385 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.263255 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:06.265683 32112 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.266039 32047 raft_consensus.cc:2376] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7. Re-sending same reply.
I20260709 12:18:06.268731 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:06.268776 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.268973 32043 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:18:06.270064 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.270435 32118 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:18:06.271013 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.271744 32121 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:06.237311  2142 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.271855 32040 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.196555  2331 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.170969  2417 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.284740 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:06.301280 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.283715 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.284592 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.348435 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:06.271836 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 468881us
I20260709 12:18:06.349485 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.313769 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:06.299024 32117 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.002s
I20260709 12:18:06.304611 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.244764  2370 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.286762 32658 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:06.355948 32117 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:06.231230  2342 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:06.346822 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.351411 32121 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:06.364604 32043 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.270740 32039 pending_rounds.cc:85] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 1
I20260709 12:18:06.365192 32039 pending_rounds.cc:107] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:18:06.244273  2335 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:06.288677  2391 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.001s
I20260709 12:18:06.352247 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:06.326460  2411 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:06.365461 32039 raft_consensus.cc:2889] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:06.252213  2392 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:06.368104 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.246980  2282 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:06.374929  2400 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:06.376006 31966 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:06.273495  2346 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.312741  2378 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:06.376050 31964 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:06.248759  2081 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 12:18:06.347501  2320 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.434440  2320 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.437448  2320 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.366684  2387 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:06.385896 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:06.386169 31970 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:06.386579 31971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:06.392493 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:06.387598  2367 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.682s	user 0.010s	sys 0.000s
I20260709 12:18:06.451463 31963 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:18:06.274025  2434 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:18:06.253398  2156 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:18:06.451159 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:06.451175 31971 raft_consensus.cc:606] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:06.430135 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.394018 32655 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:06.386184  2342 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.462716 31968 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:06.497299 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:06.366247  2213 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:18:06.484766 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:06.465400 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 241151us
I20260709 12:18:06.376812  2391 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.301142  2445 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.197s
I20260709 12:18:06.387881  2335 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:06.465543 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:06.359575  2142 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.431869 32653 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:06.496068 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:06.496757 32043 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:06.288615  2395 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.393268  2400 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:06.504331 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:06.273406  2435 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 12:18:06.506811 31972 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:06.505290  2321 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:06.507961  2455 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 12:18:06.508813  2446 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:18:06.464536  2367 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.510944  2376 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.511356  2320 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.515115  2450 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:06.516889  2454 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:06.503659  2342 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.518725  2389 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:06.527909  2375 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.900s	user 0.006s	sys 0.005s
I20260709 12:18:06.529727 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.455070  2362 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.588110  2453 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:06.588289  2362 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.588762  2453 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:06.589144  2050 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.588912  2362 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.583609  2282 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.591003  2386 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:06.591439  2386 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:06.591711  2404 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.592370  2386 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.593461  2386 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.538676 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:06.549470  2321 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:06.545598 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:06.571393 32117 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:06.573552 31969 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.582929  2320 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.584700  2450 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:06.539011 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 153849us
I20260709 12:18:06.393154  2424 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.608182 31965 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.466881  2387 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:06.601035 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62425.5us
W20260709 12:18:06.612452 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66945.1us
I20260709 12:18:06.589702  2453 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.605916 32112 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:06.578430  2350 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.627128 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:06.595610  2322 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.618881 31968 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:06.621628 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.635288 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:06.602216 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:06.636055 31968 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:06.623026 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.625145  2367 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.625330  2465 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:18:06.647089 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:06.626582  2320 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.627094  2403 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:06.598810  2346 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:06.628512 31971 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:06.654249 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:06.655546 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:06.656013  2437 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.611059 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:06.657714 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:06.658501 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:06.659657 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:06.632135  2391 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.663286 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.663929 32039 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.632727  2362 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.636807  2321 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.638438 31963 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:06.668004 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:06.669011 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:06.669516 31963 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:06.598515  2375 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:18:06.684621 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.639411 31970 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:06.614802  2415 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.005s
I20260709 12:18:06.640393 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:06.685750 32658 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:06.640583 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.615866  2069 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.872s	user 0.015s	sys 0.002s
I20260709 12:18:06.690176 31970 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:06.608585  2081 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.537773  2392 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.580729  2376 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.616343  2213 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.650661 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:06.706017 31968 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:06.650955 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:06.656319 31971 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:06.708285 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57416.7us
I20260709 12:18:06.625171  2342 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.661597 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:06.662209 32115 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:06.631527 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:06.650302 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:18:06.639034  2324 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.042s	user 0.013s	sys 0.000s
I20260709 12:18:06.670279 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:06.653676  2400 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.675489  2387 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.675832  2416 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:06.676973  2327 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.049s	user 0.010s	sys 0.000s
W20260709 12:18:06.677428  2390 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.210s	user 0.008s	sys 0.002s
I20260709 12:18:06.677757  2450 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:06.679705  2142 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:06.682133  2395 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.653628  2346 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:06.700978 32040 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:06.728708 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:06.701649  2279 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.264s	user 0.009s	sys 0.004s
I20260709 12:18:06.704030 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:06.663870  2352 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:06.615370  2335 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.719010 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:06.703603  2394 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.985s	user 0.011s	sys 0.000s
I20260709 12:18:06.751307  2394 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.753367  2394 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:06.754472  2443 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:18:06.755208  2443 consensus_queue.cc:1243] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:06.755746  2462 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:06.718166  2081 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.759404 32114 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:18:06.722481  2321 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.724188 31970 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:18:06.730639 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:06.705138  2069 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.734229 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:06.765115 32040 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:06.732103  2390 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.739248 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:06.769323 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:06.740737 31969 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:06.743736  2142 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:06.744797  2376 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.747743  2416 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.749354  2346 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.735793  2279 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.750579 31968 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:06.750811  2352 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:06.722864 31963 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:06.766664 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:06.703109  2415 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.772310 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.776836 32084 leader_election.cc:400] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8.
I20260709 12:18:06.780109 32084 leader_election.cc:403] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:06.780191 32040 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:06.787516 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.767917  2342 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:06.801103 32112 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:06.773511  2381 raft_consensus.cc:2764] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:06.802567 32656 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.774204  2394 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.771777  2142 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:06.786976  2424 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.784296  2344 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:06.775694  2378 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:18:06.819715  2352 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.800034 32040 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:06.813687 31963 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:06.794795  2453 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8.
I20260709 12:18:06.827744 31963 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.809295  2346 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.784132  2069 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.814618  2142 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.805627  2342 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:06.817664  2344 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.808517  2380 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 12:18:06.871999 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.860949  2475 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:18:06.882828  2468 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:18:06.884222  2342 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:06.884961 32120 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:06.886363 32120 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:18:06.886711 32120 pending_rounds.cc:85] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
I20260709 12:18:06.886948 32120 pending_rounds.cc:107] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:18:06.887180 32120 raft_consensus.cc:2889] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:06.818662  2424 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:06.892079  2464 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:18:06.894946  2415 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:06.896219  2480 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:06.897986  2449 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 2 to 7, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:06.902051  2279 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:06.894064 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.903436 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.904476 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:06.904958 32044 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.907557  2069 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:06.872716  2375 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:06.879618  2471 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 12:18:06.915071 32039 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:06.844575  2344 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:06.935101  2458 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.435s	user 0.004s	sys 0.000s
I20260709 12:18:06.948170  2380 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:06.949065  2380 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.951802  2050 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.952514  2050 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.954964  2378 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:06.957736  2462 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:06.958269  2462 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:06.959403  2462 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:06.960748  2462 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.997195 32044 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:07.003660  2437 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:07.006942 32047 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:18:07.007459 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 201365us
I20260709 12:18:07.017758 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:07.024716  2081 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:07.008380  2050 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:07.025815  2142 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:07.111228  2485 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 12:18:07.113664 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.118220 31971 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:07.121652 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.119623 31972 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:18:07.167873 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 159604us
I20260709 12:18:07.169296 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:07.118619 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:07.081568  2081 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:07.286535 32120 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:07.288067 31972 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:07.587790 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.750926 32120 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.290609 32115 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:07.596271 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.761157 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1687) took 1030 ms. Trace:
W20260709 12:18:07.762809 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.763198 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 593949us
I20260709 12:18:07.072631  2279 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:07.827623 32038 rpcz_store.cc:276] 0709 12:18:06.730885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.760201 (+1029316us) service_pool.cc:224] Handling call
0709 12:18:07.760583 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.760783 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:07.760819 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:07.760866 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:07.760895 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.760993 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.761104 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:07.120086  2483 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:18:07.323621  2415 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.834407  2415 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:07.835466 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1575) took 2482 ms. Trace:
I20260709 12:18:07.835857 31967 rpcz_store.cc:276] 0709 12:18:05.353198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.412727 (+ 59529us) service_pool.cc:224] Handling call
0709 12:18:05.441443 (+ 28716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.245321 (+803878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.835215 (+1589894us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b5400313154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:07.835407 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787095296}
I20260709 12:18:07.296525  2487 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:07.838855  2346 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.842762 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:07.839398  2346 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:07.648638  2344 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.718s	user 0.009s	sys 0.001s
I20260709 12:18:07.847348 32120 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:07.816256  2449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.002s	sys 0.007s
I20260709 12:18:07.851257 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.852993 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:07.853744 31963 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.846669 31967 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:07.814038  2352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.004s	sys 0.004s
W20260709 12:18:07.762490  2331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.001s	sys 0.008s
I20260709 12:18:07.304540  2380 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:07.296792  2321 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:07.308228  2081 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:07.867619 31972 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:07.884335 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:07.304823  2365 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:07.866499  2352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.004s	sys 0.008s
I20260709 12:18:07.291857  2142 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:07.889599 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.897104 32120 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:07.889079  2331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.002s	sys 0.009s
I20260709 12:18:07.861399  2415 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:07.843015  2279 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:07.844347  2342 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.893884  2346 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:07.898100  2321 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:07.902812  2081 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:07.884753  2344 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:07.899175  2365 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.944391 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1911) took 1327 ms. Trace:
I20260709 12:18:07.945004 32112 rpcz_store.cc:276] 0709 12:18:06.617397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.622769 (+  5372us) service_pool.cc:224] Handling call
0709 12:18:07.944108 (+1321339us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54002f9254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:07.944302 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366407936}
I20260709 12:18:07.948788 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.950532 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.951409 32120 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:18:07.952718 32112 raft_consensus.cc:2368] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9.
I20260709 12:18:07.952911 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:07.954798 32120 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 2.
I20260709 12:18:07.955657 31963 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:07.954156 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:07.957582 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:07.958645 31971 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:07.921559  2342 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:07.963397 31972 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:07.927779  2365 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:07.966758 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:07.967185 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:07.969434 32120 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 3: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:07.961969 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:07.969368 31971 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:18:07.995175  2344 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:07.995651  2321 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:08.000187 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.003986 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.006392  2342 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.007566 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.002148  2344 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:08.009027  2342 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:08.010169  2335 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:08.016510 32045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1671) took 1748 ms (client timeout 1807 ms). Trace:
W20260709 12:18:08.016918 32045 rpcz_store.cc:269] 0709 12:18:06.268331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.268508 (+   177us) service_pool.cc:224] Handling call
0709 12:18:08.016322 (+1747814us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54001a72d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:08.016446 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660595328}
I20260709 12:18:08.009598  2279 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.011808  2346 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.030119 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1589) took 1400 ms. Trace:
I20260709 12:18:08.030637 31968 rpcz_store.cc:276] 0709 12:18:06.629961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.706820 (+ 76859us) service_pool.cc:224] Handling call
0709 12:18:06.707159 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.778634 (+ 71475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.029791 (+1251157us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b5400315454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.030046 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99304320}
I20260709 12:18:08.029331  2498 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.040598 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1587) took 1411 ms. Trace:
I20260709 12:18:08.041173 31970 rpcz_store.cc:276] 0709 12:18:06.629293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.691023 (+ 61730us) service_pool.cc:224] Handling call
0709 12:18:06.691500 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.766185 (+ 74685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.040193 (+1274008us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b540034dd54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.040526 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61818624}
I20260709 12:18:08.016424  2335 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.028290  2346 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.057500  2335 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.062641  2449 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:08.043325  2498 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.063264  2449 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:08.064919 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1872) took 1693 ms (client timeout 1655 ms). Trace:
I20260709 12:18:08.065263  2498 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:08.065322 31969 rpcz_store.cc:269] 0709 12:18:06.371496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.371606 (+   110us) service_pool.cc:224] Handling call
0709 12:18:08.064744 (+1693138us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54001ceb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:08.064843 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282049664}
I20260709 12:18:08.066170  2499 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.066708  2499 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.067915 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1676) took 1794 ms. Trace:
I20260709 12:18:08.068259 32047 rpcz_store.cc:276] 0709 12:18:06.273885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.625950 (+352065us) service_pool.cc:224] Handling call
0709 12:18:06.626271 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.007364 (+381093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.067697 (+1060333us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b5400379e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.067858 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874644224}
I20260709 12:18:08.069559 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1910) took 1615 ms. Trace:
I20260709 12:18:08.069892 32117 rpcz_store.cc:276] 0709 12:18:06.453912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.454055 (+   143us) service_pool.cc:224] Handling call
0709 12:18:08.069386 (+1615331us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54004399d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:08.069507 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86662400}
I20260709 12:18:08.074398  2499 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.042646  2365 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.042924  2496 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.050873  2342 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.076853  2342 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.065461  2449 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.078792  2342 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.084193  2346 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.089107 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1905) took 2329 ms. Trace:
I20260709 12:18:08.089514 32114 rpcz_store.cc:276] 0709 12:18:05.761094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.761500 (+   406us) service_pool.cc:224] Handling call
0709 12:18:05.761901 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.759778 (+997877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.088815 (+1329037us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b540013a954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.089024 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2293453696}
I20260709 12:18:08.089036 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1868) took 1719 ms. Trace:
I20260709 12:18:08.091058 31965 rpcz_store.cc:276] 0709 12:18:06.369383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.369504 (+   121us) service_pool.cc:224] Handling call
0709 12:18:06.369756 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.608620 (+238864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.088749 (+1480129us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54000f7654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.088957 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547912576}
W20260709 12:18:08.096500 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 89013.8us
I20260709 12:18:08.097256 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.098338 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.107026  2156 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:08.107450  2156 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:08.108354  2156 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.109683  2156 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.113667 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1865) took 1749 ms. Trace:
I20260709 12:18:08.114521  2344 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.114081 31964 rpcz_store.cc:276] 0709 12:18:06.363821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.363945 (+   124us) service_pool.cc:224] Handling call
0709 12:18:06.364229 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.391742 (+ 27513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.113421 (+1721679us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54000b6154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.113603 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26622336}
I20260709 12:18:08.143118  2365 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.144403  2365 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:08.145082  2463 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.642s	user 0.016s	sys 0.012s
I20260709 12:18:08.149840 32120 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:08.150678 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.151989 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:08.152814 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.49115e+06us
I20260709 12:18:08.152972 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.153880 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.141821  2344 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.105268  2350 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:08.177459 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.177695 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.178349 32120 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:08.178574 32117 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:08.181169 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.47715e+06us
I20260709 12:18:08.182175 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.183357  2387 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.183736  2365 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.184062 32120 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:18:08.184563  2486 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 12:18:08.180454  2470 raft_consensus.cc:2764] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:08.185420 32120 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:08.183913  2387 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.190328 32117 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:18:08.191406 31935 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.807s (SENT) [suppressed 20 similar messages]
I20260709 12:18:08.197085  2376 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.187952  2327 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:08.187098  2213 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:08.199632 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.198588  2327 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.201608  2069 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:08.202296  2069 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:08.202685  2069 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.203025  2327 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.204496  2327 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:08.204974  2327 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.206465  2403 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.206552  2213 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.207333 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.213984 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 299) took 1616 ms. Trace:
I20260709 12:18:08.179752  2346 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:18:08.190912  2423 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:08.222127 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1684) took 1522 ms. Trace:
I20260709 12:18:08.214531 32113 rpcz_store.cc:276] 0709 12:18:06.597451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.597659 (+   208us) service_pool.cc:224] Handling call
0709 12:18:06.614300 (+ 16641us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:06.621509 (+  7209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.621553 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:06.622073 (+   520us) log.cc:844] Serialized 52 byte log entry
0709 12:18:06.794568 (+172495us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:06.813489 (+ 18921us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.085995 (+1272506us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b180032f680. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:08.208095 (+122100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.208190 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:08.208246 (+    56us) raft_consensus.cc:1685] finished
0709 12:18:08.213306 (+  5060us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.213917 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6771,"prepare.queue_time_us":18764,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":3119,"raft.queue_time_us":11929,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":3705,"spinlock_wait_cycles":4654976,"thread_start_us":265968,"threads_started":2,"wal-append.queue_time_us":175922,"wal-append.run_cpu_time_us":2036,"wal-append.run_wall_time_us":1002869}
I20260709 12:18:08.198151  2376 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.179646  2344 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.223234 32039 rpcz_store.cc:276] 0709 12:18:06.700077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.917050 (+216973us) service_pool.cc:224] Handling call
0709 12:18:06.917318 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.917515 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:07.167416 (+249901us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.092028 (+924612us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b18002c8cc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:08.221671 (+129643us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:08.221746 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:08.221813 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.221901 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.222052 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1053990,"prepare.queue_time_us":75740,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":1054961,"spinlock_wait_cycles":12375808,"thread_start_us":99504,"threads_started":1}
I20260709 12:18:08.206241  2392 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.203006  2069 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.245352 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 295) took 2216 ms. Trace:
I20260709 12:18:08.245832 32119 rpcz_store.cc:276] 0709 12:18:06.028730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.028844 (+   114us) service_pool.cc:224] Handling call
0709 12:18:06.034859 (+  6015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.144837 (+109978us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:06.167196 (+ 22359us) log.cc:844] Serialized 14 byte log entry
0709 12:18:06.167627 (+   431us) log.cc:844] Serialized 14 byte log entry
0709 12:18:06.173221 (+  5594us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:06.173267 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:06.173311 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.839735 (+1666424us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b180032f4a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f
0709 12:18:08.153455 (+313720us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b180032f4a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:08.244178 (+ 90723us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b48002301d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:18:08.245114 (+   936us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540012b6d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.245155 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.245284 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38451,"prepare.queue_time_us":2012849,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":1818,"raft.run_cpu_time_us":65044,"raft.run_wall_time_us":2073157,"spinlock_wait_cycles":308251904}
I20260709 12:18:08.231350  2424 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.237663 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 148) took 10060 ms. Trace:
I20260709 12:18:08.238749 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:18:08.240338 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.207042  2403 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.241406 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.222338  2501 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:08.321782 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.328533 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.332134 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.305717  2465 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.331676  2081 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:08.375728 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:08.246627  2392 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.308332 31867 rpcz_store.cc:276] 0709 12:17:58.177761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.190278 (+ 12517us) service_pool.cc:224] Handling call
0709 12:17:58.402747 (+212469us) spinlock_profiling.cc:243] Waited 9.34 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890ab0b2 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:18:08.237594 (+9834847us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:04.782253 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:05.102008 (+319755us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:05.122404 (+ 20396us) write_op.cc:435] Acquired schema lock
0709 12:18:05.122430 (+    26us) tablet.cc:662] Decoding operations
0709 12:18:05.123678 (+  1248us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:05.123773 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:05.123807 (+    34us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:18:05.124989 (+  1182us) tablet.cc:701] Row locks acquired
0709 12:18:05.125045 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:18:05.125205 (+   160us) write_op.cc:270] Start()
0709 12:18:05.125368 (+   163us) write_op.cc:276] Timestamp: P: 1783599485125140 usec, L: 0
0709 12:18:05.125392 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:18:05.126135 (+   743us) log.cc:844] Serialized 4362 byte log entry
0709 12:18:05.565568 (+439433us) op_driver.cc:464] REPLICATION: finished
0709 12:18:05.708513 (+142945us) write_op.cc:301] APPLY: starting
0709 12:18:05.708637 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:05.709047 (+   410us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:05.709255 (+   208us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:05.711165 (+  1910us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:05.711676 (+   511us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:18:05.711759 (+    83us) write_op.cc:312] APPLY: finished
0709 12:18:05.712274 (+   515us) log.cc:844] Serialized 83 byte log entry
0709 12:18:05.712490 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:05.712618 (+   128us) write_op.cc:454] Released schema lock
0709 12:18:05.712680 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:05.715766 (+  3086us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:05.720970 (+  5204us) write_op.cc:454] Released schema lock
0709 12:18:06.009470 (+288500us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cc32a 00007f228c151f14 00007f228c151ec3 00007f228c151e38 00007f228c151df3 00007f228c1510dc 00007f228c17ae16 00007f228c162cbb 00007f228c14d39d 00007f228c150d75
Metrics: {"mutex_wait_us":535119,"reported_tablets":23,"spinlock_wait_cycles":21485952,"child_traces":[["op",{"apply.queue_time_us":124785,"apply.run_cpu_time_us":4642,"apply.run_wall_time_us":4643,"num_ops":11,"prepare.queue_time_us":244409,"prepare.run_cpu_time_us":47934,"prepare.run_wall_time_us":596604,"raft.queue_time_us":113661,"raft.run_cpu_time_us":18501,"raft.run_wall_time_us":509181,"replication_time_us":413723,"spinlock_wait_cycles":10035328,"thread_start_us":509720,"threads_started":4,"wal-append.queue_time_us":249011,"wal-append.run_cpu_time_us":19812,"wal-append.run_wall_time_us":1337878}]]}
I20260709 12:18:08.307227  2352 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:08.401243 32623 connection.cc:256] RPC call timeout handler was delayed by -1.4334229940000114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
I20260709 12:18:08.407374  2365 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:08.376657  2142 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.226830  2423 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.323621  2218 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.413913 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 231844us
I20260709 12:18:08.434623 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:08.430090 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:08.330667  2445 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 12:18:08.430790  2142 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.335314  2267 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:18:08.411238  2331 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.413461  2449 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:08.430573  2352 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.435190  2495 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:18:08.459877 32114 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:08.414433  2447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.890s	user 0.025s	sys 0.056s
I20260709 12:18:08.411252  2081 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.417937  2362 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:08.461042 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 81328.7us
I20260709 12:18:08.518074 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.519204 32045 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:08.462800 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:08.520748 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58067.6us
I20260709 12:18:08.521656 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:08.466629 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:08.446818 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.328434  2382 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 12:18:08.417042  2365 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:08.529204 32038 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:18:08.481088  2487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.008s	sys 0.016s
I20260709 12:18:08.378365  2465 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:08.529716 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 63161.5us
I20260709 12:18:08.536063 32040 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:08.525074 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:08.536921 32046 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:08.530681 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.532189 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.451829  2403 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.331162  2501 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.538502  2403 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.538995  2403 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.540247  2501 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.541193  2465 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.541867 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.542641 32047 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:08.543713  2423 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:08.544575  2493 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.646s	user 0.007s	sys 0.000s
I20260709 12:18:08.545708  2423 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.546828  2445 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.547273  2445 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.547602  2445 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.547591 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.548575 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.549799  2506 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:08.452075  2449 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.451018  2415 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.467290  2400 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.510519  2320 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.524432  2081 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.553097 32039 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:08.553829 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:08.578226 32042 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.541085 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.579239 32041 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:08.448849  2327 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.443362  2142 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.581151 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:08.581893 32115 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
W20260709 12:18:08.515046  2447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.994s	user 0.026s	sys 0.058s
I20260709 12:18:08.455216  2387 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.563903 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 297) took 2378 ms. Trace:
I20260709 12:18:08.568356 32038 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:08.569778 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.588361 32040 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:08.570672 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.571650 32046 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:08.552021 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.587591 32116 rpcz_store.cc:276] 0709 12:18:06.185963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.186437 (+   474us) service_pool.cc:224] Handling call
0709 12:18:06.186716 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.241703 (+ 54987us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:06.454749 (+213046us) log.cc:844] Serialized 14 byte log entry
0709 12:18:06.455076 (+   327us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:06.455128 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:06.455176 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.563659 (+2108483us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.563707 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.563842 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":332479,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1361,"raft.queue_time_us":76,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":35150,"spinlock_wait_cycles":125972736,"thread_start_us":199777,"threads_started":1}
I20260709 12:18:08.589798 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:08.551723  2279 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.552249  2424 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.459287  2218 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.592175 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.573025  2465 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:18:08.565291  2449 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:08.575155  2352 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:08.534628  2487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.008s	sys 0.016s
I20260709 12:18:08.510910  2499 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:18:08.446110  2344 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.593524 31938 leader_election.cc:400] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9.
I20260709 12:18:08.552438  2267 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:18:08.574642  2392 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.450640  2331 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.606007 32042 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:18:08.612927 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 421520us
I20260709 12:18:08.621548 31938 leader_election.cc:403] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:08.593928 32047 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:08.530280  2362 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.592170  2327 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.635039 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.642607 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:08.643282 32656 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:18:08.624047 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.645287 32039 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:08.594868  2423 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:08.607401  2342 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.608821  2447 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:08.607195  2350 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:08.610927  2501 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:08.622815  2495 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:08.659704 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:08.584216  2400 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.647689  2344 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9.
I20260709 12:18:08.647799  2376 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.614423  2387 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.671816 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:08.581954  2415 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.662058 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.642114  2327 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:08.620810  2487 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:08.608726  2279 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.660475  2499 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.664973  2496 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:08.733078 32045 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:08.736258 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:08.737388 32656 raft_consensus.cc:2410] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:08.664827  2392 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:08.730500 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68500.4us
W20260709 12:18:08.699157  2326 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.009s	sys 0.015s
I20260709 12:18:08.726258 32039 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.740512 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:08.741184 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:08.732226 32660 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:08.732084  2327 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.728968  2376 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:18:08.743902  2362 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.744602  2218 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.744913  2487 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.747713  2279 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:08.748462  2499 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.750232  2400 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.750690  2496 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:08.752293  2331 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.755767 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:08.683018  2350 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:08.694676  2495 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:08.623026  2365 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.679409  2424 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:18:08.729147  2369 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:08.682078  2342 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 12:18:08.729069  2387 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.666462  2447 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:08.825158 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69491.4us
I20260709 12:18:08.828303 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:08.738749  2465 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:08.867388 32652 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:08.766893  2326 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.009s	sys 0.016s
W20260709 12:18:08.839954 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.848702 32039 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:08.870971 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 388622us
I20260709 12:18:08.873095 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.811764  2415 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:08.873994 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45857.2us
I20260709 12:18:08.823300  2499 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.874995 32039 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:08.784904  2344 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.840618  2496 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:08.891548 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.776463  2392 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.893491 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:08.893493  2331 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:18:08.896944  2218 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.897459  2218 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:08.900007  2507 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:08.898656  2331 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
W20260709 12:18:08.902904  2513 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:18:08.904204 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1674) took 2630 ms. Trace:
I20260709 12:18:08.904654  2495 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.904538 32043 rpcz_store.cc:276] 0709 12:18:06.273347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.535888 (+262541us) service_pool.cc:224] Handling call
0709 12:18:06.572918 (+ 37030us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:08.180505 (+1607587us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:08.180551 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:08.188617 (+  8066us) log.cc:844] Serialized 52 byte log entry
0709 12:18:08.414072 (+225455us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:08.414128 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.539587 (+125459us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003c0854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:08.539636 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:08.903933 (+364297us) raft_consensus.cc:1685] finished
0709 12:18:08.903953 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.904147 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1450063,"raft.queue_time_us":1470217,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":361020800,"thread_start_us":215834,"threads_started":1,"wal-append.queue_time_us":461211}
I20260709 12:18:08.910669 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:08.911552  2218 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:08.911793  2508 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260709 12:18:08.918576  2350 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.923417  2342 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:08.924711  2342 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:08.925491  2342 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:08.928349  2447 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.930605  2387 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:08.865041  2346 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.869232  2365 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.871093  2465 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:08.941982 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 282340us
I20260709 12:18:08.875098  2424 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:08.840287  2069 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.613667  2470 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:08.949141 32039 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.895810  2403 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:08.949658  2320 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:08.948807  2465 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.949072  2424 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:08.896119  2344 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:08.937471  2346 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.992726 32039 raft_consensus.cc:2410] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:08.997335  2346 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:08.996826 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 464688us
I20260709 12:18:08.989334  2382 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:08.961477 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:09.015806 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:09.021054 32010 leader_election.cc:400] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:18:09.021785 32010 leader_election.cc:403] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:09.008646  2445 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:09.016837 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55442.3us
I20260709 12:18:09.026811 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:09.023703 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:09.003958  2382 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:09.052663 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:09.009670  2403 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.053738 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1686) took 2349 ms. Trace:
W20260709 12:18:09.057827 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 171768us
I20260709 12:18:09.065356 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.067113 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:09.065093 32044 rpcz_store.cc:276] 0709 12:18:06.704192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.998503 (+294311us) service_pool.cc:224] Handling call
0709 12:18:06.999949 (+  1446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.007252 (+  7303us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:07.009175 (+  1923us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.209122 (+1199947us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.782687 (+573565us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800413140. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:09.018993 (+236306us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.019558 (+   565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:09.019628 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:09.019747 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.019903 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.053524 (+ 33621us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1198655,"prepare.queue_time_us":954226,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":834,"raft.queue_time_us":1200111,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":63335,"spinlock_wait_cycles":33527808,"thread_start_us":234541,"threads_started":1}
I20260709 12:18:09.024724  2400 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.045049  2376 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6.
I20260709 12:18:09.072252 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:09.069530 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.060984  2506 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:09.085584 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.053287  2424 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.088608  2392 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:09.089586 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.097155 32039 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:09.092582 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.093133 32653 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:09.054736  2501 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:09.099431 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:09.098404 32044 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:09.101651 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.101967 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.102238 32044 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:09.102684 32039 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:09.078177  2400 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.103137 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.103633 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:09.123222 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 236957us
I20260709 12:18:09.121582 32039 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:09.120823 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 248393us
I20260709 12:18:09.094949  2369 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:09.125418  2369 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:09.128898 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.129437 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.129967 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.133023 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.133267 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.133739 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.119609  2514 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:18:09.122372 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:09.135779 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.136694 32044 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:09.138145 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.092088  2506 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:09.076231  2403 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:09.149149 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65998.7us
I20260709 12:18:09.185801 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:09.178923 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.212421 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.213124 32039 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:09.097728  2523 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:09.122337  2400 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.119594  2524 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:09.223884 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.225111 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:09.122300  2447 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:09.214884 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.099213  2515 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.123976  2376 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:09.119622  2528 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:09.097543  2510 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:09.227118 32116 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:09.243561 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:09.244496 32116 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:09.121661  2344 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:09.229548 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.245536 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.233258 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.215907  2424 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:09.186856  2081 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.597s	user 0.004s	sys 0.006s
W20260709 12:18:09.276887 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 227818us
I20260709 12:18:09.241339 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.233414 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.246701 32115 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:09.279222 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:09.246678 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.156199  2514 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.150367  2522 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:09.246120 32659 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:09.279989 32656 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:09.280894 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:09.285392 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.286058 32116 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:09.268085 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:09.240710 32039 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:09.278841 32117 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:09.282618 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.009s	sys 0.027s
W20260709 12:18:09.288407 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.009s	sys 0.028s
W20260709 12:18:09.288707 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 740143us
I20260709 12:18:09.119971  2531 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:09.119867  2518 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 12:18:09.119634  2520 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.314845 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.234678  2069 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:09.235549  2376 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.234097  2529 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.319710 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.328243 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.252238  2392 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:09.322048 32656 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:18:09.147481  2382 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.323045 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.235401  2528 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:09.181846  2213 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:09.277580  2538 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:09.249699  2526 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:09.153769  2516 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.203s
I20260709 12:18:09.318481  2487 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:09.340306 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 113855us
I20260709 12:18:09.239059  2447 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.310115  2495 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:09.345809 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 125318us
I20260709 12:18:09.347289 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:09.315788  2445 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:09.320443  2156 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.356899 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:09.328540  2376 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.234802  2524 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:09.363190 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53550.5us
I20260709 12:18:09.303755  2515 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:18:09.334702  2392 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:09.341360  2142 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.000s
I20260709 12:18:09.229787  2523 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.366230 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:09.364106 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:09.298271  2081 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.277415  2501 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.373407 32656 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:09.374537  2392 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:09.370162  2492 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.449s	user 0.008s	sys 0.002s
I20260709 12:18:09.364814  2069 raft_consensus.cc:2764] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:09.298153  2514 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.308502  2522 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.389055 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:09.353739  2365 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:09.256042  2510 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.383702  2524 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:09.418915 32040 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.399353 31965 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:18:09.400343 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80159.4us
I20260709 12:18:09.364733  2538 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:09.355118  2369 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:09.369246 31967 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:09.423786  2369 consensus_queue.cc:1243] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:09.423914  2499 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.376156  2520 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.425361  2520 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:09.425964  2499 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:09.427311 31965 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:09.427995 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.428586 31968 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:18:09.429765 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.431608 31970 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:09.433228 32040 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:09.411924 32121 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:18:09.414865  2522 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:09.387079  2530 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:18:09.445684  2327 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.000s
I20260709 12:18:09.446640 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.446398  2327 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.449412  2327 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:09.450646 32043 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:09.451187  2522 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:09.452057  2522 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:09.451787  2538 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.452564  2522 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:09.452725  2547 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 9.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:09.453765  2547 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.453553  2522 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.420233 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:09.436498  2449 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.814s	user 0.011s	sys 0.001s
I20260709 12:18:09.434279  2403 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:09.387602 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57247.2us
I20260709 12:18:09.330761  2506 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.364503  2156 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.439837  2524 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:09.440050 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:09.498535 32047 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.454278  2547 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.452126  2423 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.359508  2487 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.381214  2445 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:18:09.455855 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.456801 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.364729  2495 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:09.458231 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 273205us
I20260709 12:18:09.354034  2526 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:09.459281 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3417275270001028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 12:18:09.390141  2142 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.375118  2213 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:18:09.486698 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.493630 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:09.455157  2327 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.460672  2069 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:09.501675 32654 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:09.506290  2423 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.509459 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.512224 31968 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:09.514513 32656 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:09.455160  2499 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:18:09.537132  2527 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:09.538065 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.538785 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1554275270000289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 12:18:09.592308 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1500275270000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
W20260709 12:18:09.592705 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0213275270000395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
W20260709 12:18:09.593030 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0024275270000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
I20260709 12:18:09.593658 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:09.540098  2526 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:09.542259  2142 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:09.500802  2524 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:09.623246  2524 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.624569  2524 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.402078  2528 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.628034  2528 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.497090  2449 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:09.620007 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:09.630715 31969 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:09.620754  2544 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 12:18:09.622042  2494 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.655s	user 0.002s	sys 0.007s
I20260709 12:18:09.581177 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:09.588253  2423 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.572474  2069 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:09.572649  2549 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:09.674849  2549 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.672029  2156 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:09.673724  2470 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.002s
I20260709 12:18:09.674970  2069 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.3, Last appended by leader: 3, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.665024 31968 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:09.664863  2465 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.006s
I20260709 12:18:09.707909  2549 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:09.591958  2527 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:09.755678 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.756481 32119 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:09.756063  2527 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.756628  2514 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:09.757182  2527 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:09.679245 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1699) took 1112 ms. Trace:
I20260709 12:18:09.726054 31969 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:09.752720  2465 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:09.578280  2327 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.149683  2487 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:10.001005  2509 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:10.168742  2449 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:09.752092  2423 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.150336  2547 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.202196  2526 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:10.022418 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1671) took 1032 ms. Trace:
W20260709 12:18:09.955801  2536 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.542s	user 0.003s	sys 0.000s
W20260709 12:18:10.203089 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.941893  2346 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.944s	user 0.012s	sys 0.000s
I20260709 12:18:09.754984  2495 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.138831  2320 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.211767  2526 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:10.007938 32112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1770) took 1859 ms (client timeout 1770 ms). Trace:
I20260709 12:18:10.254937 32045 rpcz_store.cc:276] 0709 12:18:08.567293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.570391 (+  3098us) service_pool.cc:224] Handling call
0709 12:18:09.679086 (+1108695us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:09.679185 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331157248}
W20260709 12:18:10.201898 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 608261us
I20260709 12:18:10.187306 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:10.257097 32042 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:10.257351 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:10.212982 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.189258 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:10.258260 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.992143  2213 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:10.015574 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:10.199746  2538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.015s	sys 0.012s
W20260709 12:18:09.915310  2424 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.002s
I20260709 12:18:09.964568  2546 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:10.200572  2509 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:10.257409 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:10.285311 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:10.203433  2557 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.957391 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:09.915109  2496 consensus_meta.cc:287] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.974s	user 0.006s	sys 0.005s
I20260709 12:18:10.287850  2496 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.289093 31966 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:10.146245 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:10.158815 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:09.965837  2218 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:09.957720 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 158513us
I20260709 12:18:09.969763 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.208725 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:09.985426 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:10.301546 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.297083 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:09.893101 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.301185 32121 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:10.302255 32118 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:09.757308 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.303205 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.806859 31968 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:09.961897 31963 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:10.304850 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:10.024312 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:09.982187 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 589863us
I20260709 12:18:09.757849  2081 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:09.997217 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1705) took 1428 ms. Trace:
I20260709 12:18:10.188311 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:10.311223 32043 rpcz_store.cc:276] 0709 12:18:08.569026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.910418 (+341392us) service_pool.cc:224] Handling call
0709 12:18:09.996962 (+1086544us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b540013d3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:09.997135 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239917952}
I20260709 12:18:10.311921 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:10.140105 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.312240 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:10.312789 32043 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:10.310360 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.315745 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:10.194989 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.317150 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.317811 32116 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:10.264869 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:10.318804 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:10.265116 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:10.054737  2415 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.143s	user 0.006s	sys 0.007s
I20260709 12:18:10.211882 32040 rpcz_store.cc:276] 0709 12:18:08.990372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.066892 (+ 76520us) service_pool.cc:224] Handling call
0709 12:18:10.022167 (+955275us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b540012fa54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:10.022334 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807012736}
I20260709 12:18:10.323504 32040 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:10.324117 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1664) took 1588 ms. Trace:
I20260709 12:18:10.324422 32040 rpcz_store.cc:276] 0709 12:18:08.735707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.322988 (+1587281us) service_pool.cc:224] Handling call
0709 12:18:10.323279 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.323888 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.324067 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.265628 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:10.256079 32112 rpcz_store.cc:269] 0709 12:18:08.148172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.152754 (+  4582us) service_pool.cc:224] Handling call
0709 12:18:10.007684 (+1854930us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54002b52d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:10.007847 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220443008}
I20260709 12:18:10.325953 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.259654 31969 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:10.258229 32045 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
W20260709 12:18:09.870702  2350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.005s	sys 0.008s
I20260709 12:18:10.201428  2326 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:10.272392 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71114us
I20260709 12:18:10.272797 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:10.135665  2069 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.274215 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:09.960788  2400 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.725s	user 0.008s	sys 0.002s
I20260709 12:18:10.274966 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.277202 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:09.745030  2387 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.328549 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:10.281677 31970 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:10.329175 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:10.282739 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.329844 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1921) took 1004 ms. Trace:
I20260709 12:18:10.330220 32112 rpcz_store.cc:276] 0709 12:18:09.325308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.325735 (+1000427us) service_pool.cc:224] Handling call
0709 12:18:10.329791 (+  4056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:09.878667  2362 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:10.140498  2382 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.163302  2522 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.337860 32043 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:10.139423  2506 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.349072 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1660) took 1736 ms. Trace:
I20260709 12:18:10.194213 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:10.290899 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 144751us
I20260709 12:18:10.140983  2510 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.200946  2552 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:09.881908  2523 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:09.893415  2447 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:10.219192 32047 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.720s	user 0.010s	sys 0.006s
I20260709 12:18:10.358425 32047 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
W20260709 12:18:10.301265 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.306221 32113 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:10.306788 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:10.308287 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:10.360059 32044 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:10.361387 32044 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:10.361915 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1673) took 1359 ms. Trace:
I20260709 12:18:10.308287 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:10.362571 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:09.801252  2392 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:10.369858 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63133us
I20260709 12:18:10.313913  2540 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.258718 32041 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:10.282375 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.286561 32046 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:10.327629 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1922) took 1002 ms. Trace:
I20260709 12:18:10.332574 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:10.317168 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:10.333413 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.335347  2326 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.376724 32116 rpcz_store.cc:276] 0709 12:18:09.329080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.316964 (+987884us) service_pool.cc:224] Handling call
0709 12:18:10.327574 (+ 10610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.381507 32046 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:10.381652 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1670) took 1422 ms. Trace:
I20260709 12:18:09.759261  2376 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.335471 32045 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:10.335831 32040 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:10.293357  2546 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.334826 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:10.366339  2279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.417s	user 0.012s	sys 0.014s
W20260709 12:18:10.273968  2538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.015s	sys 0.012s
I20260709 12:18:10.306752  2327 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.343382 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:10.398125 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 70785.8us
W20260709 12:18:10.398505 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55201.1us
I20260709 12:18:10.400784 31938 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:10.291574  2445 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.258965  2346 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.335709  2362 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:09.982472  2529 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.352s
I20260709 12:18:10.272737  2320 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.308039  2424 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:10.331687  2415 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.411s	user 0.006s	sys 0.022s
W20260709 12:18:10.331203  2350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.009s	sys 0.023s
I20260709 12:18:10.300555  2218 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:10.290254  2496 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.275291  2213 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.323091  2547 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.986763  2342 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.347582 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.331048  2499 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.330773  2352 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.335011  2549 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.352952 31971 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:10.330070  2519 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.352442 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:10.342458  2541 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.338593  2400 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.256222  2526 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.354712 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.353551 32043 rpcz_store.cc:276] 0709 12:18:08.612459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.321379 (+1708920us) service_pool.cc:224] Handling call
0709 12:18:10.321669 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.348818 (+ 27149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.349017 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.361930  2509 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.337831  2387 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.212711  2423 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.362214 32044 rpcz_store.cc:276] 0709 12:18:09.003086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.360726 (+1357640us) service_pool.cc:224] Handling call
0709 12:18:10.361060 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.361717 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.361870 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:10.360085 32117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.006s
I20260709 12:18:10.368383 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.370843 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:10.327694 32042 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:10.377247 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44738.5us
I20260709 12:18:10.327739 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:10.495080 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 167404us
I20260709 12:18:10.355746  2552 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:10.497388  2560 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:10.498085  2552 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.498528  2387 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.498618 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1784) took 1269 ms. Trace:
I20260709 12:18:10.451761  2342 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.499097  2552 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.499266  2365 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:10.499863  2365 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:10.500883  2342 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.500684  2365 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.382149 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:10.376732 32657 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:10.384884 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.385856 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1667) took 1522 ms. Trace:
I20260709 12:18:10.336596 32085 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:10.387122 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 272) took 1047 ms. Trace:
I20260709 12:18:10.387571 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1668) took 1499 ms. Trace:
I20260709 12:18:10.384035 32047 rpcz_store.cc:276] 0709 12:18:08.959033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.359102 (+1400069us) service_pool.cc:224] Handling call
0709 12:18:10.381220 (+ 22118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.381374 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.381398 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.381434 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:10.381463 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.381510 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.381604 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32384}
I20260709 12:18:10.389667 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:10.290825  2567 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.390837 32041 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:10.366168  2142 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.355580  2518 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:18:10.374104  2369 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.396159 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:10.393198  2279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.450s	user 0.013s	sys 0.014s
I20260709 12:18:10.404091 32038 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:10.404187 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:10.404826 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:10.373862  2470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.012s	sys 0.017s
I20260709 12:18:10.388213  2523 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:10.357468  2535 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:18:10.387391  2554 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:10.344941  2551 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:18:10.405076 31870 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.431285 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.382577  2540 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.409054  2501 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:10.440651  2376 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:10.446244 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 93894.4us
I20260709 12:18:10.412297  2528 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.460152 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1603) took 1012 ms. Trace:
I20260709 12:18:10.356370 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.495431 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:10.495961 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 125221us
I20260709 12:18:10.497049 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:10.501222  2566 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:18:10.307718  2557 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.627455 32042 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:10.628245 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 304) took 1212 ms. Trace:
W20260709 12:18:10.629009 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 292467us
W20260709 12:18:10.613492 32117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.015s	sys 0.009s
I20260709 12:18:10.519127 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:10.509689 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1607) took 1060 ms. Trace:
I20260709 12:18:10.615226 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:10.494261 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1925) took 1119 ms. Trace:
I20260709 12:18:10.464845 32119 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:10.615954 31964 rpcz_store.cc:276] 0709 12:18:09.338564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.338695 (+   131us) service_pool.cc:224] Handling call
0709 12:18:09.338999 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.358930 (+ 19931us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:09.389985 (+ 31055us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.981908 (+591923us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b1800211a20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22900a358f 00007f22890e2571 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd
0709 12:18:10.363454 (+381546us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.386032 (+ 22578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.386079 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:10.386127 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.386182 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.386278 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":314696,"prepare.queue_time_us":533076,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":93,"raft.queue_time_us":319468,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":16526,"spinlock_wait_cycles":20246784,"thread_start_us":688596,"threads_started":2,"wal-append.queue_time_us":906566}
I20260709 12:18:10.516582 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:10.618209 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 213441us
I20260709 12:18:10.618842 32659 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:10.619242 32040 rpcz_store.cc:276] 0709 12:18:08.864119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.335295 (+1471176us) service_pool.cc:224] Handling call
0709 12:18:10.335614 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.385603 (+ 49989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.385805 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.618476 32045 rpcz_store.cc:276] 0709 12:18:08.888479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.334813 (+1446334us) service_pool.cc:224] Handling call
0709 12:18:10.335197 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.387288 (+ 52091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.387508 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.618712 32116 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.620822 32121 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:10.499112 32115 rpcz_store.cc:276] 0709 12:18:09.229132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.316873 (+ 87741us) service_pool.cc:224] Handling call
0709 12:18:09.317119 (+   246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.338184 (+ 21065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.338216 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:09.346599 (+  8383us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.508364 (+161765us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:09.508661 (+   297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.512980 (+  4319us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.498419 (+985439us) raft_consensus.cc:1685] finished
0709 12:18:10.498438 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.498562 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20734,"raft.queue_time_us":44841,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":5480,"thread_start_us":116309,"threads_started":1,"wal-append.queue_time_us":1023196}
W20260709 12:18:10.572594 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.612528 32118 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:10.504251 32039 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:10.620348 31970 rpcz_store.cc:276] 0709 12:18:09.447975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.455566 (+  7591us) service_pool.cc:224] Handling call
0709 12:18:10.459902 (+1004336us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:10.460075 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1769803008}
I20260709 12:18:10.440239  2547 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.628608 32042 rpcz_store.cc:276] 0709 12:18:09.415787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.272571 (+856784us) service_pool.cc:224] Handling call
0709 12:18:10.628190 (+355619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.698741 32113 rpcz_store.cc:276] 0709 12:18:09.374752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.368140 (+993388us) service_pool.cc:224] Handling call
0709 12:18:10.494190 (+126050us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:10.688678 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.620965 31870 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 6 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.423913  2424 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:10.520870  2553 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.313s	user 0.001s	sys 0.002s
W20260709 12:18:10.703738 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 208364us
I20260709 12:18:10.702533 31870 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.705047 32042 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:10.711254 32653 raft_consensus.cc:2376] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3. Re-sending same reply.
W20260709 12:18:10.621781  2470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.012s	sys 0.021s
I20260709 12:18:10.700158 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.703300 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1927) took 1318 ms. Trace:
I20260709 12:18:10.464871  2531 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:10.713264 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:10.700007 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:10.727684 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1611) took 1277 ms. Trace:
W20260709 12:18:10.728947 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.731650 32043 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [5.8-6.9]
W20260709 12:18:10.765345 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.630007  2554 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.739945 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 306) took 1323 ms. Trace:
I20260709 12:18:10.682912  2445 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.687889 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.729162  2523 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:10.728235 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 113102us
I20260709 12:18:10.453958  2496 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.308s
I20260709 12:18:10.798593 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:10.800040 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:10.629452  2528 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.741930 32116 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 3.
I20260709 12:18:10.613137  2423 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:10.517371  2465 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.519361  2326 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.758569 32119 rpcz_store.cc:276] 0709 12:18:09.384876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.345484 (+960608us) service_pool.cc:224] Handling call
0709 12:18:10.703234 (+357750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.465072  2541 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.629257  2501 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.523686  2552 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.205s
I20260709 12:18:10.696624  2387 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.527902  2213 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.765550 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 303) took 1350 ms. Trace:
I20260709 12:18:10.810181 32043 rpcz_store.cc:276] 0709 12:18:09.415569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.699690 (+1284121us) service_pool.cc:224] Handling call
0709 12:18:10.731329 (+ 31639us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:10.765265 (+ 33936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.765492 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.520792  2346 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.739033 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:10.525380  2392 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.766933 32115 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:10.846930 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1919) took 1522 ms. Trace:
I20260709 12:18:10.847430 32115 rpcz_store.cc:276] 0709 12:18:09.324743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.766486 (+1441743us) service_pool.cc:224] Handling call
0709 12:18:10.766766 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.846586 (+ 79820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.846852 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.702013  2447 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:10.466120  2519 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.534966  2362 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.516582  2218 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:10.503986  2400 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.769328 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1928) took 1384 ms. Trace:
I20260709 12:18:10.758750 31870 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 8, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:10.771102 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:10.766417 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 364149us
W20260709 12:18:10.664626 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.863261 31870 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 5, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:10.864271 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.803211  2470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58386 (request call id 77) took 5450 ms. Trace:
I20260709 12:18:10.770193 32042 rpcz_store.cc:276] 0709 12:18:09.416222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.704317 (+1288095us) service_pool.cc:224] Handling call
0709 12:18:10.704730 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.739576 (+ 34846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.739857 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:10.795255 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 361036us
I20260709 12:18:10.744467 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1918) took 1420 ms. Trace:
I20260709 12:18:10.730667  2525 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:10.856351  2551 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.801013 32657 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:10.802264 32116 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:10.815301  2415 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:18:10.744202  2567 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:10.763897  2566 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:10.699234  2547 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.468842  2350 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.309s
I20260709 12:18:10.452291  2352 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.762735  2531 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:10.764921  2344 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:10.699122  2369 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:10.900313  2579 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:18:10.901522  2369 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.901876 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 281) took 1412 ms. Trace:
I20260709 12:18:10.883831  2218 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.902366 32085 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:10.888270  2320 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.793346  2267 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.904162  2267 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.850761 32652 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.764979 31967 rpcz_store.cc:276] 0709 12:18:09.450098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.196566 (+746468us) service_pool.cc:224] Handling call
0709 12:18:10.196920 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.197180 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:10.388908 (+191728us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.697885 (+308977us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.727333 (+ 29448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.727386 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:10.727444 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.727504 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.727622 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":316760,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":59,"raft.queue_time_us":34503,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":25991,"spinlock_wait_cycles":94665984,"thread_start_us":494987,"threads_started":2}
I20260709 12:18:10.794150  2327 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.905841  2352 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.906374  2531 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:10.768720 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55578.3us
I20260709 12:18:10.907351 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:10.908555 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:10.909619 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:10.769578 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 307) took 1353 ms. Trace:
I20260709 12:18:10.862927 32011 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:10.627439  2529 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 12:18:10.910574  2519 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:10.809695  2501 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:10.910298 32047 rpcz_store.cc:276] 0709 12:18:09.416437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.728995 (+1312558us) service_pool.cc:224] Handling call
0709 12:18:10.769004 (+ 40009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.769278 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.769304 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.769344 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:10.769374 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.769429 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.769527 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":715,"spinlock_wait_cycles":53817728}
I20260709 12:18:10.912403 32047 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:10.912940 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1677) took 1465 ms. Trace:
I20260709 12:18:10.913220 32047 rpcz_store.cc:276] 0709 12:18:09.447272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.911948 (+1464676us) service_pool.cc:224] Handling call
0709 12:18:10.912235 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.912722 (+   487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.912890 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.862149 32113 rpcz_store.cc:276] 0709 12:18:09.385202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.699696 (+1314494us) service_pool.cc:224] Handling call
0709 12:18:10.769256 (+ 69560us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.865288 31870 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 6 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.915980 31870 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.916811  2540 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:10.884439  2525 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:10.917500  2525 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:10.917352 31870 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.785377  2554 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.804756  2528 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.902393 31968 rpcz_store.cc:276] 0709 12:18:09.489610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.304406 (+814796us) service_pool.cc:224] Handling call
0709 12:18:10.304712 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:10.321943 (+ 17231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.321990 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:10.337393 (+ 15403us) log.cc:844] Serialized 52 byte log entry
0709 12:18:10.461771 (+124378us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:10.511718 (+ 49947us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.512041 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.512189 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.512258 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.901659 (+389401us) raft_consensus.cc:1685] finished
0709 12:18:10.901679 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.901817 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52992,"prepare.queue_time_us":63289,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":27865,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":16756,"thread_start_us":105652,"threads_started":1,"wal-append.queue_time_us":462089}
I20260709 12:18:10.928202 32119 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:10.929179 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 314) took 1409 ms. Trace:
I20260709 12:18:10.929564 32119 rpcz_store.cc:276] 0709 12:18:09.519887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.927528 (+1407641us) service_pool.cc:224] Handling call
0709 12:18:10.927953 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.928859 (+   906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.929108 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.845072  2552 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:10.931540  2557 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:10.932598 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:10.933029  2424 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:10.850975  2553 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:10.873974 32121 rpcz_store.cc:276] 0709 12:18:09.324407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.704206 (+1379799us) service_pool.cc:224] Handling call
0709 12:18:10.743830 (+ 39624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.744063 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.744100 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.744156 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:10.744204 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.744271 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.744398 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.935679 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 315) took 1415 ms. Trace:
I20260709 12:18:10.875399 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:10.915282 32113 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:10.699622 31972 rpcz_store.cc:276] 0709 12:18:09.449080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.454414 (+  5334us) service_pool.cc:224] Handling call
0709 12:18:09.454769 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.984097 (+529328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.984121 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.148952 (+164831us) log.cc:844] Serialized 26 byte log entry
0709 12:18:10.376263 (+227311us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:10.431206 (+ 54943us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.433709 (+  2503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.433779 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.509422 (+ 75643us) raft_consensus.cc:1685] finished
0709 12:18:10.509449 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.509611 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":531227,"prepare.queue_time_us":134655,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":35,"raft.queue_time_us":548795,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":14206,"spinlock_wait_cycles":11520,"thread_start_us":249192,"threads_started":2,"wal-append.queue_time_us":358159}
I20260709 12:18:10.936039 32121 rpcz_store.cc:276] 0709 12:18:09.520221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.934839 (+1414618us) service_pool.cc:224] Handling call
0709 12:18:10.935119 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.935318 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.935344 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.935397 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:10.935446 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.935513 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.935629 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.885406 32660 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.682996  2509 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.887856 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1929) took 1502 ms. Trace:
W20260709 12:18:10.884810 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 387825us
I20260709 12:18:10.947597 32116 rpcz_store.cc:276] 0709 12:18:09.385554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.801517 (+1415963us) service_pool.cc:224] Handling call
0709 12:18:10.801857 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.887608 (+ 85751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.887798 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.948103 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:10.948915 32116 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:10.868852  2346 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.869673  2279 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:10.891249 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 255917us
I20260709 12:18:10.853992  2465 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:18:10.891532  2567 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.964404  2567 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.881673  2362 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.968505  2554 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:10.889163  2506 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.000s
I20260709 12:18:10.903456  2524 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:10.970767  2524 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.970923  2596 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.971201  2524 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.971383 32120 raft_consensus.cc:2410] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:10.971536  2596 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.971913  2449 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.972469  2449 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.910774  2523 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.872464  2470 rpcz_store.cc:276] 0709 12:18:05.344163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.367921 (+ 23758us) service_pool.cc:224] Handling call
0709 12:18:05.390860 (+ 22939us) log.cc:844] Serialized 487 byte log entry
0709 12:18:05.726610 (+335750us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b1800390220. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c128b0b 00007f228c1adb63 00007f228c1d5752 00007f228c1cd9fa
0709 12:18:08.330955 (+2604345us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b540019ba20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c144dbe 00007f228c14f930 00007f228c14f8d8 00007f228c14f858 00007f228c14f813 00007f228c14ea9c 00007f229203a2f6 00007f2292034aa8 00007f228c18b7cb 00007f228c18e2c0
0709 12:18:10.782443 (+2451488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153249,"raft.queue_time_us":175501,"raft.run_cpu_time_us":1543,"raft.run_wall_time_us":14603,"spinlock_wait_cycles":38236288,"thread_start_us":101961,"threads_started":1,"wal-append.queue_time_us":245380,"wal-append.run_cpu_time_us":53111,"wal-append.run_wall_time_us":3746016}
I20260709 12:18:10.915241  2541 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.937019 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1789) took 1506 ms. Trace:
I20260709 12:18:10.805851  2423 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:10.977125 32113 rpcz_store.cc:276] 0709 12:18:09.430960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.914724 (+1483764us) service_pool.cc:224] Handling call
0709 12:18:10.915064 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.936784 (+ 21720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.936969 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:10.978487  2423 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:10.938120 32121 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:10.979457 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1936) took 1300 ms. Trace:
I20260709 12:18:10.979777 32121 rpcz_store.cc:276] 0709 12:18:09.679189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.937586 (+1258397us) service_pool.cc:224] Handling call
0709 12:18:10.937826 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.979215 (+ 41389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.979405 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:10.867352 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 128362us
I20260709 12:18:10.942410 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:10.948026 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 85250.5us
I20260709 12:18:10.983126 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:10.940742 31870 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 5, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.954058 32660 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:10.953735  2376 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:10.955137  2392 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:10.985144 31870 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 7, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.906131  2267 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:10.961043  2320 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.978551 32113 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:18:10.988917 32113 pending_rounds.cc:85] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 3
I20260709 12:18:10.989161 32113 pending_rounds.cc:107] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:18:10.989367 32113 raft_consensus.cc:2889] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:10.953015 32116 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:18:10.987128 31870 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:10.981534 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:11.004318 31870 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.006560 31870 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 6, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 6 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.008579 31870 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 8, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.956713 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:10.955945  2586 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.067s	user 0.002s	sys 0.001s
I20260709 12:18:11.057029 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:11.057674 31969 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:10.976635  2528 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.016052 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:11.071022 31968 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:11.033819 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:11.071175 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 301) took 1656 ms. Trace:
I20260709 12:18:11.072211 31972 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:11.023980 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:11.056213 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 256242us
I20260709 12:18:11.075124 31964 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:11.075552 31968 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:10.955204  2279 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:11.059590 32654 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20260709 12:18:11.079255 31870 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.082805  2470 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:11.033644 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.023056 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:11.084584 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:11.084918  2470 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:11.086052 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:11.069726 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.088449 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:11.012091  2423 raft_consensus.cc:2764] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:11.011052 32652 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:10.971961  2596 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.002244  2376 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.010931  2519 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:18:11.085136 32113 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.954216  2346 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.071535 32046 rpcz_store.cc:276] 0709 12:18:09.415129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.385525 (+970396us) service_pool.cc:224] Handling call
0709 12:18:10.385788 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.386850 (+  1062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.386878 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.392452 (+  5574us) log.cc:844] Serialized 26 byte log entry
0709 12:18:10.857073 (+464621us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:11.048184 (+191111us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.070868 (+ 22684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.070937 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.070979 (+    42us) raft_consensus.cc:1685] finished
0709 12:18:11.070991 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.071121 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7142,"raft.queue_time_us":775,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":14713,"thread_start_us":593332,"threads_started":2,"wal-append.queue_time_us":525807}
I20260709 12:18:11.051251  2518 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:18:11.092180 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:11.123956 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.089921 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:11.125641 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:11.067987  2506 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:18:11.050954  2531 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.002537  2320 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.085023 31967 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.094933 31968 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:11.134393 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:11.099701  2502 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.104993 31972 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 12:18:11.123128  2527 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.906s	user 0.014s	sys 0.000s
I20260709 12:18:11.108767 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:11.115520 31964 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:11.091534 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.081737  2279 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:11.121649 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.057936  2557 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:18:11.126940 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:11.025285 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:11.190441 31971 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:11.024807 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:11.191175 31970 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:11.101215  2526 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:11.099664 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1606) took 1650 ms. Trace:
W20260709 12:18:11.107900  2522 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.762s	user 0.005s	sys 0.005s
I20260709 12:18:11.081029  2218 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:18:11.203747 31965 rpcz_store.cc:276] 0709 12:18:09.448818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.448923 (+   105us) service_pool.cc:224] Handling call
0709 12:18:09.449267 (+   344us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:09.984034 (+534767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.984062 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:10.148953 (+164891us) log.cc:844] Serialized 78 byte log entry
0709 12:18:10.378537 (+229584us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:10.794463 (+415926us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.805137 (+ 10674us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.099311 (+294174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.099403 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.099443 (+    40us) raft_consensus.cc:1685] finished
0709 12:18:11.099456 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.099614 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":583990,"prepare.queue_time_us":143070,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":29457,"raft.queue_time_us":653945,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":23878,"spinlock_wait_cycles":12946304,"thread_start_us":866710,"threads_started":3,"wal-append.queue_time_us":553478}
I20260709 12:18:11.128458  2495 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:11.082569 31870 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 5, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.212968 31870 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 7, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 7 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:11.218223 31870 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 8, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.108918  2595 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:11.120647  2346 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.188349 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:11.202706 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 75860.5us
I20260709 12:18:11.250717 32116 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:11.252215 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:11.253448 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.253775 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:11.137843 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:11.187402 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 230734us
I20260709 12:18:11.136050  2499 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:11.108542  2376 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:11.137019  2320 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:11.185050  2593 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:18:11.108357  2554 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:11.219842 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 190896us
I20260709 12:18:11.206350  2523 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.226712  2218 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:11.222667 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 826cffce8bb540c8beb56cc00a702782 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:18:11.296403 31854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 202063us
I20260709 12:18:11.206084  2415 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:11.136930  2267 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:11.238886 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.215346 32114 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:11.248704 32113 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
W20260709 12:18:11.320297 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.203682 32657 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:11.263348 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:11.220446 31967 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:18:11.321765 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58508.1us
I20260709 12:18:11.207089  2526 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:11.250625  2527 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:11.299187  2599 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:18:11.299866  2578 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.847s	user 0.003s	sys 0.000s
I20260709 12:18:11.311369  2528 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:11.271004  2382 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.926s	user 0.010s	sys 0.002s
I20260709 12:18:11.221131 31870 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 2, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 2 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.272097  2595 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:11.270359  2495 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:11.380805 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57670.6us
I20260709 12:18:11.216940  2501 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.318746  2415 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:11.392597 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 137217us
W20260709 12:18:11.397760 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.398562 31870 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.320248  2156 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.418092 32121 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:11.419905 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1658) took 2808 ms. Trace:
I20260709 12:18:11.442374 32039 rpcz_store.cc:276] 0709 12:18:08.611992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.289137 (+677145us) service_pool.cc:224] Handling call
0709 12:18:09.289466 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.710998 (+1421532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.419637 (+708639us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001a68d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:11.419846 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2793249024}
I20260709 12:18:11.331324  2526 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.271198  2547 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.458164 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1674) took 2454 ms. Trace:
I20260709 12:18:11.431282 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1939) took 1143 ms. Trace:
I20260709 12:18:11.458675 32041 rpcz_store.cc:276] 0709 12:18:09.004274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.375911 (+1371637us) service_pool.cc:224] Handling call
0709 12:18:10.376186 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.619070 (+242884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.457914 (+838844us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54000569d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:11.458103 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33290496}
I20260709 12:18:11.459318 32112 rpcz_store.cc:276] 0709 12:18:10.287977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.395913 (+107936us) service_pool.cc:224] Handling call
0709 12:18:11.431018 (+1035105us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400139a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:11.431204 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302706432}
I20260709 12:18:11.391845  2527 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:11.436393 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.439246 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:11.440574 31870 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 5 to 6, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 6 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.466250 32041 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:11.467447 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.467864 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1790) took 1268 ms. Trace:
I20260709 12:18:11.456681  2156 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:11.439284 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 357222us
I20260709 12:18:11.339345  2552 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:11.385779  2509 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:11.469798  2509 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.469663  2552 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.470196  2509 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:11.470441  2423 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.470988  2423 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.471889  2552 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:11.461452 32039 raft_consensus.cc:1217] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-8.5]   Dedup: 6.4->[8.5-8.5]
I20260709 12:18:11.473351 32039 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [4.3-8.5]
I20260709 12:18:11.473629  2423 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:11.391445  2445 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:18:11.474756 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.474773 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:11.475986 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.476375 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.476959 32039 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:11.477073 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.477766 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:11.463338 32112 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:18:11.478593  2279 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.480146 32113 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:11.480657  2279 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:11.481515  2531 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.481482 32039 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:11.483315  2362 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
I20260709 12:18:11.482991  2595 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.433557 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:11.463521 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.486505 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:11.487248 32045 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:18:11.487632 32045 pending_rounds.cc:85] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 2
I20260709 12:18:11.321236  2523 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.485136 31870 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 2 to 7, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 7 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:11.487430 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.487872 32045 pending_rounds.cc:107] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:18:11.489874 31963 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:11.490023 32045 raft_consensus.cc:2889] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:11.478082 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.468220 32041 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:11.468209 32119 rpcz_store.cc:276] 0709 12:18:10.199935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.944123 (+744188us) service_pool.cc:224] Handling call
0709 12:18:10.961673 (+ 17550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.962041 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.962068 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.963024 (+   956us) log.cc:844] Serialized 26 byte log entry
0709 12:18:11.170613 (+207589us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:11.170654 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.170721 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.467686 (+296965us) raft_consensus.cc:1685] finished
0709 12:18:11.467704 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.467814 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8884,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":207206,"threads_started":1,"wal-append.queue_time_us":355698}
I20260709 12:18:11.454175  2565 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.445745  2487 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.494668 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:11.495221 31870 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 5, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 5 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:11.460426  2499 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:11.493795 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60297.7us
I20260709 12:18:11.505468 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:11.491173 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:11.499725 31870 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 6 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:11.496178 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57020us
I20260709 12:18:11.507942 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:11.491209 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:11.455422  2557 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:18:11.496253  2279 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:11.491585  2362 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:11.513227 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:11.508241  2327 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.005s
I20260709 12:18:11.522535 32039 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:11.510296 31870 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:11.523818 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:11.514669 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.524740 32039 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:11.506763  2213 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.488963  2320 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.513204  2470 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:11.528143 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:11.518503  2601 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:18:11.530097 32039 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:11.478094  2376 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.531149 32043 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:11.521525  2567 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.003s
I20260709 12:18:11.510665  2423 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:11.510711  2535 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:11.512009  2614 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:11.498189  2346 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:11.514897  2524 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.003s
I20260709 12:18:11.507637  2487 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.526945  2327 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:11.507995  2565 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.513820  2518 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:18:11.530984  2470 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:11.522739  2362 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.573258 31966 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:11.581030 32042 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:11.532554  2213 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.560963  2535 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.599530  2376 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.600406  2213 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.600956  2535 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:18:11.511727  2607 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:18:11.577220  2470 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:11.601781  2607 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:11.532732  2527 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.548508  2614 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.606128 32043 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:11.609076 32046 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:11.566948  2392 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.612911 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:11.561769  2423 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:11.614651 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1785) took 2371 ms. Trace:
I20260709 12:18:11.615127 32118 rpcz_store.cc:276] 0709 12:18:09.242770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.701505 (+1458735us) service_pool.cc:224] Handling call
0709 12:18:10.742344 (+ 40839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.764152 (+ 21808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.764183 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.778929 (+ 14746us) log.cc:844] Serialized 26 byte log entry
0709 12:18:11.134931 (+356002us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:11.134975 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.135078 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.614368 (+479290us) raft_consensus.cc:1685] finished
0709 12:18:11.614395 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.614580 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":379,"raft.queue_time_us":23793,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":23753,"thread_start_us":261320,"threads_started":1,"wal-append.queue_time_us":478877}
I20260709 12:18:11.606927  2614 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.553864  2524 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.627699  2346 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.549320  2567 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.559820  2499 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:11.641198  2606 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:18:11.625284  2501 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:11.648149  2610 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:18:11.602295  2607 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:11.650367  2525 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.731s	user 0.007s	sys 0.003s
I20260709 12:18:11.657357  2567 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.654072  2547 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.664548  2607 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:11.665253  2596 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.010s
I20260709 12:18:11.666457  2546 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:11.666719  2611 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.137s	user 0.000s	sys 0.003s
W20260709 12:18:11.667084  2573 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.160s	user 0.003s	sys 0.002s
I20260709 12:18:11.667151  2594 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:11.667778  2594 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:11.668659  2594 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.670207  2594 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.665832  2596 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.678833  2614 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.680578  2614 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:11.524344 31870 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 3, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2), VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:11.731542 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 656491us
W20260709 12:18:11.734921 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 860834us
I20260709 12:18:11.736456 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:11.736855 32038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 308) took 2320 ms (client timeout 2281 ms). Trace:
W20260709 12:18:11.738859 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 863290us
W20260709 12:18:11.737344 32038 rpcz_store.cc:269] 0709 12:18:09.416658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.311708 (+895050us) service_pool.cc:224] Handling call
0709 12:18:11.736251 (+1424543us) spinlock_profiling.cc:243] Waited 91.3 ms on lock 0x7b5400217d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:11.736571 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210102144}
I20260709 12:18:11.761217 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.764873 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.765936 32118 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:11.767289 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 302) took 2351 ms. Trace:
I20260709 12:18:11.767577 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:11.764907 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:11.767841 32044 rpcz_store.cc:276] 0709 12:18:09.415357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.530749 (+1115392us) service_pool.cc:224] Handling call
0709 12:18:10.531239 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.531650 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.531690 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.532251 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:18:10.860951 (+328700us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:11.067194 (+206243us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.766243 (+699049us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b54000cb5a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14a733 00007f228c1cfdbf 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c
0709 12:18:11.766376 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.766570 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.766680 (+   110us) raft_consensus.cc:1685] finished
0709 12:18:11.766823 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.767013 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12290,"prepare.queue_time_us":246794,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":30294,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":200,"spinlock_wait_cycles":37263232,"thread_start_us":508818,"threads_started":2,"wal-append.queue_time_us":383120}
W20260709 12:18:11.777658 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.782156 32656 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:11.784096  2369 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:11.785219  2369 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.785763  2369 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:11.786439 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.787822  2369 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.619345  2392 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.792900  2400 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.651788  2423 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:11.798252 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.801271  2344 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.652070  2523 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:11.836504  2524 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:11.852275 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89433us
I20260709 12:18:11.853351 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:11.866567  2596 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.867909  2547 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.868860 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:11.866083 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 311) took 2448 ms. Trace:
I20260709 12:18:11.658144  2525 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.876821 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.874588 32042 rpcz_store.cc:276] 0709 12:18:09.417299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.872695 (+1455396us) service_pool.cc:224] Handling call
0709 12:18:10.873004 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.581501 (+708497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.865614 (+284113us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54001a4fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:11.865885 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1627792256}
I20260709 12:18:11.868883  2547 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:11.881331 32120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1916) took 2557 ms (client timeout 1804 ms). Trace:
W20260709 12:18:11.881821 32120 rpcz_store.cc:269] 0709 12:18:09.323598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.327957 (+  4359us) service_pool.cc:224] Handling call
0709 12:18:11.880836 (+2552879us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540010efd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:11.881090 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3776867584}
W20260709 12:18:11.892359 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:11.897368 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 105803us
I20260709 12:18:11.899462 32120 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:11.904125  2565 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.904788 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:11.905684 32651 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:11.907521 32660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 609) took 2500 ms (client timeout 2212 ms). Trace:
W20260709 12:18:11.908020 32660 rpcz_store.cc:269] 0709 12:18:09.407415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.407542 (+   127us) service_pool.cc:224] Handling call
0709 12:18:09.968364 (+560822us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:18:10.264988 (+296624us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5800004598. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aae9c 00007f22890aaa5b 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:18:11.907051 (+1642063us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b5400526cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:11.907286 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336332672}
W20260709 12:18:11.916322 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63189.7us
I20260709 12:18:11.917523  2267 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:11.918469 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 549) took 1004 ms. Trace:
I20260709 12:18:11.918740  2590 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:11.918166  2267 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:11.927260  2320 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:11.928773  2509 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.930485 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1678) took 2466 ms. Trace:
W20260709 12:18:11.933235 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.934895  2547 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:11.938733 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 202607us
I20260709 12:18:11.918887 32657 rpcz_store.cc:276] 0709 12:18:10.909138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.921744 (+ 12606us) service_pool.cc:224] Handling call
0709 12:18:10.922027 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.321268 (+399241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.911083 (+589815us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400543654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:11.913782 (+  2699us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617032064}
I20260709 12:18:11.944080 32047 rpcz_store.cc:276] 0709 12:18:09.447495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.913867 (+1466372us) service_pool.cc:224] Handling call
0709 12:18:10.914128 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.914306 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:11.740645 (+826339us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b1800422320. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf
0709 12:18:11.740949 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.882895 (+141946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.886342 (+  3447us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:11.887251 (+   909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.887363 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.913911 (+ 26548us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":192250,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":8653,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":44098,"spinlock_wait_cycles":78504832,"thread_start_us":365354,"threads_started":2,"wal-append.queue_time_us":109319}
I20260709 12:18:11.954404  2565 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.955147  2565 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.919343  2590 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:11.965132  2213 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.967475  2213 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.969924  2213 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:11.957204  2344 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:11.973356  2487 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:11.973647 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:11.974079 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55753.9us
I20260709 12:18:11.974828 32660 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:11.975343 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:11.976310 32657 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:11.976406 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:11.977524  2346 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:11.979449  2346 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:11.979846  2519 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:11.980394  2519 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:11.981376 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.982831 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.983194 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:11.983639 32044 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:11.984014 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:11.985083  2525 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:11.986145  2525 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:11.986544  2552 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:11.987397 32119 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:11.987636  2552 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:11.988051  2376 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.988205  2552 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:11.988816  2376 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:11.989207 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:11.989858 32113 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:11.989447  2552 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:11.962721  2590 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:11.978076 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:11.987327  2525 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:11.979895  2346 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:11.995694 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:11.983659 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:11.982211 32039 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
W20260709 12:18:11.998328 32040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.025s	sys 0.075s
I20260709 12:18:11.981868  2519 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.000970  2519 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.001154  2565 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.007030 32651 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:12.007264 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.008841  2614 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.009395  2614 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.012821 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:12.013707 32660 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 12:18:12.014411  2628 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:18:12.014712 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:12.015005  2540 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:12.015443 32657 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:12.015705  2540 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:12.016242 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.016937  2614 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:12.018394  2608 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:18:12.018981  2392 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.019562  2614 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:12.020058  2614 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:12.021565  2614 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.022950  2522 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.023358  2614 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.022950 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1930) took 2637 ms. Trace:
I20260709 12:18:11.990818  2596 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:18:12.024323 32115 rpcz_store.cc:276] 0709 12:18:09.385878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.848618 (+1462740us) service_pool.cc:224] Handling call
0709 12:18:10.849010 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.849285 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:10.873038 (+ 23753us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.871931 (+998893us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b18003ba940. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:11.963925 (+ 91994us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.972690 (+  8765us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:12.006978 (+ 34288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.022550 (+ 15572us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.022731 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20784,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":36288,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11352,"spinlock_wait_cycles":7541632,"thread_start_us":17385,"threads_started":1,"wal-append.queue_time_us":1133937}
I20260709 12:18:12.026357  2596 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:12.026760  2596 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:12.027835  2596 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.029250  2596 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:12.032847 32653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 614) took 1733 ms (client timeout 1975 ms). Trace:
W20260709 12:18:12.033296 32653 rpcz_store.cc:269] 0709 12:18:10.299509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.318602 (+ 19093us) service_pool.cc:224] Handling call
0709 12:18:12.032490 (+1713888us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54004c13d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:12.032659 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901310848}
I20260709 12:18:12.003361 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.034880 32120 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:12.038111 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.038836 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 615) took 1666 ms. Trace:
I20260709 12:18:12.038977 32115 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:12.039376  2449 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:12.040207 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.040644 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.041594 32112 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:12.041427 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.043337  2470 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.044550  2156 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:12.045594  2470 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.046563 32115 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 12:18:12.046763 32652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 520) took 1598 ms (client timeout 1895 ms). Trace:
I20260709 12:18:12.047148 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:12.047252 32652 rpcz_store.cc:269] 0709 12:18:10.447835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.493973 (+ 46138us) service_pool.cc:224] Handling call
0709 12:18:12.046334 (+1552361us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:12.046539 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345300224}
I20260709 12:18:12.048280 32085 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:12.049959 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:12.050704 32113 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:12.050879 32660 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
I20260709 12:18:12.035540 32047 raft_consensus.cc:2376] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:18:11.997752  2346 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.048668  2326 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:12.054409 32038 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.054659 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:12.054834  2423 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:12.001163  2552 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:12.055310 32653 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:18:12.056299  2447 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.057724  2344 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.995986  2525 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.039397 32654 rpcz_store.cc:276] 0709 12:18:10.372427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.373334 (+   907us) service_pool.cc:224] Handling call
0709 12:18:10.373683 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.079139 (+705456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.038270 (+959131us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400544054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:12.038595 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576445824}
I20260709 12:18:12.057301  2616 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:12.060139  2326 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:12.080826 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.058732  2267 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.071360  2447 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.084612 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.090468  2502 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:12.091580  2502 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:12.091940  2502 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:12.094553  2326 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.093168  2502 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.096099  2326 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.096662  2362 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:12.100459 32114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1920) took 2775 ms (client timeout 1835 ms). Trace:
W20260709 12:18:12.100903 32114 rpcz_store.cc:269] 0709 12:18:09.325028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.015205 (+690177us) service_pool.cc:224] Handling call
0709 12:18:12.100090 (+2084885us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b540012bbd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:12.100245 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190382848}
I20260709 12:18:12.103536  2376 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.083987  2616 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.096927  2502 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.104422 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.107884 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:12.114022 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.116941 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.078605  2525 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.087960  2423 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:12.119764 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 141792us
I20260709 12:18:12.126796  2369 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:12.128094 32038 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:12.132431  2267 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:12.114615  2415 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.138833  2415 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.140087 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.140262  2522 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:12.140775  2522 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.136453 31966 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:12.136549  2540 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:12.136847  2616 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.137939  2630 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.120407  2509 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:12.163004 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45549.6us
I20260709 12:18:12.159453 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:12.172597 32652 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:12.173578 32114 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:12.187784 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 98934.2us
W20260709 12:18:12.194391  2595 consensus_queue.cc:1175] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "38c927f0ee104b57972899f5b27aa99b" responder_term: 7 status { last_received { term: 4 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:12.196978  2447 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:12.198400 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:12.198374  2636 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:18:12.214324 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1782) took 2995 ms. Trace:
W20260709 12:18:12.202821  2638 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:18:12.219111  2549 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:12.220139  2549 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:12.221177  2549 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:12.167537  2630 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.221767  2549 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.207471  2342 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:12.222788 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.223071  2598 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.214689 32117 rpcz_store.cc:276] 0709 12:18:09.218434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.288542 (+ 70108us) service_pool.cc:224] Handling call
0709 12:18:09.320110 (+ 31568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.642830 (+322720us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:09.851633 (+208803us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.794727 (+943094us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.872662 (+1077935us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:18:11.874215 (+  1553us) log.cc:844] Serialized 26 byte log entry
0709 12:18:11.966106 (+ 91891us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:11.972765 (+  6659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.997663 (+ 24898us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:12.213889 (+216226us) raft_consensus.cc:1685] finished
0709 12:18:12.214006 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.214127 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148876,"prepare.queue_time_us":552185,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":159497,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":18190,"spinlock_wait_cycles":15108352,"thread_start_us":1383251,"threads_started":4,"wal-append.queue_time_us":714014,"wal-append.run_cpu_time_us":1211,"wal-append.run_wall_time_us":1023427}
I20260709 12:18:12.223596  2598 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.226447 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:12.227420 31967 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 12:18:12.227840  2495 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.780s	user 0.006s	sys 0.013s
I20260709 12:18:12.228327  2495 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:12.228660 31965 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:18:12.229144 32118 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:12.228920 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.229928 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.231624 31972 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:18:12.232394  2598 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:12.227394  2526 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.007s
W20260709 12:18:12.210340 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.222812  2342 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:12.233551  2342 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:12.215427  2423 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:12.221231  2527 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:12.208701  2585 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:12.234467 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:12.235971  2342 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.237275  2342 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.239001  2267 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:12.239847  2630 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.240293  2267 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:12.240533  2495 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:12.240969  2495 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.234519  2527 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.242359  2527 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.242633 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:12.243677  2527 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.242434  2495 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.239780 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:12.205045  2369 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:12.250062 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:12.255136 32114 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:12.258543 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:12.235091  2526 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.264613 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.265647 31972 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:12.250711  2344 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:12.278409 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.278332  2495 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.279635 31968 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:12.286008 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1675) took 2839 ms. Trace:
I20260709 12:18:12.256132  2590 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.282544  2326 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.292207 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:12.285816  2540 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:12.296082 31963 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.300724 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.287817 32040 rpcz_store.cc:276] 0709 12:18:09.446710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.911499 (+1464789us) service_pool.cc:224] Handling call
0709 12:18:10.911800 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.980672 (+ 68872us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:11.334525 (+353853us) log.cc:844] Serialized 14 byte log entry
0709 12:18:12.011099 (+676574us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.011205 (+   106us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:12.011275 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.284395 (+273120us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b540032ad54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:12.284436 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.285951 (+  1515us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12071,"prepare.queue_time_us":210844,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":40,"raft.queue_time_us":12634,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":157888896,"thread_start_us":1017476,"threads_started":2,"wal-append.queue_time_us":575959}
W20260709 12:18:12.301792 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62132.4us
I20260709 12:18:12.302981 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:12.307780 31964 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:12.308156 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.308832 32046 raft_consensus.cc:2376] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8. Re-sending same reply.
I20260709 12:18:12.321650 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:12.322328 32040 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:12.325057 32038 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:12.327828 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:12.328471 32043 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
W20260709 12:18:12.329638 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1948) took 1236 ms (client timeout 1521 ms). Trace:
W20260709 12:18:12.330047 32121 rpcz_store.cc:269] 0709 12:18:11.093259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.123659 (+ 30400us) service_pool.cc:224] Handling call
0709 12:18:12.329331 (+1205672us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54003e4ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:12.329470 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674845568}
I20260709 12:18:12.331887  2447 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.300798  2540 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.337354 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:12.349700  2526 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:12.353091 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.359056 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:12.371896 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:12.373894 32043 pending_rounds.cc:85] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 6
I20260709 12:18:12.343300  2522 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.391299 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.386126 32043 pending_rounds.cc:107] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:18:12.392565 32043 raft_consensus.cc:2889] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:12.399144 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.393162  2526 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.402942 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:12.403339  2644 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:18:12.403726 32039 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:12.404096  2506 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:12.403017 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.403632  2522 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:12.407192 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.408020 32042 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:12.410246 32038 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
W20260709 12:18:12.410255 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:12.412642 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:12.413007 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.416121 31965 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:18:12.416505 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.341315  2549 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.418967 32114 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:12.424494 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.425454 32119 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.397776  2590 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.429162 32651 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:12.429226 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70307.9us
I20260709 12:18:12.430560 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.438529  2346 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.440522 32654 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:12.441651 31966 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:12.426478  2522 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.444954  2522 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.452574 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.454178 32042 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:18:12.463924 32114 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.469825  2506 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:12.470597  2506 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.471526  2630 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:12.451463  2415 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:18:12.472770  2506 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:12.472961  2415 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:12.473403  2415 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:12.474255  2415 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.475736  2415 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.432546  2540 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.466486  2400 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:18:12.479519  2400 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:12.479846  2400 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:12.480684  2400 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.483395  2522 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:12.473407 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:12.484723 32044 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:12.488858 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:12.490646 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.495251 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.498927  2525 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.500873  2346 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:12.482206  2400 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.502163 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.502449 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.503060 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.504602 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.505421 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:12.506423 31969 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:12.506520 32117 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:12.464952  2549 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.507573 32119 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:12.512045 31972 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:18:12.534986 31971 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:18:12.551740 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:12.554764 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 106148us
I20260709 12:18:12.557435 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.558558 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:12.558885 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63774.3us
I20260709 12:18:12.502449  2142 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.564289  2142 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:12.565372 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61619.2us
I20260709 12:18:12.566196 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:12.567920  2470 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.568426  2470 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.568487 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:12.569001  2470 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:12.569975 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.574360 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.577929 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:12.579381 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:12.579383  2362 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260709 12:18:12.589990  2645 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:18:12.591004  2400 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.591522  2400 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:12.591949  2400 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:12.592307 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.593895 32040 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:12.594480 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:12.596257 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.601926 32119 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:12.580560  2590 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:12.595141 32044 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:12.609747 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.613689 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.614275 32040 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.618783 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:12.620528  2346 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.621255  2346 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:12.621650 32040 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:12.623011 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:12.623519 32116 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.625202  2400 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.611517  2590 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:12.639688 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63577.5us
I20260709 12:18:12.645479 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.644526 32114 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:12.612468  2142 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:12.639518  2648 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:18:12.636252  2369 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:12.627791  2502 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:18:12.671275 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.647189  2590 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.666667  2362 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.667999  2376 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:12.680756 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.689621 32120 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:12.680380 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:12.670977  2369 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.680717  2142 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:12.680253  2362 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.694599 32044 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.694479  2142 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.683893 32040 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.680476  2376 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:12.692863 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:12.699126 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 130778us
I20260709 12:18:12.697234  2369 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.714977 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 319) took 1444 ms. Trace:
I20260709 12:18:12.717682 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:12.710593 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:12.704892  2595 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:12.715586 32116 rpcz_store.cc:276] 0709 12:18:11.270742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.271525 (+   783us) service_pool.cc:224] Handling call
0709 12:18:11.271866 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.623957 (+1352091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.706535 (+ 82578us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54001491d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:12.714893 (+  8358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992527744}
I20260709 12:18:12.703528  2400 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.725952 31938 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:12.704977  2362 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.711087  2495 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:12.732595 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.743985 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:12.741492  2344 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:12.746044  2499 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.746471  2499 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:12.746838  2499 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.752205 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:12.753649 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:12.739432  2415 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.754751  2415 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:12.755144  2415 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:12.758958 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58476.2us
I20260709 12:18:12.759878 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:12.743280 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:12.760983 32045 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.732820  2495 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.768400 32117 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:12.769387 32046 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:12.775424  2506 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.775928  2506 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:12.776190  2585 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:12.776374  2506 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.776796  2585 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:12.779958 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:12.780824  2526 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:12.782111  2527 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.782625  2526 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:12.782701  2527 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.737818  2595 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:12.785822 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50583.6us
I20260709 12:18:12.790040 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:12.789917  2653 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:18:12.797360 32040 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:12.800596 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.777796  2585 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.3, Last appended by leader: 3, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.784556  2495 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.791033  2614 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:12.808905 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.811815 32115 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:12.807161 32113 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:12.790226  2526 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:12.807772 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:12.812975 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:12.821708 32046 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:12.744356  2376 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:12.829682 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49831.2us
W20260709 12:18:12.842197 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52281.6us
I20260709 12:18:12.843325 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:12.814231  2614 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:12.849335 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.823983  2342 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.844784 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:12.846582 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:12.846460 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:12.874282 32113 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:12.877316 32117 pending_rounds.cc:85] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:18:12.877714 32117 pending_rounds.cc:107] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:18:12.877991 32117 raft_consensus.cc:2889] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:12.873610 32041 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:12.831672  2470 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.866427  2614 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:12.860643  2662 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:18:12.905757 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.867388  2595 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:12.918267 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.925007  2502 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:12.927096 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.926609  2502 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.930291  2502 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.931015  2502 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.882296  2342 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.902355  2344 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:12.906102  2376 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:12.942684 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.904940  2400 raft_consensus.cc:2764] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:12.905628  2546 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:12.942966 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.882623  2527 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.949450 32044 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:12.919848  2660 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.931730  2502 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:12.965044 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.971072  2677 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.971686  2677 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.972239  2502 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:12.972770  2502 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:12.973263  2502 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.939555  2549 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:18:12.978546 31971 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:12.983704 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.979029 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:12.991892 32045 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:12.987377 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 140887us
I20260709 12:18:12.995220 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:12.975265  2502 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:12.974407  2677 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:13.014235 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:12.978463  2660 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:12.974305  2415 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:13.101801  2669 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:18:12.974604  2546 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.013327  2495 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.104084  2495 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:13.105681 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.106817 31971 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:13.108464  2369 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.109496  2660 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:13.114974 32120 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:12.963934  2526 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.128700 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:13.129601  2415 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.006525  2590 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.088703  2552 raft_consensus.cc:2764] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:18:13.149539 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.975276  2344 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.169431  2470 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:13.172515  2525 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.001s
W20260709 12:18:13.175702  2657 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.472s	user 0.002s	sys 0.002s
I20260709 12:18:13.119633  2502 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.173547  2614 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:13.178172  2665 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 12:18:13.225744 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 164131us
I20260709 12:18:13.247043  2369 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.249068 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.250094 31972 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:13.225807 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.248626  2342 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.254087 32043 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:13.254398  2142 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:13.208285  2661 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.231369 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.259689 32045 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:13.232087 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.261111 32044 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11.
I20260709 12:18:13.233194 32042 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:13.234220 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:13.236558 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.877s (SENT) [suppressed 34 similar messages]
I20260709 12:18:13.237187 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:13.225615 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 230452us
I20260709 12:18:13.230154  2376 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:13.263939  2376 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:13.264426  2376 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.266646 32047 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:13.267426  2447 raft_consensus.cc:2764] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:18:13.271544  2590 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.274534  2614 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:18:13.274962 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:13.276645  2423 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:13.277045  2423 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:13.278755 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 432416us
W20260709 12:18:13.280180  2499 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.002s
I20260709 12:18:13.280591  2499 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.263209 32046 raft_consensus.cc:2410] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:13.286918 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:13.289719 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.290170 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.305719 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.305765 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.307049 31972 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:13.313582 32114 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.324262 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:13.325536 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:13.325824 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:13.326145 32112 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.326596 32113 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:13.332791 32119 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:13.335608 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:13.306119  2415 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.345005 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.335772 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:13.310182  2678 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.356580 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:13.355348 32115 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:13.357224 32118 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:13.329526  2595 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.343354 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:13.278007  2423 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.285071  2502 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:18:13.362216 32116 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:13.389122 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.344669  2346 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.379141  2342 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:13.403627 31971 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:13.365640  2678 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:13.400812 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:13.417817 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.421510 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 471699us
W20260709 12:18:13.418308 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 137367us
I20260709 12:18:13.418510 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.427309 31970 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:13.427841 31966 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:13.398875  2681 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:13.429273 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:13.427168 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:13.433102 32652 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:13.433718 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:13.412469  2499 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:13.431584 31964 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:13.432246 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.421480  2423 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:13.395171  2595 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.404274  2590 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:18:13.423504  2596 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:13.436172 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:13.437477 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:13.453109 31969 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:18:13.491122 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.506356 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68956.1us
W20260709 12:18:13.436833 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.472903 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:13.500366 32654 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.507709 32047 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.443112 32038 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:13.464579 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:13.473871 32656 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:13.488557 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:13.509306 32660 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:13.472769 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:13.426448  2447 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.473753 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.456496 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.488575 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:13.519830 32042 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:13.485153  2691 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:18:13.487047  2687 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:18:13.473946  2684 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 12:18:13.486245  2683 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:13.529552 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:13.485255  2682 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:18:13.453442  2506 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.677s	user 0.007s	sys 0.007s
I20260709 12:18:13.432391  2369 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.534180 32623 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11.
I20260709 12:18:13.527740 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.535715 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:13.454517  2630 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.512120  2415 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:13.537217 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.538045 32658 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:13.556691 32044 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:13.556849 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:13.540346 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.540977 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:13.542054  2692 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:18:13.542537  2689 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.542738 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:13.574064 32043 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:13.542991  2698 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.543558 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:13.542743 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.547237  2693 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.526880  2670 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.548027 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.596124 32045 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:13.548606  2697 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.552585 32112 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:13.584057  2689 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.435935  2690 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:13.622644 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.453644  2342 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:18:13.473074  2674 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.202s
I20260709 12:18:13.645985  2674 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:13.646366  2674 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:13.647172  2674 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.600572  2698 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.563335  2630 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.649029  2630 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.473222  2681 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.651757  2681 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.577118  2595 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.436259  2346 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.578363  2369 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:13.653100 31968 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:13.653339  2369 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.653992  2346 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.586341 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:13.585227 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.655828 32044 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:13.553521 32118 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:13.590608 32042 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:13.555043 32116 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:13.600064 32040 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:13.628337 31963 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:13.655216 31967 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:13.636150 31966 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:13.627573 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:13.661770 31965 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:13.601117 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:13.556013  2506 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.584448  2678 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.628700  2376 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:13.644441  2344 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:13.644645 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.645319  2342 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.648391  2674 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:13.554134  2447 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.681861 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:13.682354 32043 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:18:13.683176  2666 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.737s	user 0.001s	sys 0.007s
I20260709 12:18:13.608085  2670 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.685408 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:13.623261 31969 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:13.664660 32047 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:18:13.629649  2694 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:18:13.627404  2684 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:18:13.670396 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:13.689836 32042 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:13.673112  2654 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.014s	sys 0.007s
I20260709 12:18:13.620090  2661 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:13.673764 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 238691us
W20260709 12:18:13.626483  2687 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:18:13.638530  2689 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:13.676185 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.595561  2693 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:13.680524  2546 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:13.680773  2506 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:13.688964 31964 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:13.619763  2502 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:13.690769 32040 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:13.691644 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:13.623232  2415 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.607897  2697 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:13.693179 32043 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:13.743436 32044 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:14.167352 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:13.921663 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.166731 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:13.741232 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 630) took 1008 ms. Trace:
I20260709 12:18:14.183522 31968 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:13.772747 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.183552 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:13.982141 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.185392 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.183689 31967 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:13.880723 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.187418 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.188457 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:14.189131 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.189695 32118 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:14.189249 32119 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.198025 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:14.202895 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.877044 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1812) took 1078 ms. Trace:
I20260709 12:18:14.197943 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:14.207887 32119 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.209942 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.202529 32651 rpcz_store.cc:276] 0709 12:18:12.732371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.732543 (+   172us) service_pool.cc:224] Handling call
0709 12:18:12.732823 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.783724 (+ 50901us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:12.891415 (+107691us) log.cc:844] Serialized 14 byte log entry
0709 12:18:13.267202 (+375787us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b18003d8ee0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22900a358f 00007f22890e2571 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd
0709 12:18:13.417697 (+150495us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.425122 (+  7425us) log.cc:844] Serialized 26 byte log entry
0709 12:18:13.425229 (+   107us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:13.425251 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.425293 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:13.677103 (+251810us) raft_consensus.cc:1685] finished
0709 12:18:13.740982 (+ 63879us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.741159 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":355227,"prepare.queue_time_us":453669,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":506181,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":17787,"spinlock_wait_cycles":15219072,"thread_start_us":236309,"threads_started":3,"wal-append.queue_time_us":388149}
W20260709 12:18:14.212165 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.706681  2654 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.014s	sys 0.007s
I20260709 12:18:13.677670  2342 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:14.215180 32113 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:14.214720 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:13.879937  2686 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.162011  2447 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.139683  2614 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.850704  2685 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.740944  2596 consensus_queue.cc:1243] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:14.306281 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.309556 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.305608 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.315316 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.325562 31969 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.325592 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:14.326596 31965 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:14.216830 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:14.176761  2661 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.879637  2688 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.204052 32121 rpcz_store.cc:276] 0709 12:18:12.798556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.798674 (+   118us) service_pool.cc:224] Handling call
0709 12:18:12.808471 (+  9797us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:12.824761 (+ 16290us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:12.943552 (+118791us) log.cc:844] Serialized 14 byte log entry
0709 12:18:13.180766 (+237214us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:13.227323 (+ 46557us) log.cc:844] Serialized 52 byte log entry
0709 12:18:13.227576 (+   253us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:13.227843 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 12:18:13.876189 (+648346us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003c21a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:13.876416 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.876515 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:13.876664 (+   149us) raft_consensus.cc:1685] finished
0709 12:18:13.876685 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.876850 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1372,"prepare.queue_time_us":20958,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":81,"raft.queue_time_us":683284,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":9926,"spinlock_wait_cycles":26528512,"thread_start_us":311544,"threads_started":2,"wal-append.queue_time_us":333689}
I20260709 12:18:13.839900  2376 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.334738 31963 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:14.183851  2693 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.355930  2376 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.356760 32653 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.356514  2376 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.357127 32653 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:13.883127  2527 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:14.358692 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.359556 31971 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:14.217010  2590 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.362653 31968 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:14.359576 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.363698 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.362044 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:14.365013 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.365532 31967 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:14.363709 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.366920 32113 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:13.883361  2499 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.324018 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.357980 32653 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.321257  2506 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.208168  2362 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.686676  2344 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.211977  2142 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:14.369704 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.381099 32118 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:14.370287  2499 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.371701 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.346668  2670 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:14.376127 32119 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:14.327306 32117 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:14.381618 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.383441 32113 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 3. Preceding OpId from leader: term: 11 index: 4. (index mismatch)
I20260709 12:18:14.383942  2142 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.385648 31968 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:14.214190  2526 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.154s
I20260709 12:18:14.382737 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.387804 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:14.214430  2662 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.155s
I20260709 12:18:14.329231  2686 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.330458  2502 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:14.369613  2590 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.367391  2527 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.391776 32653 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.359109  2688 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.394985  2546 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:14.396442  2376 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:14.397583 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.374717  2342 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.438920  2585 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:14.400835  2630 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.751s	user 0.005s	sys 0.005s
I20260709 12:18:14.404626  2499 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:14.405375  2698 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:14.407765  2423 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.893s	user 0.012s	sys 0.000s
I20260709 12:18:14.323190  2614 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.407987  2344 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.408731 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.485513 31971 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:18:14.396462 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.426419 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.474926 32623 connection.cc:256] RPC call timeout handler was delayed by -1.1706218919998719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:18:14.486191 32623 connection.cc:256] RPC call timeout handler was delayed by -1.0248589890002222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 12:18:14.431622 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:14.486810 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99100.7us
I20260709 12:18:14.435731 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.328830  2685 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:14.378404 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.389434 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:14.386134  2362 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.385958  2506 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.478227 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.477968 32121 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:14.429316 32116 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:14.324576  2447 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:18:14.472187  2526 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:14.492934 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 103571us
I20260709 12:18:14.496363 32112 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:18:14.474619  2502 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:14.498277 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:14.501724 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 172457us
I20260709 12:18:14.505653 32660 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:14.511941 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.512477 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.487886  2342 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.496763  2362 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.518957 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:14.521222 32651 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:14.523764  2614 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.524334  2614 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.524792  2614 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.527077 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1704) took 1084 ms. Trace:
I20260709 12:18:14.528780 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.483355  2630 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.498356  2526 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:14.536270 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:14.530795 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:14.597146 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 66433.3us
W20260709 12:18:14.501475 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.545487 31968 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:14.540047  2693 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:14.514223  2376 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:14.527603 32046 rpcz_store.cc:276] 0709 12:18:13.442635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.442832 (+   197us) service_pool.cc:224] Handling call
0709 12:18:13.443147 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.477253 (+ 34106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.477298 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.532205 (+ 54907us) log.cc:844] Serialized 26 byte log entry
0709 12:18:13.689154 (+156949us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:13.806934 (+117780us) log.cc:844] Serialized 14 byte log entry
0709 12:18:13.807435 (+   501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.807532 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.526797 (+719265us) raft_consensus.cc:1685] finished
0709 12:18:14.526834 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.527005 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117296,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":19461,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"spinlock_wait_cycles":17132288,"thread_start_us":151879,"threads_started":1,"wal-append.queue_time_us":877935}
I20260709 12:18:14.529006  2697 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.737557 32047 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:14.497061  2585 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.560007 31970 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:14.560459  2664 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.617s	user 0.011s	sys 0.011s
I20260709 12:18:14.560636 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:14.562455  2674 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.873s	user 0.011s	sys 0.000s
I20260709 12:18:14.562961  2502 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.564652  2688 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:14.712038  2369 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:14.567160  2685 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:14.568571 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:14.554204 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43746.9us
I20260709 12:18:14.575554  2527 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:14.579038  2686 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:14.580421 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.761687 32117 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:14.581081  2630 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.583240  2690 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.583941 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.585990 32013 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:14.810935 32013 leader_election.cc:403] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:14.520704  2546 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:18:14.586796  2142 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:14.586969 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1719) took 1011 ms. Trace:
W20260709 12:18:14.589500  2701 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.752s	user 0.004s	sys 0.012s
I20260709 12:18:14.541153  2526 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.591692  2506 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:14.593174  2689 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:14.595271 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:14.529403 31965 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 5.
W20260709 12:18:14.632961 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.639230 31969 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:14.638991 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:14.657094 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 275854us
I20260709 12:18:14.599274  2662 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.707578 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:14.710281 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1652) took 1086 ms. Trace:
I20260709 12:18:14.709975 32653 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:14.715507 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:14.715857 32044 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 12:18:14.719871 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 505192us
W20260709 12:18:14.733928 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.736865 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:14.740480 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:14.520412  2344 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:18:14.755918 31966 raft_consensus.cc:2410] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:14.755561 32656 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.742875  2376 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:14.814503 32043 rpcz_store.cc:276] 0709 12:18:13.575293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.683184 (+107891us) service_pool.cc:224] Handling call
0709 12:18:13.692768 (+  9584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.167037 (+474269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.586711 (+419674us) spinlock_profiling.cc:243] Waited 9.2 ms on lock 0x7b54000334d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:14.586879 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21154176}
I20260709 12:18:14.753587  2540 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.759845  2369 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.759685  2688 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:14.820308 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 259720us
I20260709 12:18:15.057096 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 563) took 1624 ms. Trace:
W20260709 12:18:14.890017 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.866344 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.062722  2674 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:15.063302  2674 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:14.866858 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 346375us
I20260709 12:18:15.064527  2674 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:15.065923 32010 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.536s (SENT)
I20260709 12:18:15.066565  2674 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.067160 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1653) took 1442 ms. Trace:
I20260709 12:18:15.079403 31965 rpcz_store.cc:276] 0709 12:18:13.624304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.501125 (+876821us) service_pool.cc:224] Handling call
0709 12:18:15.067099 (+565974us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.068449 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:14.975142 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:15.069077 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1905) took 1225 ms (client timeout 1504 ms). Trace:
W20260709 12:18:15.069592 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.081892 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.081604 31969 rpcz_store.cc:269] 0709 12:18:13.843814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.183164 (+339350us) service_pool.cc:224] Handling call
0709 12:18:15.069021 (+885857us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.070670  2709 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:15.081993 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.082834 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.083307 31964 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:15.083895 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.084378 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.084972 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.085666 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1913) took 1239 ms. Trace:
I20260709 12:18:15.057308 31968 rpcz_store.cc:276] 0709 12:18:13.624094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.495739 (+871645us) service_pool.cc:224] Handling call
0709 12:18:14.710207 (+214468us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.086048 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 324891us
I20260709 12:18:15.071076  2685 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:15.079023  2540 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:15.070144 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 484290us
I20260709 12:18:15.070370  2614 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:14.899811 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1723) took 2278 ms. Trace:
I20260709 12:18:15.066461  2662 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.057884 32653 rpcz_store.cc:276] 0709 12:18:13.432794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.442419 (+  9625us) service_pool.cc:224] Handling call
0709 12:18:13.453388 (+ 10969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.356669 (+903281us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54003dcd20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d11c0 00007f228c1d2231 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c
0709 12:18:15.056706 (+700037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.057011 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003156864}
I20260709 12:18:15.085971 31969 rpcz_store.cc:276] 0709 12:18:14.001553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.084750 (+1083197us) service_pool.cc:224] Handling call
0709 12:18:15.085614 (+   864us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.065742 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.098207 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:15.059651 32657 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 3.
I20260709 12:18:15.101833 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:15.109604 31971 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:15.113528 31964 raft_consensus.cc:2368] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:15.097440  2678 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:15.104480 32039 rpcz_store.cc:276] 0709 12:18:12.542289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.542535 (+   246us) service_pool.cc:224] Handling call
0709 12:18:12.560216 (+ 17681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.651199 (+ 90983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.651245 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:12.659902 (+  8657us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.733205 (+ 73303us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:12.733231 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.733277 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.528848 (+1795571us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:14.815533 (+286685us) raft_consensus.cc:1685] finished
0709 12:18:14.815563 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.820678 (+  5115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6494,"raft.queue_time_us":62101,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":7428,"spinlock_wait_cycles":3745024,"thread_start_us":112462,"threads_started":2,"wal-append.queue_time_us":152865}
I20260709 12:18:15.114843  2678 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.117089  2678 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.117671  2678 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.117919 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:15.118764 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.119513 31969 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:15.120321 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.120828 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:14.874541  2142 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.121184 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.121450 32117 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:15.121474 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.121932 31971 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:15.117149  2662 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.090723  2369 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.122009 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1636) took 1807 ms. Trace:
I20260709 12:18:15.122790 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.134245 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1910) took 1288 ms. Trace:
I20260709 12:18:15.134797 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:15.102058  2696 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:18:15.090890  2590 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.134702 31971 rpcz_store.cc:276] 0709 12:18:13.845363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.122556 (+1277193us) service_pool.cc:224] Handling call
0709 12:18:15.134183 (+ 11627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.138921 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.137218 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:15.161230 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 80148.8us
I20260709 12:18:15.165395 32116 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:15.167555 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.168880 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:15.169822 32660 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.024s	sys 0.038s
W20260709 12:18:15.170157 32660 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.025s	sys 0.038s
I20260709 12:18:15.170215 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.136780 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.174656 32112 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:15.141016 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1640) took 1683 ms (client timeout 2243 ms). Trace:
I20260709 12:18:15.176162 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.176748 32117 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:15.177299 31968 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:15.178637 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.179214 31969 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:14.814141  2690 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.179981 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.185667 31969 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:18:15.175727 31970 rpcz_store.cc:269] 0709 12:18:13.445303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.445597 (+   294us) service_pool.cc:224] Handling call
0709 12:18:15.128102 (+1682505us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000799d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:15.128257 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2399381248}
W20260709 12:18:15.121181  2362 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.568s	user 0.006s	sys 0.001s
I20260709 12:18:15.132668 31972 rpcz_store.cc:276] 0709 12:18:13.314204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.314362 (+   158us) service_pool.cc:224] Handling call
0709 12:18:13.314722 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.359596 (+ 44874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.359627 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.413272 (+ 53645us) log.cc:844] Serialized 26 byte log entry
0709 12:18:13.655384 (+242112us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:13.655423 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.655494 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.588848 (+933354us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:15.121674 (+532826us) raft_consensus.cc:1685] finished
0709 12:18:15.121694 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.121936 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2241,"raft.queue_time_us":8536,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":15870,"spinlock_wait_cycles":33667840,"thread_start_us":211609,"threads_started":1,"wal-append.queue_time_us":257301}
I20260709 12:18:15.165966  2662 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:15.195087  2662 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:15.185731  2614 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:15.196044  2662 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:15.197862  2342 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.000s
I20260709 12:18:15.133069  2344 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:15.202570 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.210978 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:15.211899 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.214557 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.215271 31969 raft_consensus.cc:2368] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:15.118126  2678 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.136924  2686 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:18:15.229754 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.232904 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.226647  2686 consensus_queue.cc:1243] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:15.233606 32041 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.104084  2506 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:18:15.139286  2716 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:15.102272  2688 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.241779 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:15.242451 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.242660 31967 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 12:18:15.104235  2685 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.242991 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.248958 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 637) took 1045 ms. Trace:
I20260709 12:18:15.258173 31964 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:18:15.253134 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 114306us
I20260709 12:18:15.259423 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.121483  2142 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.257735 32656 rpcz_store.cc:276] 0709 12:18:14.203081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.203245 (+   164us) service_pool.cc:224] Handling call
0709 12:18:14.203525 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.078469 (+874944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.248661 (+170192us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b5400544054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:15.248885 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269082368}
I20260709 12:18:15.254442 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:15.267728 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1897) took 1426 ms. Trace:
I20260709 12:18:15.267179  2716 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.270890  2732 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.271030  2344 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:15.271484  2710 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:18:15.271603 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60736us
I20260709 12:18:15.275812 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:15.288055 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.274559 31966 rpcz_store.cc:276] 0709 12:18:13.841273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.359326 (+518053us) service_pool.cc:224] Handling call
0709 12:18:15.246464 (+887138us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54001e82d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:15.267652 (+ 21188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901825792}
I20260709 12:18:15.272089 31965 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:18:15.284546 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.291115 31964 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:15.292608 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 134306us
I20260709 12:18:15.293666 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.295436 31969 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:15.296347 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1893) took 1620 ms. Trace:
I20260709 12:18:15.296811 31965 rpcz_store.cc:276] 0709 12:18:13.675651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.271448 (+1595797us) service_pool.cc:224] Handling call
0709 12:18:15.271826 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:15.296011 (+ 24185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.296276 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.299410 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.302502 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1896) took 1461 ms. Trace:
I20260709 12:18:15.303006 31969 rpcz_store.cc:276] 0709 12:18:13.840916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.294725 (+1453809us) service_pool.cc:224] Handling call
0709 12:18:15.295131 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:15.302173 (+  7042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.302435 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.207665  2726 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:15.306594 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.256052  2712 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.315403 31966 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:15.304224 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.205657  2362 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.320225 31964 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:15.324098 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1899) took 1482 ms. Trace:
I20260709 12:18:15.290239  2546 raft_consensus.cc:2764] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:15.281311  2716 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.320567 31970 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:15.323138 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.324488 31964 rpcz_store.cc:276] 0709 12:18:13.841923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.309424 (+1467501us) service_pool.cc:224] Handling call
0709 12:18:15.309755 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.323864 (+ 14109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.324038 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.333614 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.336354 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.336578 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:15.293502  2693 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:15.344003 31966 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:15.344720 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1900) took 1502 ms. Trace:
I20260709 12:18:15.345039 31966 rpcz_store.cc:276] 0709 12:18:13.842249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.322251 (+1480002us) service_pool.cc:224] Handling call
0709 12:18:15.322640 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.344468 (+ 21828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.344667 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.349406 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.352855 31972 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:15.357959 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.358827 31966 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:15.374922 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.375788 31971 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:15.283790  2713 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:18:15.341815  2506 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.381418 32657 pending_rounds.cc:85] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 6
I20260709 12:18:15.382956 32657 pending_rounds.cc:107] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:18:15.383303 32657 raft_consensus.cc:2889] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.381194 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.275477  2376 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.380656  2686 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.390552 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.391252  2678 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.393175  2423 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:15.393555  2423 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:15.395563  2344 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:15.398918  2722 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 12:18:15.399854  2724 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:18:15.401021  2709 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
W20260709 12:18:15.402856 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1894) took 1592 ms (client timeout 2014 ms). Trace:
W20260709 12:18:15.403218 31967 rpcz_store.cc:269] 0709 12:18:13.810039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.371415 (+561376us) service_pool.cc:224] Handling call
0709 12:18:15.402667 (+1031252us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54000f7654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:15.402799 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970907392}
I20260709 12:18:15.404201 32038 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:15.404441  2527 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:15.404825  2527 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:15.320163  2726 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.407709 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1825) took 2057 ms. Trace:
I20260709 12:18:15.418399 32114 rpcz_store.cc:276] 0709 12:18:13.350413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.358896 (+  8483us) service_pool.cc:224] Handling call
0709 12:18:13.386418 (+ 27522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.882804 (+496386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.882847 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:14.216484 (+333637us) log.cc:844] Serialized 26 byte log entry
0709 12:18:14.495396 (+278912us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:14.495450 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.495548 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.407498 (+911950us) raft_consensus.cc:1685] finished
0709 12:18:15.407527 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.407657 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":477423,"raft.queue_time_us":479145,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":145717,"threads_started":1,"wal-append.queue_time_us":780048}
I20260709 12:18:15.379695  2546 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.423208 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.321875  2712 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.430188 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.431627 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.433211 32117 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:15.433332 31967 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:15.434633 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:15.436480 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 126339us
I20260709 12:18:15.439874 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.440871 32653 raft_consensus.cc:2410] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:15.405702  2527 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.445147 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.447293 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:15.428555  2661 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.059s	user 0.007s	sys 0.004s
I20260709 12:18:15.348484  2730 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
W20260709 12:18:15.457321 32010 leader_election.cc:336] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.748s (SENT)
I20260709 12:18:15.457966 32041 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
W20260709 12:18:15.459481 32010 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.752s (SENT)
I20260709 12:18:15.460302 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.431458  2447 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.389125  2506 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.437413  2709 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:15.426067  2739 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.472127 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.002s	sys 0.002s
I20260709 12:18:15.407073  2729 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.424896  2737 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.472324 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:15.472844 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.002s	sys 0.003s
I20260709 12:18:15.471302 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:15.476590 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.449649  2690 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:15.497638 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 176784us
W20260709 12:18:15.500346 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.432482  2546 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.503199 32117 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.505494 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.478046  2678 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:15.475739  2549 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:15.472226  2709 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.445933  2715 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:18:15.539000 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:15.467208  2693 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 12:18:15.543107 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83735.9us
W20260709 12:18:15.543747 32010 leader_election.cc:336] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.802s (SENT)
I20260709 12:18:15.542351 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:15.537201 32042 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.009s	sys 0.000s
W20260709 12:18:15.546741 32042 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.000s
I20260709 12:18:15.507058  2527 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.474026  2732 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:15.473724  2742 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.461350  2661 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.531250 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.432792  2423 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.548815 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.541726 31968 raft_consensus.cc:2955] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:15.558935 31971 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:15.559099 31967 raft_consensus.cc:2368] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:15.549835 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.550305 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.551956 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:15.538992  2733 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:18:15.531857  2688 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:15.549067  2736 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
I20260709 12:18:15.538926  2590 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.531759  2746 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:15.576309 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71267.4us
I20260709 12:18:15.540251  2678 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.530750  2743 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.603928  2662 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:15.579715 32010 leader_election.cc:336] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.804s (SENT)
I20260709 12:18:15.556525  2686 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:15.605038 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:15.606112 32010 leader_election.cc:336] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.806s (SENT)
I20260709 12:18:15.606523 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.606711  2546 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.606881  2362 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.607545  2362 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.607864  2718 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.607787  2369 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.553536  2142 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:15.560177  2732 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.610031  2369 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.586086 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.586395  2590 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:15.590691  2738 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:18:15.590596  2747 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:15.591077  2678 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.592191 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:15.593523  2661 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.594014  2688 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:15.639110  2688 consensus_queue.cc:1243] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:18:15.639443  2734 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.234s	user 0.003s	sys 0.000s
I20260709 12:18:15.639639  2688 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:15.641216  2688 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.641580  2688 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:15.643105  2716 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.644328 32656 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:18:15.647547 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1907) took 1803 ms (client timeout 2105 ms). Trace:
W20260709 12:18:15.647958 31963 rpcz_store.cc:269] 0709 12:18:13.844423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.379289 (+534866us) service_pool.cc:224] Handling call
0709 12:18:15.647352 (+1268063us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54001e8cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:15.647488 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1683110656}
I20260709 12:18:15.582556  2376 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.649169 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.649117  2376 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.651494  2376 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.651880  2342 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:15.652380  2342 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:15.653561  2423 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.653220  2342 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.654984 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.655308 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.656294 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 1972) took 1078 ms. Trace:
I20260709 12:18:15.656747 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:15.656651 32116 rpcz_store.cc:276] 0709 12:18:14.577961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.638591 (+ 60630us) service_pool.cc:224] Handling call
0709 12:18:15.656131 (+1017540us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b54003e49d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:15.656244 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223588224}
I20260709 12:18:15.657778  2688 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.659049  2688 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.660353  2549 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:15.660820  2549 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.661612  2705 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:15.663054  2705 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:15.663403  2705 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.663753  2705 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:15.666424 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.596922  2712 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.483691  2729 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.670387  2729 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:15.597837 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 79766.9us
I20260709 12:18:15.598667 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.671201 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:15.599483 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.671327 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:15.585824  2727 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:15.672209  2715 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.672618 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.600863  2506 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:15.499851  2737 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.483158  2739 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.602685  2540 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.603543  2726 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:15.541182  2709 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:15.610600 31969 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:15.572220  2742 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:15.611382  2731 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:18:15.580689  2685 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.611699  2732 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.606722  2693 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.607558 32010 leader_election.cc:336] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.835s (SENT)
I20260709 12:18:15.674254 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.692447 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.618392 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:15.618515 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.634039 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:15.625945 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.627619 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:15.680716 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.616622  2526 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.687s	user 0.002s	sys 0.006s
I20260709 12:18:15.629289  2747 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:15.747705 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.748066 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:15.748659 32119 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.630936 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.632162 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.634035  2590 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.638552  2527 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:15.600178  2596 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:15.632211 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.674614  2715 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:15.698037 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.698837 32112 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:15.677943 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.737615 32658 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.585788  2614 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:15.628379 31965 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:15.721992 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1898) took 1880 ms. Trace:
W20260709 12:18:15.740841 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.743263 32121 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:18:15.745255 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71058.4us
I20260709 12:18:15.738867 31971 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:15.749197 31967 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:18:15.749641 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 115660us
I20260709 12:18:15.753896 32041 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 6: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:15.696483  2745 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.755203 32042 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:18:15.823536 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.678934  2585 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.750372  2342 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.760239 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.762650 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:15.806110 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.763094 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.765928 32114 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:15.759405 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.763439 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.763787 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.842147 32045 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:15.770783 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.813645 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.760809 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.844173 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.845059 32116 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:15.757108 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.806131 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:15.818346 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:15.705505 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:15.848837 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:15.807156 31968 rpcz_store.cc:276] 0709 12:18:13.841580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.296232 (+1454652us) service_pool.cc:224] Handling call
0709 12:18:15.321236 (+ 25004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:15.342130 (+ 20894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:15.342157 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:15.437214 (+ 95057us) log.cc:844] Serialized 487 byte log entry
0709 12:18:15.515015 (+ 77801us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:15.658070 (+143055us) log.cc:844] Serialized 14 byte log entry
0709 12:18:15.721659 (+ 63589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.721740 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.721782 (+    42us) raft_consensus.cc:1685] finished
0709 12:18:15.721795 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.721932 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1289,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":45692,"threads_started":1,"wal-append.queue_time_us":127277}
I20260709 12:18:15.835568 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1965) took 1957 ms. Trace:
I20260709 12:18:15.675164  2737 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.839376 32044 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:18:15.839531 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.858058 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.858346 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1827) took 1540 ms. Trace:
I20260709 12:18:15.859755 32119 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:15.858822 32115 rpcz_store.cc:276] 0709 12:18:14.317782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.317911 (+   129us) service_pool.cc:224] Handling call
0709 12:18:14.594112 (+276201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:14.594601 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:14.594642 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:14.739451 (+144809us) log.cc:844] Serialized 26 byte log entry
0709 12:18:15.473419 (+733968us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:15.473462 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.473536 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.763903 (+290367us) raft_consensus.cc:1685] finished
0709 12:18:15.763923 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.833542 (+ 69619us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400148cd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:15.858250 (+ 24708us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2238,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":21275,"spinlock_wait_cycles":572801024,"thread_start_us":712120,"threads_started":1,"wal-append.queue_time_us":989623}
I20260709 12:18:15.848969 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.861586 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:15.825544 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.850050 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 43588.3us
I20260709 12:18:15.851097 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:15.862910 32112 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.863116 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:15.850520 32120 rpcz_store.cc:276] 0709 12:18:13.878619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.207609 (+328990us) service_pool.cc:224] Handling call
0709 12:18:14.245698 (+ 38089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.491675 (+245977us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:15.009190 (+517515us) log.cc:844] Serialized 14 byte log entry
0709 12:18:15.750330 (+741140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.750383 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:15.750434 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.835355 (+ 84921us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540013a454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:15.835398 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.835515 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6953,"prepare.queue_time_us":686997,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":33292,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":549873920,"thread_start_us":988929,"threads_started":3,"wal-append.queue_time_us":405820}
I20260709 12:18:15.866456 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.867053 32042 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:15.867465 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.867901 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.852084 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:15.868413 32042 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.868786 31967 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:15.852753 32660 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:18:15.847512 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 142099us
I20260709 12:18:15.753358  2527 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.749364  2729 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:18:15.876083 32040 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 3. Preceding OpId from leader: term: 11 index: 4. (index mismatch)
I20260709 12:18:15.751199  2447 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:15.870285 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:15.759974  2742 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:15.758780  2596 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:15.839457 32043 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:15.888020 31971 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:15.873842 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:15.764089  2711 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:15.744271  2693 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.874531 32654 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:15.856140 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 248581us
I20260709 12:18:15.891891 32044 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:18:15.891970 32010 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.860s (SENT)
I20260709 12:18:15.823626  2745 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:15.855798  2415 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.879796 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.752276  2526 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:15.893481 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 87450.6us
I20260709 12:18:15.849189  2686 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:15.875298  2527 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.898926 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.858729  2685 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:15.872052  2376 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.876155 32041 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:15.906726 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.890656  2711 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.917871 32117 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:15.910662 32042 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:15.875624  2590 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:18:15.920306 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.922158 32112 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:15.900142  2499 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:15.927611 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:15.888885  2502 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:18:15.941200 32120 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:15.942407  2596 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:15.943302  2752 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:18:15.944981 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53033.3us
W20260709 12:18:15.946364 32010 leader_election.cc:336] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.869s (SENT)
I20260709 12:18:15.946900 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.948177  2376 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:15.948285 32010 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.929s (SENT)
W20260709 12:18:15.949473 32010 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.954s (SENT)
I20260709 12:18:15.949232  2686 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.953457  2759 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:15.954457  2745 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:15.955842  2745 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.960270  2716 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:15.974189  2740 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:15.974861  2740 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:15.976047  2740 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.977792  2740 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.978830  2527 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:15.981669  2527 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:15.982079 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:15.950055 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.983196  2376 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:15.983677 32010 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.962s (SENT)
I20260709 12:18:15.984289 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.985251  2376 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:15.986773 32010 leader_election.cc:336] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.014s (SENT)
I20260709 12:18:15.987304 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.994556  2376 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:15.996110 32010 leader_election.cc:336] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.033s (SENT)
I20260709 12:18:15.996645 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:15.999490  2376 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.007282 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.010394 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.012400 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:16.013434  2376 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:16.013681  2742 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.014137  2742 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.014525  2742 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.019104 32116 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:16.020473 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.021235 32116 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:16.022372  2142 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:16.042834  2142 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:16.044088  2142 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.045496  2142 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.022799 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:16.047092 32119 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.022275 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.025115 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.051365 32117 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:16.052615 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:16.025231 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.055078 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.025513 32121 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:16.040220  2696 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:18:16.057448  2696 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:16.057922  2696 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:16.059376  2696 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.061591  2696 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:16.065194 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64312.5us
I20260709 12:18:16.070811 32116 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:15.925289  2711 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.072392 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:16.073205 32112 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.073379 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:16.082422 32116 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:16.084102 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.096645 32038 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:15.966009  2732 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.101565 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:16.103962 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:16.106863 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:15.938870  2499 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:16.120476 32658 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:16.112466 32010 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.504s (SENT)
I20260709 12:18:16.116073 32119 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:16.116938 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93165.2us
I20260709 12:18:16.097365  2685 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:16.118571 31935 connection.cc:256] RPC call timeout handler was delayed by -2.0506941599999209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 12:18:16.119285 32120 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:18:16.125619 31935 connection.cc:256] RPC call timeout handler was delayed by -2.0281941599998845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
W20260709 12:18:16.126044 31935 connection.cc:256] RPC call timeout handler was delayed by -1.945394159999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
W20260709 12:18:16.126467 31935 connection.cc:256] RPC call timeout handler was delayed by -1.8967941599998994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:18:16.126967 31935 connection.cc:256] RPC call timeout handler was delayed by -1.8598941599998398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 12:18:16.124423 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 222824us
W20260709 12:18:16.127249 31935 connection.cc:256] RPC call timeout handler was delayed by -1.7968941599999653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 12:18:16.127713 31935 connection.cc:256] RPC call timeout handler was delayed by -1.6483941599999525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 12:18:16.128022 31935 connection.cc:256] RPC call timeout handler was delayed by -1.6213941599999089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
W20260709 12:18:16.128290 31935 connection.cc:256] RPC call timeout handler was delayed by -1.4827941599999122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260709 12:18:16.128548 31935 connection.cc:256] RPC call timeout handler was delayed by -1.4539941599998656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 12:18:16.128808 31935 connection.cc:256] RPC call timeout handler was delayed by -1.4530941599998641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 12:18:16.129069 31935 connection.cc:256] RPC call timeout handler was delayed by -1.2784941599998092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2383s for a response
W20260709 12:18:16.129316 31935 connection.cc:256] RPC call timeout handler was delayed by -1.2577941599997757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24060000000000004s for a response
W20260709 12:18:16.129910 31935 connection.cc:256] RPC call timeout handler was delayed by -1.1659941599998547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2508s for a response
W20260709 12:18:16.130235 31935 connection.cc:256] RPC call timeout handler was delayed by -1.0399941599998783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26480000000000004s for a response
I20260709 12:18:16.131861 32658 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:16.129256 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.138767 32112 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:16.141146 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:16.136505 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.143247 32084 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:16.144074 32084 leader_election.cc:403] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:16.147076 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:16.149103 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.149117 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.123363  2499 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.125381  2630 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:16.147230  2716 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:18:16.160679 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 59200.8us
I20260709 12:18:16.161653 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:16.162535 32119 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:16.158056  2732 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.155294  2743 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:16.166607 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.167188  2743 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:16.168156  2705 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.003s
I20260709 12:18:16.168614  2705 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.168385  2743 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.131561  2502 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:16.171861 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.145601  2737 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.152832  2678 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.003s
W20260709 12:18:16.176643 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64206.4us
I20260709 12:18:16.155848  2686 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:16.177340 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:16.156127  2670 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:16.187990 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.187496 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:16.188934 32658 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:16.169987  2674 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:16.160408  2526 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.169523  2712 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:16.172541  2758 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.143234  2540 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 12:18:16.195284 32010 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.877s (SENT)
I20260709 12:18:16.205330 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.174266 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:16.176385  2527 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:16.197410 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.169701  2546 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:18:16.190249  2670 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.159672  2630 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:16.194783  2674 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.194201  2678 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.192058  2739 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:16.208527  2549 consensus_meta.cc:287] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.003s
W20260709 12:18:16.210574 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44065.3us
I20260709 12:18:16.210978  2689 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:16.211045  2693 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:16.224869 32117 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:16.226722 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:16.201423  2590 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:16.226722 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:16.228258 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:16.323838 32046 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.229058 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:16.231449  2761 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:18:16.234643 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:16.191182  2737 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.340890  2737 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:16.237243  2711 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.342903  2711 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:16.239663 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.345829 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.346285 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.346822 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.347723 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.348325 31967 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:16.349174 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.349756 31969 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:16.255805  2674 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.350682 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.352066 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.354239 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.355566 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:16.358855 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:16.359228 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:16.359478 31968 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.363132 31972 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:16.243826  2670 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.366609 31968 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
W20260709 12:18:16.368233 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.241159 31965 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:16.196751  2713 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:16.243970 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48713.2us
W20260709 12:18:16.376551 32010 leader_election.cc:336] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.879s (SENT)
I20260709 12:18:16.252110 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.206718  2712 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:16.196475  2730 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:18:16.245115 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.191718  2502 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.203038  2369 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 12:18:16.398883 32010 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.950s (SENT)
I20260709 12:18:16.223399  2526 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:16.399303  2678 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:16.302224 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.399415 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.401000  2712 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.402643 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.403256 32044 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:16.403491 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.404321 32039 raft_consensus.cc:2410] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:16.404229 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.311582 32038 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.404811 32044 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:16.405652 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.406913  2502 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.407167 32042 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:16.407554  2713 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:16.408234 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.408815 32040 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:16.408713  2713 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.409885 31963 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.244521  2732 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:16.239126  2527 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:16.412221 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:16.412050  2527 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:16.413691 32010 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.951s (SENT)
I20260709 12:18:16.413849  2689 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:16.317894 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.326820 32047 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:16.358136  2688 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.002s
I20260709 12:18:16.379995 32043 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:16.212380  2758 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.414150 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:16.429297  2767 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:18:16.430516  2678 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:16.430567  2758 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:16.431416 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 127400us
I20260709 12:18:16.418803 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.432337 32040 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:16.432571 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.418023 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.433655 32042 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:16.417642  2732 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.428550 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.434990 32044 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:16.416730 32119 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
W20260709 12:18:16.435248  2756 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:18:16.239073  2760 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.313s	user 0.004s	sys 0.000s
I20260709 12:18:16.420630 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:16.436653 32039 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:16.436484 32010 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.180s (SENT)
I20260709 12:18:16.435978 31966 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:16.431187  2630 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.439464  2630 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:16.417667 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.435447 32119 pending_rounds.cc:85] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
I20260709 12:18:16.442351 32119 pending_rounds.cc:107] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:16.442494  2678 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.442683 32119 raft_consensus.cc:2889] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:16.442996  2678 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:16.446343 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 194313us
I20260709 12:18:16.447188 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.447093 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:16.447475 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.307667  2549 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.447806 32042 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:16.451297 31970 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:16.447885 32047 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:16.453457 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 259975us
I20260709 12:18:16.392630 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:16.454115 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:16.238293  2709 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.455202 31963 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:16.456380 31972 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:16.452198 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.457306 32038 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:16.463342 32045 raft_consensus.cc:2368] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11.
I20260709 12:18:16.467476 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:16.468140 32652 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:16.475626 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:16.476320 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:16.477138 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:16.477878 32042 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12. Re-sending same reply.
I20260709 12:18:16.479259 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:16.480363 32042 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.483688 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:16.414268  2689 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:16.463178  2344 raft_consensus.cc:2764] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:18:16.507141 32084 connection.cc:256] RPC call timeout handler was delayed by -1.5533394159999716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 12:18:16.515038 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4516394160000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:18:16.515527 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4300394159999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
W20260709 12:18:16.515925 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4102394159999676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 12:18:16.516294 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3670394159998978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 12:18:16.516652 32084 connection.cc:256] RPC call timeout handler was delayed by -1.196039416000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
W20260709 12:18:16.517025 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1924394160000702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 12:18:16.517405 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0961394159999145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
W20260709 12:18:16.517766 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0223394160000225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 12:18:16.518137 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0142394160000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
W20260709 12:18:16.518757 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0079394159999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260709 12:18:16.455031  2678 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.511222 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.462790  2732 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:16.522929 32042 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:16.469611  2739 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:16.524226 32116 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:18:16.526897 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 72877.5us
I20260709 12:18:16.537626 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:16.529773 32044 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:16.532894 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.538404 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.482764  2737 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:16.515108  2742 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.000s
I20260709 12:18:16.542794 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:16.462904  2716 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 12:18:16.546236 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46119.7us
W20260709 12:18:16.550333 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 98192.6us
I20260709 12:18:16.551155 32043 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:16.555492 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:16.561219 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.569026 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:16.571671 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44755.3us
I20260709 12:18:16.572635 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.534152  2502 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:16.536528  2739 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:16.545732  2741 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:18:16.534960  2526 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:16.538388  2585 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:16.591440  2585 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.591832  2585 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.580535 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.588416  2549 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.588775  2689 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.535648  2685 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:18:16.586225  2344 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:16.590595  2526 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.594182 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.596026 32039 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:16.535382  2423 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:18:16.606053 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.577946  2661 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.609319 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:16.585764  2743 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.610193 32657 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:16.610227  2743 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.610808  2743 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.582333  2686 raft_consensus.cc:2764] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:16.613286 32113 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:16.614094  2747 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.614279 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.616024 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.587244  2739 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:16.621529 32656 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:18:16.617364 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 79083.5us
I20260709 12:18:16.541013  2732 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.575410  2775 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:16.614543  2747 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.618680 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.620438 32652 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.622498 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.629160 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:16.630273  2764 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.426s	user 0.002s	sys 0.000s
I20260709 12:18:16.630601  2423 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.631384  2423 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:16.631784  2705 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:16.634756  2499 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.643085  2775 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.643517  2775 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:16.603034  2726 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.603454  2344 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.648325  2344 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:16.648849 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.649578  2686 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:16.651250  2747 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.622531  2689 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.593545  2527 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.566848  2742 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.634783  2709 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:16.623379 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68019.7us
W20260709 12:18:16.638103  2740 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.565s	user 0.004s	sys 0.003s
I20260709 12:18:16.676407 32660 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:16.681804 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52693.7us
W20260709 12:18:16.666286 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.689589 32115 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:16.691111 32041 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:16.691368 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:16.684180 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.696954 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:16.616040  2540 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:16.698009 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 79391.8us
I20260709 12:18:16.675283  2678 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:16.702361 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:16.703886 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74147.6us
I20260709 12:18:16.699152 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:16.715994 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.690796  2726 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:16.707758  2590 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:16.682988  2696 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:16.700886  2740 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:16.703711  2678 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:16.690371  2674 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.687987  2527 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.782718  2590 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:16.727157  2670 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.869486  2590 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:16.871124  2670 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:16.871361  2782 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.235s	user 0.003s	sys 0.010s
I20260709 12:18:16.872035 31964 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:16.795288 31867 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:16.874217 31867 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 7, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:16.861634  2784 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.135s	user 0.004s	sys 0.003s
W20260709 12:18:16.879916 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 299553us
W20260709 12:18:16.861744 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 137702us
W20260709 12:18:16.862079 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.882453 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:16.882680 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.883109 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 122226us
I20260709 12:18:16.864637  2747 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.865582  2585 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.866444  2546 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:16.723846  2369 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:16.707094  2737 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.876078 31867 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:16.876869 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:16.878623 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 179555us
I20260709 12:18:16.876320  2674 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.883659 32118 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:16.864869  2495 raft_consensus.cc:2764] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:16.894012 31971 raft_consensus.cc:2410] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:16.893162 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:16.894747  2730 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:16.896003  2502 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:16.897221  2499 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.896592  2678 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:16.884127  2726 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:16.930552 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:16.931444 32115 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:16.931262 31867 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:16.946950 31867 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 8, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:16.948493 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71664.7us
I20260709 12:18:16.949153 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.950284 32119 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:16.949863 31867 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:16.925550 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.892062  2342 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:18:16.864691  2743 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.913432  2737 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:16.958236 32084 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:16.899791  2540 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:16.967953 32118 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:16.912352  2546 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:16.919937  2711 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:18:16.919699  2549 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:18:16.924271  2759 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:16.968545 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 77036.2us
I20260709 12:18:16.925719  2740 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:16.979368 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.037449 31867 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 7, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:16.954692  2678 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.953020  2674 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:18:17.038764 32115 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:18:16.990715 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.042172 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 92493.3us
I20260709 12:18:17.017805 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:16.969668 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58544us
I20260709 12:18:17.046622 32660 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:17.046862 32010 leader_election.cc:400] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:17.047727 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:17.048826 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:17.049964 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:17.051141 32115 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.048852 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:17.053785  2423 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:17.054625 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.059691 32115 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.974009  2495 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:17.051676  2740 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:17.045418 31867 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 7, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:17.060753 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 96776.7us
I20260709 12:18:17.068899 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:16.967939  2713 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:18:16.975800 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:16.946870  2502 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.043885 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:17.066341 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:17.075237 32652 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.075716 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:17.076514  2792 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:17.076545  2787 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.077296  2787 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:17.077643  2787 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:17.078419  2791 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:18:17.079123  2788 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:17.079545  2788 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.080464  2788 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.081784  2342 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:17.082722  2705 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:17.083287  2705 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:17.084064  2743 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.084232  2705 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.10, Last appended by leader: 10, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:17.085304 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.086536 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:17.087114 31971 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:17.087924  2802 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.088253 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:17.088373  2802 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.088800 31964 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12. Re-sending same reply.
I20260709 12:18:17.087915  2447 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:17.090421  2447 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.090775  2447 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:17.091161  2730 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:17.082201  2342 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:17.092437  2342 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.094154 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:17.094400  2797 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:17.094738 31963 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.094832  2797 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.095829  2797 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.068334  2744 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:17.096306  2798 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:17.019001  2732 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:16.956529  2790 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.064302  2369 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:17.071693 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 95998us
I20260709 12:18:17.099992 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:17.097857 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:16.959255  2688 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:18:17.104055 32658 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:17.104387 31963 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:17.044407  2742 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.131256 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:17.132128 32658 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:17.114738 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45908.6us
I20260709 12:18:16.956398  2686 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.141s
I20260709 12:18:17.101536  2798 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:17.121150  2764 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:16.960136  2737 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.126639 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.118257  2797 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.154397 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.155400 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.148329 32118 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.126569 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:17.164217 32652 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:17.172995 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56611.7us
I20260709 12:18:17.174571 31867 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 1 to 2, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:17.186466 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.134097  2802 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.115932  2744 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.199855 32118 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:17.196265  2716 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:17.196795  2804 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:17.176057  2739 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.005s
I20260709 12:18:17.129503  2775 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.207733 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:17.202709 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:17.214504 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 161435us
I20260709 12:18:17.216796 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:17.121428  2790 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.217059 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:17.225754 32658 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:17.223642 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68312.3us
W20260709 12:18:17.196259  2794 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:18:17.224330 32117 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:17.136162  2799 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:18:17.147426  2502 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.229923 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.206785  2716 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.200347  2711 raft_consensus.cc:2764] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:17.230726  2630 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:17.231432  2344 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.001s
I20260709 12:18:17.208690  2804 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:17.234733  2789 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:18:17.238406  2802 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.206367  2495 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:18:17.288882 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 219030us
I20260709 12:18:17.227448  2730 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:17.241212 31867 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 9, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 9 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:17.247154 32112 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:17.253739 32658 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:17.226881 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.266535 32113 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.190011  2805 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:17.280627  2759 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:17.281869 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 356395us
I20260709 12:18:17.224903  2585 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.286252 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.224001  2739 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.277973  2716 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.293800 31867 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 6, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 6 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:17.299160 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 201357us
I20260709 12:18:17.311376 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:17.313799 32658 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:17.313269 31867 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 9, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 9 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:17.315701 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:17.319996 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:17.319725 31867 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 7, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:17.321846  2705 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:17.287256  2804 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.289054  2802 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.286677  2630 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.293156  2740 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.292824  2540 consensus_queue.cc:1243] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:17.321697 31867 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:17.288098  2678 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 12:18:17.318101  2689 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.662s	user 0.006s	sys 0.006s
I20260709 12:18:17.213169  2798 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.315070 31936 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:17.380340 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 65350us
I20260709 12:18:17.297955  2805 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:17.310389  2696 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.317559  2732 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:17.315171  2546 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.284507  2344 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.389783  2344 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:17.390601 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.390929  2759 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.391963 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.392490  2758 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.382848  2805 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.392941  2758 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.395176  2758 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:17.396226 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.396791 32112 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.332444  2527 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.332679  2802 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.398710 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.399574 32119 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:17.322410  2705 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.323530  2747 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.284718  2369 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.386658  2732 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:17.386713  2801 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.344241  2787 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.462643  2801 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.464385  2787 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.352466 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:17.400383 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 173567us
I20260709 12:18:17.465909 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:17.401172 31867 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:17.403486  2674 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:17.403767 31967 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:17.328433  2447 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:17.406020  2809 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:18:17.331347  2811 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:17.330096  2730 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.418483 32115 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:17.425374 32112 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:17.427093 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:17.504187 32652 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:17.418594  2705 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:17.433995  2790 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.443156 32654 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:17.504719 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.374935  2800 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:18:17.442167  2766 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:17.412725  2696 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.452498  2788 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.453555  2775 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:17.454628 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.455969  2342 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.467514 31867 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 6, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 6 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:17.469089 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 116714us
W20260709 12:18:17.470700 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 150801us
I20260709 12:18:17.474161 32113 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:17.479962 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:17.465166  2801 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:17.480473 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.469465 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1667) took 1115 ms. Trace:
W20260709 12:18:17.485324 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53043.2us
I20260709 12:18:17.569758 31969 rpcz_store.cc:276] 0709 12:18:16.357790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.365175 (+  7385us) service_pool.cc:224] Handling call
0709 12:18:16.365468 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.382244 (+ 16776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.382288 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.448717 (+ 66429us) log.cc:844] Serialized 26 byte log entry
0709 12:18:16.705548 (+256831us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:16.705616 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.705686 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:17.469251 (+763565us) raft_consensus.cc:1685] finished
0709 12:18:17.469275 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.469404 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65698,"raft.queue_time_us":16975,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":65359,"thread_start_us":179355,"threads_started":1,"wal-append.queue_time_us":475756}
I20260709 12:18:17.488147 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:17.488713 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:17.430198 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.506207 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.506951 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:17.573216 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67123.8us
I20260709 12:18:17.373553  2740 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.575362 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:17.579604 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:17.484826  2585 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:17.523921 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:17.482731  2787 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:17.589717 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:17.549988 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:17.551962 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.552661 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:17.581643 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57828us
I20260709 12:18:17.516701  2798 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.481683  2674 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:17.560778 32660 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:17.561730 31867 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 3 to 6, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 6 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:17.519837  2803 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.465508  2802 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.525413  2766 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:17.550895  2747 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:17.488173  2716 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.577956 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:17.591223 32120 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:17.556321 31968 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:17.591847 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:17.575455  2590 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.705s	user 0.014s	sys 0.000s
I20260709 12:18:17.581729  2423 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:17.557380  2744 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:17.603343 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.606149 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:17.603000 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:17.602613 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:17.588060  2678 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:17.624624 31870 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 243) took 15850 ms (client timeout 15000 ms). Trace:
I20260709 12:18:17.519568  2630 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.599437 32656 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:17.624830 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:17.598680  2670 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:17.759656 31963 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:17.617538 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:17.618604 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53913.4us
I20260709 12:18:17.606763  2142 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.598901  2803 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:17.619936  2742 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.772868 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:17.610756  2801 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:17.581169  2732 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:17.779058 32010 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.516s (SENT)
I20260709 12:18:17.627746  2758 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:17.631124 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:17.630898 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:17.599184  2674 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:17.781518 32653 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 12:18:17.781101 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 219801us
I20260709 12:18:17.637717 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:17.805409 32040 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:18:17.790069 32114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.007s	sys 0.032s
W20260709 12:18:17.806614 32114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.008s	sys 0.032s
I20260709 12:18:17.598997  2686 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:17.610870  2585 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.613253 32112 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:17.623800  2799 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:17.752933  2423 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.827078  2423 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.827844  2423 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.828307  2423 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.830439 32041 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.830209  2423 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.793354 32120 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:17.759927  2739 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.781066  2502 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:17.835788  2502 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:17.836037 31969 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.836743  2502 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.780364  2709 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:17.838690  2709 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.840482 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:17.840781 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:17.840956  2709 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.841320 32046 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:17.841728 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:17.842291 32044 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:17.841214 32047 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:17.846642 32044 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:17.847292 32041 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:17.847962 31969 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:17.610620  2688 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.187s
I20260709 12:18:17.625614  2747 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:17.600972  2787 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.797999  2803 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.630385  2590 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.942154  2590 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.869552 31965 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.869588 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:17.943365 31967 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.794492  2831 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.160s
I20260709 12:18:17.945240  2831 consensus_queue.cc:1243] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:17.945832  2831 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:17.874927 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:17.946194  2831 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:17.869824  2747 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.947255  2831 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.878340  2447 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:17.883469 31972 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:17.896570  2821 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.309s	user 0.000s	sys 0.002s
W20260709 12:18:17.897861  2824 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:18:17.898697  2812 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
W20260709 12:18:17.899705  2823 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:18:17.900534  2820 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 12:18:17.904907 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:17.824486  2585 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.857363  2814 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.814980  2742 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.934847  2342 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:17.936260  2802 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:17.964429 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.064560 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:18.147650 32660 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:17.992014 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 89201.2us
I20260709 12:18:17.855248  2758 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:17.797879  2142 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.859735  2686 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:18.050244 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:18.051085 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:18.200394 32656 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:18.051141 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 445055us
I20260709 12:18:18.010468  2549 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:18.145330 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 245971us
I20260709 12:18:17.984308 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:17.798573  2766 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:17.623981  2730 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:17.979558  2540 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:17.970214 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:17.854929  2822 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.205s
I20260709 12:18:18.007292  2696 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:18.167434 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 113085us
W20260709 12:18:18.177722 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.237506 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.192613 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.992753  2711 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.201591 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:18.239938 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:18.013513 31967 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:18.202834 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:18.151175  2743 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.246701 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:17.779048 31870 rpcz_store.cc:269] 0709 12:18:01.567357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.573810 (+  6453us) service_pool.cc:224] Handling call
0709 12:18:17.416992 (+15843182us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:12.074064 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:12.075010 (+   946us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:12.075084 (+    74us) write_op.cc:435] Acquired schema lock
0709 12:18:12.075126 (+    42us) tablet.cc:662] Decoding operations
0709 12:18:12.203275 (+128149us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:12.203352 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:12.203379 (+    27us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:18:12.215990 (+ 12611us) tablet.cc:701] Row locks acquired
0709 12:18:12.216289 (+   299us) write_op.cc:260] PREPARE: finished
0709 12:18:12.218358 (+  2069us) write_op.cc:270] Start()
0709 12:18:12.220832 (+  2474us) write_op.cc:276] Timestamp: P: 1783599492218083 usec, L: 0
0709 12:18:12.248448 (+ 27616us) op_driver.cc:348] REPLICATION: starting
0709 12:18:12.310221 (+ 61773us) log.cc:844] Serialized 10113 byte log entry
0709 12:18:12.736717 (+426496us) op_driver.cc:464] REPLICATION: finished
0709 12:18:13.649888 (+913171us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f22890c50ac 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228fc05ef0
0709 12:18:13.673636 (+ 23748us) write_op.cc:301] APPLY: starting
0709 12:18:13.691390 (+ 17754us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:13.822440 (+131050us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:13.822499 (+    59us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:14.485660 (+663161us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:14.527306 (+ 41646us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=26
0709 12:18:14.527357 (+    51us) write_op.cc:312] APPLY: finished
0709 12:18:14.527884 (+   527us) log.cc:844] Serialized 176 byte log entry
0709 12:18:14.925663 (+397779us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:15.091178 (+165515us) write_op.cc:454] Released schema lock
0709 12:18:15.130900 (+ 39722us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:15.198739 (+ 67839us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:15.198772 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":600167,"reported_tablets":48,"spinlock_wait_cycles":155904,"child_traces":[["op",{"apply.queue_time_us":930386,"apply.run_cpu_time_us":37443,"apply.run_wall_time_us":1525062,"num_ops":26,"prepare.queue_time_us":129912,"prepare.run_cpu_time_us":40258,"prepare.run_wall_time_us":334831,"raft.queue_time_us":254877,"raft.run_cpu_time_us":62614,"raft.run_wall_time_us":1523862,"replication_time_us":465900,"spinlock_wait_cycles":7562624,"thread_start_us":1893284,"threads_started":5,"wal-append.queue_time_us":776283,"wal-append.run_cpu_time_us":20642,"wal-append.run_wall_time_us":2367058}]]}
I20260709 12:18:18.160552  2790 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.865092  2818 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.675s
W20260709 12:18:18.214687 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 398581us
I20260709 12:18:18.252815 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:18.254662 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:18.221447  2737 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.052s	user 0.010s	sys 0.000s
I20260709 12:18:18.257536 31935 leader_election.cc:400] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11.
I20260709 12:18:18.257908 31935 leader_election.cc:403] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:18:18.237566  2805 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.844s	user 0.011s	sys 0.000s
W20260709 12:18:18.251634 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48874.9us
W20260709 12:18:18.256335 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 286172us
W20260709 12:18:18.250969 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 3.351s (SENT) [suppressed 25 similar messages]
W20260709 12:18:18.263939 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.246037  2342 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:18.261473  2709 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:18.267020  2342 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.267648  2807 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.244789  2495 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:18.243661  2766 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:18.270679 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:18.242900  2716 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:18.212476  2549 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:18.272428 31971 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.006s	sys 0.009s
I20260709 12:18:18.248835  2802 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:18.277825 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.278910 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:18.244495  2688 consensus_queue.cc:1175] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ec7f26ea46944ea498ad253b1b1819b0" responder_term: 8 status { last_received { term: 6 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.6-7.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:18.267196  2709 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:18.275648 31971 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.006s	sys 0.009s
I20260709 12:18:18.281383 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:18.286880 32041 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.287289 32044 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:18:18.268951  2590 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:18.284556 32039 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
W20260709 12:18:18.290480 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.283079 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.002s	sys 0.014s
W20260709 12:18:18.291268 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.002s	sys 0.014s
I20260709 12:18:18.287752 32044 pending_rounds.cc:85] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 5
W20260709 12:18:18.291548 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 688948us
I20260709 12:18:18.291723 32044 pending_rounds.cc:107] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:18:18.292127 32044 raft_consensus.cc:2889] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:18.268163  2807 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:18.272698  2740 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:18.248710  2743 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:18.303321 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.296692 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.304543 32039 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:18.248659  2711 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:18.269484  2775 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:18:18.252526  2801 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:18:18.256147  2790 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:18.312891 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.313104  2815 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.156s
I20260709 12:18:18.269100  2737 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:18.304940  2797 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.146s	user 0.006s	sys 0.005s
W20260709 12:18:18.323647 31966 consensus_meta.cc:287] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.000s
I20260709 12:18:18.341730 31966 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:18:18.328981 32041 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:18.330013 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:18.330669 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.345675 32046 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:18.331352 32044 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20260709 12:18:18.270107  2805 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:18.358210 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:18.358966 32038 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:18.353994 32039 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:18.350170 32118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.005s	sys 0.013s
W20260709 12:18:18.370142 32118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.005s	sys 0.013s
I20260709 12:18:18.379874 31964 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:18.381647  2759 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:18.276687  2758 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:18.354514  2811 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.000s	sys 0.018s
I20260709 12:18:18.388132 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:18.356961  2825 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.498s	user 0.001s	sys 0.000s
I20260709 12:18:18.288959 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:18.390266  2759 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:18.390866 31963 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:18.392647 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.393996 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:18.402904 31966 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:18.403831 31967 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:18.390805 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:18.404958 31969 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.388512 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.390899  2835 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:18.380004  2798 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.778s	user 0.011s	sys 0.000s
W20260709 12:18:18.437683 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 453413us
I20260709 12:18:18.413808 32038 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
W20260709 12:18:18.398669  2811 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.001s	sys 0.018s
I20260709 12:18:18.429426 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:18.430294 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.436357 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.437111 31969 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:18.288866  2686 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:18.454036 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:18.388067  2787 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:18.452960 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.519230 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.519609 31966 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:18.462522  2747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.009s
I20260709 12:18:18.511075 31968 raft_consensus.cc:2368] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12.
I20260709 12:18:18.391526  2670 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.388622  2740 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:18.521543 32040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.004s	sys 0.013s
I20260709 12:18:18.432175  2447 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:18.453035  2832 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:18:18.453006  2810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.002s	sys 0.015s
I20260709 12:18:18.391033  2836 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:18.453097  2819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.018s	sys 0.026s
I20260709 12:18:18.534598 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.551612 31969 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:18:18.540836 32040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.004s	sys 0.013s
I20260709 12:18:18.553686 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.557188 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:18.542186 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:18.542665 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1863) took 1367 ms. Trace:
I20260709 12:18:18.557842 32039 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:18.559542 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:18.538347  2747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.006s	sys 0.010s
I20260709 12:18:18.460258  2742 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:18.447541  2142 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:18.457116  2743 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:18.459438  2840 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.455879  2813 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:18.439978  2833 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:18:18.566268 32116 rpcz_store.cc:276] 0709 12:18:17.175140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.209666 (+ 34526us) service_pool.cc:224] Handling call
0709 12:18:17.210078 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.224832 (+ 14754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.224865 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.225376 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.405487 (+180111us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b180032cc80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:18:17.569415 (+163928us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:17.569450 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.569510 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.533054 (+963544us) raft_consensus.cc:1685] finished
0709 12:18:18.533080 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.542586 (+  9506us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2841,"raft.queue_time_us":4805,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":3059,"spinlock_wait_cycles":77442944,"thread_start_us":163729,"threads_started":1,"wal-append.queue_time_us":524802}
I20260709 12:18:18.439416 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.568176 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:18.571107 31966 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:18.571235 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:18.536607  2844 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:18.487002  2811 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:18.439029 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:18.630060 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 1999) took 1223 ms. Trace:
I20260709 12:18:18.631312 32044 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:18.629712 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1677) took 1005 ms. Trace:
I20260709 12:18:18.647892 31964 rpcz_store.cc:276] 0709 12:18:17.624200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.624310 (+   110us) service_pool.cc:224] Handling call
0709 12:18:17.624584 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.380514 (+755930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.629480 (+248966us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54004303d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:18.629647 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736403200}
I20260709 12:18:18.647099 32118 rpcz_store.cc:276] 0709 12:18:17.406726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.407244 (+   518us) service_pool.cc:224] Handling call
0709 12:18:17.407575 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.407922 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.407952 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.408411 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.370643 (+962232us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:18.370701 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.370792 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.629867 (+259075us) raft_consensus.cc:1685] finished
0709 12:18:18.629885 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.630005 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25655,"raft.queue_time_us":7884,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":11111,"thread_start_us":961833,"threads_started":1,"wal-append.queue_time_us":1048720}
W20260709 12:18:18.557334  2810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.002s	sys 0.017s
W20260709 12:18:18.625963  2839 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 12:18:18.556728  2549 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:18.557241  2836 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:18.557585  2819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.019s	sys 0.027s
I20260709 12:18:18.639269 31972 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:18.636644 32660 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:18.440137  2835 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:18.588586  2842 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:18.552618  2740 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:18.547722  2787 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:18.655062 32044 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:18.656868 32041 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:18.588686 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:18.547945  2670 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:18.537477  2759 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:18.636612 31968 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:18.674540 32657 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:19.004112 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 565102us
I20260709 12:18:18.676288 32656 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:19.008392 32653 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:18:18.967746  2590 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.002s	sys 0.016s
W20260709 12:18:18.836757 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.049803  2590 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.004s	sys 0.020s
W20260709 12:18:18.712946 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.051254  2743 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.051788  2743 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.053069 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.053846  2743 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.051985 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.054859 32118 raft_consensus.cc:2368] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:19.055015 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:18.605679  2447 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:19.055662 32044 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:19.056255  2447 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:18.681617 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.058722 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:19.059319  2858 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.059834  2858 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:19.060552  2857 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:18:19.061260  2705 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.540s	user 0.003s	sys 0.006s
I20260709 12:18:19.060813  2759 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:18.756019  2711 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:19.062108  2851 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.456s	user 0.001s	sys 0.000s
W20260709 12:18:18.782282 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.059840 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:19.061172 32046 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:19.084906  2688 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:19.085466  2688 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.085916 32121 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:19.086056 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:18.587975  2862 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.063645 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1748) took 1223 ms. Trace:
I20260709 12:18:19.062403  2858 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.065325  2787 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.066082  2742 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.108028  2711 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.088025  2862 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.123387  2742 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.123700  2711 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.124480  2705 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.066973 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.125806  2862 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.126129 32041 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:19.126399  2853 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:19.127754  2705 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.128548  2853 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:19.129067  2853 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.089020  2836 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.125844  2787 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:18.641485  2813 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.133104  2853 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:18.959002 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.125679  2711 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:19.134428  2813 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.108500  2726 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.719604  2142 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:18.647076  2844 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.068310  2549 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:18.740619  2709 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:19.136873  2709 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.137624  2844 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.070504  2811 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:18.641618  2840 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:18.719880  2747 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:19.072042 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1743) took 1293 ms. Trace:
W20260709 12:18:19.076861  2788 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.526s	user 0.008s	sys 0.004s
W20260709 12:18:19.077481  2804 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.742s	user 0.012s	sys 0.000s
I20260709 12:18:19.077406 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:19.078650  2803 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.098s	user 0.010s	sys 0.000s
I20260709 12:18:19.085081 32046 pending_rounds.cc:85] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 4
I20260709 12:18:19.088619 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:19.005717 32042 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:19.156471 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.157734 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:18.706279 32039 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:19.158900 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.159165  2810 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:19.159632  2810 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:19.160192 32039 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:19.160542  2810 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.108357 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.162083 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.162355 31968 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:19.163106  2858 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.163771 31968 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:19.134213 32041 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:19.158254 32042 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.165067 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.165272  2835 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.165659 31972 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.166044  2803 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.168833  2803 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.169415  2836 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.170969 31963 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:19.008292  2737 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.122480  2670 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.135454  2142 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.056664 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:18.647529  2678 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.009s	sys 0.038s
W20260709 12:18:19.001328  2841 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
I20260709 12:18:19.141278 32045 rpcz_store.cc:276] 0709 12:18:17.778285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.779227 (+   942us) service_pool.cc:224] Handling call
0709 12:18:17.779537 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.005880 (+226343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.005935 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.063507 (+ 57572us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.476811 (+413304us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:18.476864 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.476966 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.071827 (+594861us) raft_consensus.cc:1685] finished
0709 12:18:19.071858 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.071986 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195,"raft.queue_time_us":47763,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"thread_start_us":466284,"threads_started":2,"wal-append.queue_time_us":732567}
I20260709 12:18:19.144876 31970 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:18.801205  2852 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
W20260709 12:18:18.919106  2823 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:18:19.146699 32046 pending_rounds.cc:107] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 8.5
W20260709 12:18:19.155382 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.233162 32046 raft_consensus.cc:2889] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:19.146291 31965 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:19.163604  2858 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.164225 31968 pending_rounds.cc:85] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 5
I20260709 12:18:19.239115 31968 pending_rounds.cc:107] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:18:19.239405 31968 raft_consensus.cc:2889] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:19.088148 32043 rpcz_store.cc:276] 0709 12:18:17.840697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.842122 (+  1425us) service_pool.cc:224] Handling call
0709 12:18:17.842459 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.842832 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.842865 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.843295 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.220485 (+377190us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:18.479005 (+258520us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.524103 (+ 45098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.524200 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.063442 (+539242us) raft_consensus.cc:1685] finished
0709 12:18:19.063461 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.063591 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43104,"prepare.queue_time_us":61672,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":86824,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":180,"thread_start_us":612883,"threads_started":2,"wal-append.queue_time_us":559615}
I20260709 12:18:19.120906 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:19.240903 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 120250us
I20260709 12:18:19.112643 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.088011  2688 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:19.176964  2799 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:19.177417 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.006s	sys 0.006s
W20260709 12:18:19.251984 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.007s	sys 0.007s
W20260709 12:18:19.252274 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 953377us
I20260709 12:18:19.249604 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:18.829320  2807 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:19.037171  2790 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:18.968858  2805 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.179191 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1678) took 1554 ms. Trace:
I20260709 12:18:19.259339 31971 rpcz_store.cc:276] 0709 12:18:17.624460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.625258 (+   798us) service_pool.cc:224] Handling call
0709 12:18:17.752835 (+127577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:18.295194 (+542359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.295229 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:18.324658 (+ 29429us) log.cc:844] Serialized 52 byte log entry
0709 12:18:18.570208 (+245550us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:18.801753 (+231545us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.089346 (+287593us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.176583 (+ 87237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.176689 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.176744 (+    55us) raft_consensus.cc:1685] finished
0709 12:18:19.176764 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.179136 (+  2372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87458,"raft.queue_time_us":491047,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":73553,"thread_start_us":897330,"threads_started":3,"wal-append.queue_time_us":632036}
W20260709 12:18:19.156883  2674 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.297s	user 0.012s	sys 0.003s
I20260709 12:18:19.215775 32044 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.261111  2674 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.262444  2344 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:19.163522  2837 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:19.265218  2674 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.225239 32042 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:18:19.229035 32047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.021s	sys 0.033s
W20260709 12:18:19.266026 32047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.021s	sys 0.034s
I20260709 12:18:19.047806  2740 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.245905 31964 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:19.176321 31972 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:19.255039 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:19.255764 32624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.010s
I20260709 12:18:19.219290  2142 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:19.227169  2678 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.426s	user 0.015s	sys 0.057s
I20260709 12:18:19.256878 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:19.268433 32624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.004s	sys 0.010s
W20260709 12:18:19.317111 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62106us
W20260709 12:18:19.238024  2342 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.971s	user 0.011s	sys 0.003s
I20260709 12:18:19.154865  2804 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.141129  2747 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.317725 31968 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:19.247258  2840 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.316550  2678 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:19.359038  2342 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11.
I20260709 12:18:19.448887 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1972) took 1052 ms. Trace:
I20260709 12:18:19.449268 31963 rpcz_store.cc:276] 0709 12:18:18.396216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.429024 (+ 32808us) service_pool.cc:224] Handling call
0709 12:18:19.448684 (+1019660us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400312c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:19.448815 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536217984}
I20260709 12:18:19.450896 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.250458  2549 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:19.452016 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1962) took 1062 ms. Trace:
I20260709 12:18:19.452436 31965 rpcz_store.cc:276] 0709 12:18:18.389847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.389949 (+   102us) service_pool.cc:224] Handling call
0709 12:18:19.451850 (+1061901us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54001e9bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:19.451959 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462087552}
I20260709 12:18:19.453678 31969 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:19.271119  2799 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.267613  2716 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:18:19.434134 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.467133 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 661) took 1123 ms. Trace:
I20260709 12:18:19.440203  2540 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:19.441820 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.442839 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.268777  2688 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:19.473760 31964 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:19.432669 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.479434 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.449467 32042 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:19.436478  2805 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:19.461536 31966 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:19.469208 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.472101 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.481155 32041 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.481386 31963 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:19.434895 32044 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:19.475036 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.486171 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.487697 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.488317 32044 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:19.473347 32658 rpcz_store.cc:276] 0709 12:18:18.343461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.343656 (+   195us) service_pool.cc:224] Handling call
0709 12:18:18.343953 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.344151 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:18.675242 (+331091us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.718380 (+ 43138us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.466788 (+748408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.466846 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:19.466894 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.466957 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.467074 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":385657,"prepare.queue_time_us":1450128,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3624,"raft.queue_time_us":588456,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":189,"thread_start_us":636317,"threads_started":2}
I20260709 12:18:19.477216 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.491375 32041 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:19.493961 32116 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:19.462570  2549 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:19.495375 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.472153  2342 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.502511 31964 raft_consensus.cc:2368] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:19.506431 31968 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:19.511025 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:19.477476  2540 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.513266 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.514951 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.477056  2759 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:19.516786 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.518448 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1968) took 1123 ms. Trace:
I20260709 12:18:19.505730 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:19.518855 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:19.519060 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:19.518833 31969 rpcz_store.cc:276] 0709 12:18:18.394812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.568588 (+173776us) service_pool.cc:224] Handling call
0709 12:18:19.453341 (+884753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.454059 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.518242 (+ 64183us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54001e9bd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:19.518391 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033881856}
I20260709 12:18:19.503185  2549 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.525058  2678 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.528748 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:19.530171 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.530916  2858 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.482944  2737 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.532835  2737 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.517397 32042 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:19.536989 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.490443  2711 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:19.545509 32652 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:19.504158  2805 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:19.506431  2447 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:19.541543  2852 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:19.564576 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.539103  2585 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.532557  2831 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:19.584728 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 152666us
I20260709 12:18:19.574596 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.588946 32118 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 3.
I20260709 12:18:19.542191  2369 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:19.574632 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.543032  2835 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.516270  2342 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.569406  2689 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:19.589737 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:19.511492  2688 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:18:19.572253  2726 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:19.588155  2758 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.561725  2802 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:19.541706  2759 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.565425  2861 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.592665  2836 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:19.594691 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 379354us
I20260709 12:18:19.609812  2758 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.631420  2747 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:19.631505  2804 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:19.596586  2369 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.625263  2861 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:19.633091  2861 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.633303  2758 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:19.564903  2711 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.633818  2369 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:19.634572  2873 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:18:19.635982  2711 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.600469  2737 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.636739 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1744) took 1815 ms. Trace:
I20260709 12:18:19.627290 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:19.637163 32040 rpcz_store.cc:276] 0709 12:18:17.821743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.821859 (+   116us) service_pool.cc:224] Handling call
0709 12:18:17.822157 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:18.553451 (+731294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.553485 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:18.554573 (+  1088us) log.cc:844] Serialized 52 byte log entry
0709 12:18:18.957659 (+403086us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:18.957718 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.957822 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.636500 (+678678us) raft_consensus.cc:1685] finished
0709 12:18:19.636521 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.636672 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12455,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":71,"thread_start_us":1045787,"threads_started":2,"wal-append.queue_time_us":629096}
I20260709 12:18:19.638500  2688 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:19.639021  2688 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:19.640252  2759 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.639933  2805 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.640849  2759 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:19.641244  2759 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:19.627744 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:19.647071 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1764) took 1078 ms. Trace:
I20260709 12:18:19.647507 32047 rpcz_store.cc:276] 0709 12:18:18.568186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.628430 (+ 60244us) service_pool.cc:224] Handling call
0709 12:18:18.628738 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:18.629133 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.629160 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:18.629647 (+   487us) log.cc:844] Serialized 52 byte log entry
0709 12:18:19.266484 (+636837us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:19.266809 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.435746 (+168937us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.466601 (+ 30855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.466678 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.646862 (+180184us) raft_consensus.cc:1685] finished
0709 12:18:19.646882 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.647013 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64625,"prepare.queue_time_us":458,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":74,"raft.queue_time_us":200153,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":9287,"spinlock_wait_cycles":25574272,"thread_start_us":636438,"threads_started":1,"wal-append.queue_time_us":959519}
I20260709 12:18:19.646131  2759 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.650769  2890 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:18:19.651196  2766 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.374s	user 0.010s	sys 0.013s
I20260709 12:18:19.599509  2835 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.570196  2447 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.653607  2549 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.656302  2447 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.658380  2747 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:19.659240  2747 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:19.659612  2747 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:19.659438  2742 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.660478  2747 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.653319  2759 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.662281 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46882 (request call id 11) took 1403 ms. Trace:
I20260709 12:18:19.591610  2585 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.662678 31967 rpcz_store.cc:276] 0709 12:18:18.276522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.276640 (+   118us) service_pool.cc:224] Handling call
0709 12:18:19.662088 (+1385448us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b54001e8cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1c9853 00007f229119760a 00007f228a71d7d4 00007f228a71d762 00007f228a71d691 00007f228a71d60c 00007f228a71c8c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:19.662171 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10380160}
I20260709 12:18:19.664323  2585 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.661643  2747 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.668449 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 585) took 1429 ms. Trace:
I20260709 12:18:19.668844 32653 rpcz_store.cc:276] 0709 12:18:18.239482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.241171 (+  1689us) service_pool.cc:224] Handling call
0709 12:18:19.008181 (+767010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.049571 (+ 41390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.668240 (+618669us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54003bae50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:19.668392 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763024128}
I20260709 12:18:19.650139  2142 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.672740  2810 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:19.673511  2342 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.673660 32116 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:19.686537  2827 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.478s	user 0.003s	sys 0.001s
I20260709 12:18:19.688094 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1856) took 2639 ms. Trace:
I20260709 12:18:19.688997  2799 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.688431 32114 rpcz_store.cc:276] 0709 12:18:17.048643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.196785 (+148142us) service_pool.cc:224] Handling call
0709 12:18:17.197142 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.197456 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.197485 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.197912 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.887566 (+689654us) spinlock_profiling.cc:243] Waited 8.59 ms on lock 0x7b180032cec0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1
0709 12:18:17.887704 (+   138us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:17.887738 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.887810 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.685908 (+1798098us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:19.687899 (+  1991us) raft_consensus.cc:1685] finished
0709 12:18:19.687919 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.688036 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3121,"raft.queue_time_us":1560,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":5049,"spinlock_wait_cycles":53494400,"thread_start_us":608801,"threads_started":1,"wal-append.queue_time_us":601924}
I20260709 12:18:19.691478 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.654157  2835 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.691793  2814 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.692469  2814 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.694657 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.680642  2142 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.696871  2858 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.697343  2858 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.698776 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1766) took 1068 ms. Trace:
I20260709 12:18:19.698897  2369 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.699446  2369 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.639957  2688 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.700300  2837 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:19.699189 32046 rpcz_store.cc:276] 0709 12:18:18.630672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.647081 (+ 16409us) service_pool.cc:224] Handling call
0709 12:18:19.005294 (+358213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.233852 (+228558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.698557 (+464705us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540032ad54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:19.698718 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127680256}
I20260709 12:18:19.699749  2858 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:19.698951 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.690418 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:19.724733 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.725538 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:19.727473 31966 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:19.728941 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.729751 31969 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:19.730813 32047 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [5.6-7.8]
I20260709 12:18:19.731621 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.732332 31971 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.733953 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.734956 31967 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:19.724144  2732 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.740976 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.740947  2732 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:19.741493  2369 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:19.742151 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.742805 31963 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:19.743095 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.743837 31972 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:19.744264 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.744836 32046 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:19.747993 31972 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:19.763902 31969 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 10. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20260709 12:18:19.773588 32013 leader_election.cc:400] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:19.774067 32013 leader_election.cc:403] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:18:19.744743 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54423.4us
W20260709 12:18:19.774530 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 255742us
I20260709 12:18:19.775774 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:19.776503 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.776787 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.777400 32046 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.777278 32040 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:19.778735 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.780555 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.781283 31969 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
W20260709 12:18:19.782800 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57384.4us
I20260709 12:18:19.785393 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.786146 32040 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:19.786296 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.782912 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.811303 32117 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:18:19.820839 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47406us
I20260709 12:18:19.823304 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.824986 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.826812 32043 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:19.830300 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46420.6us
W20260709 12:18:19.830384 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45854.9us
I20260709 12:18:19.830857 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:19.743835  2885 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:19.834389 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.822747  2860 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.835021 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.835611 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.835867 32042 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:19.836160 31965 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.835317  2860 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.842157 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:19.845939 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:19.777808  2742 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.851967 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:19.852802 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.852974 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.853454 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.854295 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:19.857436 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.785295  2837 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.895807 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.859674 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:19.896950 32041 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.897517 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:19.898379 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54029.8us
I20260709 12:18:19.858472 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.899326 31963 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:19.900777 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:19.898532 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46639.9us
I20260709 12:18:19.901367 32043 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:19.823071  2737 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:19.902397 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:19.904800 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:19.826783  2790 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:19.822949  2802 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:19.948385 32120 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:19.823655  2880 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:19.907487 32660 raft_consensus.cc:2410] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:19.950727 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:19.898376 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52468.6us
W20260709 12:18:19.971864 32086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.014s
I20260709 12:18:19.830243  2803 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:19.907258 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.867193 32046 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:19.937209 32041 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:19.954581 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:19.980404 31963 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:19.980887 31963 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:19.953066 31965 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:19.981617 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.906301 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:19.977151 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.978840 32044 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:19.977629 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75343.5us
W20260709 12:18:19.973768 32086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.004s	sys 0.014s
I20260709 12:18:19.837805  2885 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:19.837706  2890 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:19.832531  2540 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:19.989718 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:20.041663 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.591081 32043 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:20.382421 32623 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.011s	sys 0.011s
I20260709 12:18:19.851934  2799 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.046083 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.858031  2342 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:19.903255  2888 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.588110 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:19.989601 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:20.611495 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:20.612507 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 623472us
I20260709 12:18:20.612974 31965 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:20.613229 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:20.156037 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.003s	sys 0.031s
I20260709 12:18:19.987769 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:20.614511 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.015s	sys 0.038s
I20260709 12:18:20.615011 32121 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:20.616293 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.617075 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.617599 32652 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:20.618140 32652 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:18:20.505611 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 531709us
I20260709 12:18:20.619959 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:20.322486  2759 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.007s
I20260709 12:18:20.102082 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:19.990140 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:20.622911 32113 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:20.623322 31966 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:18:19.973388  2686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.018s	sys 0.050s
I20260709 12:18:19.857580  2766 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:19.895376  2549 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.016781 32114 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:18:20.603796 32623 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.019s	sys 0.015s
I20260709 12:18:19.946911  2819 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:19.946858  2747 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:20.088222 32120 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:19.896574  2833 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.728s
I20260709 12:18:20.325505 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 1983) took 1109 ms. Trace:
I20260709 12:18:20.298466 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.607594  2142 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.946743  2447 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:20.190578  2805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.012s	sys 0.018s
I20260709 12:18:20.092578 32119 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:20.612157  2858 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:20.648118 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.002s	sys 0.014s
W20260709 12:18:20.648473 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.002s	sys 0.014s
W20260709 12:18:20.593981 31971 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.861s	user 0.010s	sys 0.002s
W20260709 12:18:20.101677  2892 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:19.906209  2837 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.627591 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:19.977634  2745 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.625561  2709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.015s	sys 0.020s
I20260709 12:18:19.976845  2880 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:20.280995  2732 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:19.946760  2674 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.661s
I20260709 12:18:20.016314  2711 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:20.323186  2814 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.630s	user 0.006s	sys 0.003s
W20260709 12:18:20.633332 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 832460us
I20260709 12:18:20.654049 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:20.321544  2861 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.001s
I20260709 12:18:20.629833 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:20.121408  2860 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:20.047969  2803 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.639108  2819 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.664916  2819 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:20.592657 32117 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.620s	user 0.004s	sys 0.008s
I20260709 12:18:20.666517 32117 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:20.667430 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.641376 32651 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:20.639536 31970 rpcz_store.cc:276] 0709 12:18:19.215938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.239851 (+ 23913us) service_pool.cc:224] Handling call
0709 12:18:19.432514 (+192663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.467898 (+ 35384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.467981 (+    83us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.490650 (+ 22669us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.624047 (+133397us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:19.624364 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.624584 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.624674 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.325243 (+700569us) raft_consensus.cc:1685] finished
0709 12:18:20.325269 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.325434 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15285,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":30330,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":3555,"thread_start_us":82026,"threads_started":1,"wal-append.queue_time_us":532445}
I20260709 12:18:20.669514 31970 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:20.638643  2747 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:20.611763  2790 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.670590  2747 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:20.670935  2790 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.673017  2790 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.674110  2674 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:20.628813  2899 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:20.666149  2803 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.677167  2837 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:20.626818  2888 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.678063  2861 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.679090 32117 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:20.646427 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.385149  2831 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.914878  2742 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.680078  2888 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.680577  2888 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.681005  2888 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.681680  2742 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.682507 32042 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.682927 32651 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:20.684238 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.684803 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:20.685691  2861 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.686617 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:20.689846  2810 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.017s	user 0.007s	sys 0.002s
I20260709 12:18:20.691105  2590 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.691593  2590 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.693140 32651 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:20.693606  2590 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.695629 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1780) took 1233 ms. Trace:
I20260709 12:18:20.696692 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:20.696280 32045 rpcz_store.cc:276] 0709 12:18:19.462043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.462182 (+   139us) service_pool.cc:224] Handling call
0709 12:18:19.462530 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.462849 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.462889 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.474651 (+ 11762us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.588426 (+113775us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:19.588471 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.588543 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.695347 (+1106804us) raft_consensus.cc:1685] finished
0709 12:18:20.695384 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.695556 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":679,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":7173,"thread_start_us":74182,"threads_started":1,"wal-append.queue_time_us":340287}
I20260709 12:18:20.629168  2758 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:20.698757 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.699465  2742 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.699966 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 371) took 1195 ms. Trace:
I20260709 12:18:20.699985  2742 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.653041  2447 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.658643  2759 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.702147  2759 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.115491  2800 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.523s
I20260709 12:18:20.659017  2745 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:20.659825  2709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.016s	sys 0.020s
W20260709 12:18:20.660084  2896 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:18:20.660156  2880 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:20.650520  2805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.014s	sys 0.033s
I20260709 12:18:20.661538  2711 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.661927  2814 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.662668 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.736202 32121 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:20.663241  2712 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.664212 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:20.739207 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75057.6us
I20260709 12:18:20.740201 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:20.695606  2861 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:20.697873 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:20.746800 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48998.2us
I20260709 12:18:20.700455 32116 rpcz_store.cc:276] 0709 12:18:19.505350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.505487 (+   137us) service_pool.cc:224] Handling call
0709 12:18:20.699783 (+1194296us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54003e49d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:20.699908 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352975104}
I20260709 12:18:20.628968  2342 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:20.658950 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.655726  2799 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.643234  2737 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.004s	sys 0.009s
I20260709 12:18:20.649019 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:20.752589 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 317) took 1021 ms. Trace:
I20260709 12:18:20.753083 31971 rpcz_store.cc:276] 0709 12:18:19.731093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.731342 (+   249us) service_pool.cc:224] Handling call
0709 12:18:20.752511 (+1021169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:20.756292 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.757020 32116 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:20.710363 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:20.607530  2890 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.642486  2833 consensus_queue.cc:1243] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:20.721129  2759 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.713344  2837 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.723863  2888 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.727324 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.777834 32120 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:20.734936 32043 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:19.977985  2835 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.656s
I20260709 12:18:20.781891 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.783036 32119 raft_consensus.cc:2410] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:20.674431  2893 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:20.635083  2766 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.749720 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:20.717820  2836 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:18:20.703768  2447 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.715286  2742 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.676059  2899 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:18:20.760651 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50369.8us
W20260709 12:18:20.767670 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 136698us
I20260709 12:18:20.765538 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.793088 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.735654  2711 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.652104  2142 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.779222 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:20.759050  2737 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.005s	sys 0.010s
I20260709 12:18:20.785377 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:20.709415 32042 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:20.752796  2590 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:20.756151  2713 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.753048  2790 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:20.704720  2858 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.789273 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.799486 32120 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:20.719480  2831 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.795313 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:20.796190 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.790673  2836 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:20.727772  2885 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.809801 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:20.802717 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.753396  2712 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.790632  2905 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.801510  2142 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.811084 32119 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:20.814276 32120 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.760625  2799 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:18:20.794530  2344 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:20.817389 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.797147  2737 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:20.798579  2713 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.784961  2837 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.836186 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.837141 32119 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:20.823619  2888 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.807642  2590 raft_consensus.cc:2764] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:20.815945  2790 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:20.842743 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:20.853310 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.853475 32119 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:20.854125 32043 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:20.848984 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.855268 32041 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:20.849238 32121 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:20.828500 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:20.858462 32044 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:20.859367  2790 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.833851  2885 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:20.872653 32044 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:20.835680  2344 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:20.882732 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77895.7us
I20260709 12:18:20.839141  2713 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.885842 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.886564 32041 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:20.878247 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.844805  2888 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.847687  2540 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.888197 32121 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.891324 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:20.905992 32041 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.906023 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.892210 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:20.908286 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:20.908061 32652 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:20.835076  2906 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:18:20.909947 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:20.916616 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:20.894829 32044 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:20.894758  2897 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:20.895296 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.895191  2805 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:20.896786 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.930871 32046 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:20.897368  2898 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.900128  2888 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.808151  2342 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.898914 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.919534 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:20.877646  2674 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:20.921690 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 210123us
I20260709 12:18:20.934890 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:20.897558 32045 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:18:20.938266 32010 connection.cc:256] RPC call timeout handler was delayed by -2.2601481060000879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 12:18:20.925870 32043 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:20.938073 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:20.939093 32010 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.131s (SENT)
I20260709 12:18:20.939630 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:20.943058 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.945870 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:20.946100 32010 connection.cc:256] RPC call timeout handler was delayed by -2.2474339729999429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2647s for a response
I20260709 12:18:20.911358  2907 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.946502 32121 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:20.947535 32010 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.256s (SENT)
I20260709 12:18:20.948000 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:20.948572 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:20.949947 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.946512 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:20.950603 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:20.951004 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.951412 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:20.922513  2709 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.931231  2836 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.930807  2540 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.935148  2344 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:20.931502  2732 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:20.957100 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:20.932938  2810 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.960299 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:20.968051 32043 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:20.952554  2860 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:20.935016 32041 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:20.935325  2898 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.962229 32045 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:20.963498 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:20.972999 32651 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:20.947132  2814 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.922820  2447 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:18:20.962190  2540 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.952831  2790 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:20.981345 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 152961us
I20260709 12:18:20.978374 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:20.970499 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:20.965696  2890 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:20.966040  2914 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.970736  2810 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.970980  2766 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:20.994336 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:20.935561  2826 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:18:20.935669  2737 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.975643  2898 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.991302  2903 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:18:20.976015  2911 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.984622  2732 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:20.990386  2540 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.990888  2790 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.991492  2709 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:21.094656 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.966825  2836 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.999498  2897 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:20.996033  2831 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:20.978686  2800 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:18:20.998921  2914 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:21.167089  2914 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.000893  2810 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.960240  2907 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:20.970832  2917 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
W20260709 12:18:21.123530  2901 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 12:18:21.136927  2831 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:21.116642  2897 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:21.019534  2911 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.263114  2689 raft_consensus.cc:993] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:18:21.108850  2790 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.109221  2709 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.107647  2712 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:21.005019  2674 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.519713 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:21.527901 32112 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:18:21.283761 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:21.542236 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 594664us
W20260709 12:18:21.543118 32010 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.400s (SENT)
W20260709 12:18:21.545472 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.006s	sys 0.015s
W20260709 12:18:21.546142 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.015s
W20260709 12:18:21.546561 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 595192us
I20260709 12:18:21.547678 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:21.473286  2742 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.001s
W20260709 12:18:21.513860  2880 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.002s
W20260709 12:18:21.487613  2925 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
W20260709 12:18:21.501507  2747 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.002s
I20260709 12:18:21.265666  2858 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:21.396289  2922 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.388s	user 0.000s	sys 0.000s
W20260709 12:18:21.395159  2921 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
W20260709 12:18:21.551424 32010 connection.cc:256] RPC call timeout handler was delayed by -2.1377481060001173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
I20260709 12:18:21.279125  2911 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:21.551605 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.001s	sys 0.011s
W20260709 12:18:21.551961 32625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.005s
I20260709 12:18:21.277838  2897 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:21.543941  2819 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.879s	user 0.007s	sys 0.004s
I20260709 12:18:21.331190  2840 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:21.553560 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:21.279376  2709 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.554849 32112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:21.286875  2799 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:21.530776  2447 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:21.554255  2742 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:21.572757 32625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.008s	sys 0.005s
W20260709 12:18:21.586681 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.569168  2898 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:21.577111  2713 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.000s
I20260709 12:18:21.552655  2814 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:21.584685 32112 raft_consensus.cc:606] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:21.572903 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.001s	sys 0.011s
W20260709 12:18:21.589514 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 633768us
I20260709 12:18:21.589430 32112 raft_consensus.cc:614] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:21.594424 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:21.603351 32044 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:21.595203 32040 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:21.587574  2885 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.001s
I20260709 12:18:21.556982  2747 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:21.554365  2880 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.587759  2836 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:21.585286  2840 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:21.618182  2935 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.635161  2742 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:21.639992 32652 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:21.641224 32658 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:21.586449  2819 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.645948 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:21.650184 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:21.594519  2447 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.603503  2814 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:21.651757 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:21.666539 32656 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:21.595674  2713 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:21.664470 32112 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:21.664736 32658 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:21.668052 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:21.669445  2898 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:21.653041  2935 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.623818  2934 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 12:18:21.658443  2711 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.856s	user 0.009s	sys 0.004s
W20260709 12:18:21.678085  2540 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.000s
W20260709 12:18:21.929850 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 134447us
W20260709 12:18:21.696682 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.965970 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:21.974550 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2000) took 1054 ms. Trace:
I20260709 12:18:21.975127 31968 rpcz_store.cc:276] 0709 12:18:20.919991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.965600 (+1045609us) service_pool.cc:224] Handling call
0709 12:18:21.974471 (+  8871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:21.698704 31970 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:21.975627  2938 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.927073  2805 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.929962  2840 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.979339  2938 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:21.691813  2447 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:21.980297  2805 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.981725  2713 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:21.983470 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2020) took 1196 ms. Trace:
I20260709 12:18:21.983911 32112 rpcz_store.cc:276] 0709 12:18:20.787317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.982265 (+1194948us) service_pool.cc:224] Handling call
0709 12:18:21.982948 (+   683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:21.983136 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:21.983159 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:21.983199 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:21.983231 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.983287 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:21.983410 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:21.985087  2907 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:21.685799  2814 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:21.981910  2938 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:21.933085  2888 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.987s	user 0.007s	sys 0.004s
I20260709 12:18:21.978711 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:21.965304  2810 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.000s
W20260709 12:18:21.951912  2914 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
I20260709 12:18:21.989580 31968 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:21.935942  2819 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:21.990121  2888 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:21.991904  2914 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:21.989645  2790 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.000s
I20260709 12:18:21.995301  2747 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:21.997069  2932 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.442s	user 0.008s	sys 0.020s
I20260709 12:18:21.998718  2810 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:21.977607  2540 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:22.042845  2810 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:22.043952 32121 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:22.044029  2540 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:21.977777  2711 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.046927  2711 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:22.048216  2846 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 3.409s	user 0.000s	sys 0.061s
W20260709 12:18:22.053084 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.060446 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1777) took 1157 ms. Trace:
W20260709 12:18:22.060659 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 513017us
I20260709 12:18:22.061726 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:22.062438  2837 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.061028 32046 rpcz_store.cc:276] 0709 12:18:20.903281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.960632 (+ 57351us) service_pool.cc:224] Handling call
0709 12:18:20.974745 (+ 14113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.974962 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:20.989685 (+ 14723us) log.cc:844] Serialized 14 byte log entry
0709 12:18:21.118962 (+129277us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.059897 (+940935us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b18002ddf60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:22.060074 (+   177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:22.060126 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:22.060180 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.060254 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.060379 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":935815,"prepare.queue_time_us":21577,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":946277,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":213,"spinlock_wait_cycles":348374528,"thread_start_us":124582,"threads_started":1,"wal-append.queue_time_us":617663}
I20260709 12:18:22.005208 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2007) took 1076 ms. Trace:
I20260709 12:18:22.067742 31968 rpcz_store.cc:276] 0709 12:18:20.929024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.977181 (+1048157us) service_pool.cc:224] Handling call
0709 12:18:22.005126 (+ 27945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:21.994195  2880 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.036607  2805 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:22.037793  2914 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.062870 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:22.100136 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 561) took 1290 ms. Trace:
I20260709 12:18:22.063789  2926 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.064196  2860 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:22.106302  2927 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:22.106983  2927 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.107982  2860 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.108440  2860 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:22.108922  2927 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.067143 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:22.110625 32045 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:22.111145 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1780) took 1092 ms. Trace:
I20260709 12:18:22.111434 32045 rpcz_store.cc:276] 0709 12:18:21.019202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.110005 (+1090803us) service_pool.cc:224] Handling call
0709 12:18:22.110293 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.110922 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.111093 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.112668 32045 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:22.112702  2742 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.114025 32045 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:22.046167 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2026) took 1154 ms. Trace:
I20260709 12:18:22.115545 32121 rpcz_store.cc:276] 0709 12:18:20.891393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.974947 (+ 83554us) service_pool.cc:224] Handling call
0709 12:18:22.045899 (+1070952us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002b66d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:22.046084 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395696384}
I20260709 12:18:22.074620 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:22.117096 32046 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:22.074641 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 102936us
I20260709 12:18:22.063544  2732 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:22.103704 32119 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:22.104074 32651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.077s	sys 0.130s
W20260709 12:18:22.120853 32651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.077s	sys 0.130s
I20260709 12:18:22.123147 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:22.124678 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:22.125362 32121 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:22.127038  2732 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:22.127525  2732 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:22.127599 32660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 553) took 2613 ms (client timeout 1931 ms). Trace:
I20260709 12:18:22.127650  2805 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:22.128400  2805 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:22.128571  2732 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.100750 32659 rpcz_store.cc:276] 0709 12:18:20.809256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.809403 (+   147us) service_pool.cc:224] Handling call
0709 12:18:20.809691 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:20.821443 (+ 11752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.821480 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:20.825501 (+  4021us) log.cc:844] Serialized 26 byte log entry
0709 12:18:21.282835 (+457334us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:21.282886 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.282969 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:21.996491 (+713522us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:22.067570 (+ 71079us) raft_consensus.cc:1685] finished
0709 12:18:22.067594 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.100058 (+ 32464us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16020,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12345,"spinlock_wait_cycles":6452608,"thread_start_us":396769,"threads_started":1,"wal-append.queue_time_us":699914}
I20260709 12:18:22.131469 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1776) took 1228 ms. Trace:
W20260709 12:18:22.132247  2897 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.001s
I20260709 12:18:22.133656 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:22.128203 32660 rpcz_store.cc:269] 0709 12:18:19.514529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.514671 (+   142us) service_pool.cc:224] Handling call
0709 12:18:22.127389 (+2612718us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:22.127513 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900467072}
I20260709 12:18:22.002104  2911 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.130096  2732 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.073190  2790 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.120790  2860 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:22.173136  2790 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.132776  2897 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:22.001367 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.20748e+06us
I20260709 12:18:21.999418 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.570627 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46506.7us
I20260709 12:18:22.528904 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:22.638762 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:22.524159 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.605538 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.633205 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:22.670612  2732 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.670497 32041 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11. Re-sending same reply.
I20260709 12:18:22.131970 32044 rpcz_store.cc:276] 0709 12:18:20.902933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.935368 (+ 32435us) service_pool.cc:224] Handling call
0709 12:18:22.131227 (+1195859us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54000578d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:22.131395 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526523264}
I20260709 12:18:22.632930 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:22.679407 32118 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:22.305275  2888 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:22.636581 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 601) took 1527 ms. Trace:
W20260709 12:18:22.682509 32010 connection.cc:256] RPC call timeout handler was delayed by -1.982048106000093s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
W20260709 12:18:22.683220 32010 connection.cc:256] RPC call timeout handler was delayed by -1.9586481060000551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:18:22.313967  2908 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.443s	user 0.019s	sys 0.028s
I20260709 12:18:22.683797 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:22.681005 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1704) took 1828 ms (client timeout 1936 ms). Trace:
W20260709 12:18:22.685006 31967 rpcz_store.cc:269] 0709 12:18:20.852935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.604942 (+1752007us) service_pool.cc:224] Handling call
0709 12:18:22.680861 (+ 75919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.173838  2909 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:22.686301  2909 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:22.687616 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:22.688398  2344 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:22.688856 32044 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:22.332159  2709 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.000s
W20260709 12:18:22.690394  2678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.010s	sys 0.023s
I20260709 12:18:22.326618  2911 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:22.637171 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:22.683917 32010 connection.cc:256] RPC call timeout handler was delayed by -1.8619151450000118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
W20260709 12:18:22.636479 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.567190 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.173925  2935 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.693698  2630 raft_consensus.cc:1081] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:18:22.640813 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:22.697813 32010 connection.cc:256] RPC call timeout handler was delayed by -1.8209481060000599s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
I20260709 12:18:22.693990 31963 raft_consensus.cc:2410] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:18:22.635082 32042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1762) took 2796 ms (client timeout 2064 ms). Trace:
W20260709 12:18:22.690936  2678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.010s	sys 0.023s
I20260709 12:18:22.701041  2840 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.702064 32039 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:22.699810 32042 rpcz_store.cc:269] 0709 12:18:19.838537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.838804 (+   267us) service_pool.cc:224] Handling call
0709 12:18:22.634561 (+2795757us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b540003a054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:22.634809 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609177984}
I20260709 12:18:22.703253 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:22.704062 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 319) took 2945 ms (client timeout 2672 ms). Trace:
W20260709 12:18:22.704512 31963 rpcz_store.cc:269] 0709 12:18:19.758604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.086764 (+328160us) service_pool.cc:224] Handling call
0709 12:18:21.999257 (+1912493us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:18:22.703917 (+704660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4985600}
I20260709 12:18:22.705611 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:22.565479 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1692) took 2776 ms. Trace:
I20260709 12:18:22.318979  2826 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:22.706849 32047 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.706664 31966 rpcz_store.cc:276] 0709 12:18:19.789200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.624258 (+835058us) service_pool.cc:224] Handling call
0709 12:18:20.635876 (+ 11618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.636156 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:20.856605 (+220449us) log.cc:844] Serialized 14 byte log entry
0709 12:18:20.969266 (+112661us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.129334 (+1160068us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b18004098a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:22.563465 (+434131us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b18004098a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:22.564532 (+  1067us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.564786 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:22.564865 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:22.564942 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.565084 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.565283 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17456,"prepare.queue_time_us":1582945,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":5398,"raft.queue_time_us":1840436,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":8312,"spinlock_wait_cycles":14290048,"thread_start_us":306500,"threads_started":2,"wal-append.queue_time_us":146956}
I20260709 12:18:22.577224 32046 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:22.704063 32039 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.642623 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:22.711884 32114 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:22.512110 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.712908 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:22.713235 31968 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1703) took 1879 ms (client timeout 1827 ms). Trace:
W20260709 12:18:22.713702 31968 rpcz_store.cc:269] 0709 12:18:20.833861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.068514 (+1234653us) service_pool.cc:224] Handling call
0709 12:18:22.713126 (+644612us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.714517 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.714896 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.715989 31968 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:22.706028 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.688978  2344 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.717404 31966 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:22.676403 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.703220 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:22.541960 32121 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:22.636876 32654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 656) took 5102 ms (client timeout 1531 ms). Trace:
W20260709 12:18:22.660665 32116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.013s	sys 0.012s
W20260709 12:18:22.716490 31968 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1708) took 1712 ms (client timeout 1972 ms). Trace:
W20260709 12:18:22.719048 31968 rpcz_store.cc:269] 0709 12:18:21.003782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.714517 (+1710735us) service_pool.cc:224] Handling call
0709 12:18:22.716396 (+  1879us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:22.720090 32010 connection.cc:256] RPC call timeout handler was delayed by -1.7471481060001679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
I20260709 12:18:22.717330 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1797) took 1763 ms. Trace:
I20260709 12:18:22.709949 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 324) took 2879 ms. Trace:
I20260709 12:18:22.718796 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:22.706918  2826 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:22.719455  2344 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.717345 31963 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:22.724345 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.724226 32010 connection.cc:256] RPC call timeout handler was delayed by -1.7372481060001519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 12:18:22.726563 32116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.014s	sys 0.012s
W20260709 12:18:22.731462 32010 connection.cc:256] RPC call timeout handler was delayed by -1.6950317929999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:18:22.732143 32010 connection.cc:256] RPC call timeout handler was delayed by -1.651315145000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 12:18:22.730603 32045 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:22.731451 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1705) took 1838 ms (client timeout 2074 ms). Trace:
I20260709 12:18:22.730894 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:22.733793 31963 rpcz_store.cc:269] 0709 12:18:20.890039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.705572 (+1815533us) service_pool.cc:224] Handling call
0709 12:18:22.727991 (+ 22419us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:22.718560 32654 rpcz_store.cc:269] 0709 12:18:17.551555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.556443 (+  4888us) service_pool.cc:224] Handling call
0709 12:18:22.636301 (+5079858us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400323f54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:22.636581 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4333062016}
W20260709 12:18:22.728257 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2005) took 1791 ms (client timeout 2046 ms). Trace:
W20260709 12:18:22.735877 31966 rpcz_store.cc:269] 0709 12:18:20.928438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.710060 (+1781622us) service_pool.cc:224] Handling call
0709 12:18:22.719405 (+  9345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.721539 31970 rpcz_store.cc:276] 0709 12:18:19.830537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.670172 (+839635us) service_pool.cc:224] Handling call
0709 12:18:20.670485 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:21.977683 (+1307198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.709314 (+731631us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001e7dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:22.709702 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364085120}
I20260709 12:18:22.738783  2888 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:18:22.739601 32113 pending_rounds.cc:85] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
I20260709 12:18:22.739948 32113 pending_rounds.cc:107] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:18:22.740226 32113 raft_consensus.cc:2889] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:22.742929 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.743479  2732 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.744297 31968 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
W20260709 12:18:22.745159 31968 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2001) took 1824 ms (client timeout 2184 ms). Trace:
I20260709 12:18:22.747660 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1890) took 2090 ms. Trace:
W20260709 12:18:22.750423 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1999) took 1815 ms (client timeout 1978 ms). Trace:
I20260709 12:18:22.750757 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.751027 31967 rpcz_store.cc:269] 0709 12:18:20.919773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.686033 (+1766260us) service_pool.cc:224] Handling call
0709 12:18:22.735280 (+ 49247us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:22.751586 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1711) took 1157 ms (client timeout 1512 ms). Trace:
W20260709 12:18:22.751997 31963 rpcz_store.cc:269] 0709 12:18:21.594366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.750360 (+1155994us) service_pool.cc:224] Handling call
0709 12:18:22.751532 (+  1172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.753053 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.753878 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1995) took 1835 ms (client timeout 2222 ms). Trace:
I20260709 12:18:22.752925  2826 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:22.362377  2745 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.754967 32044 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:22.755326 32010 connection.cc:256] RPC call timeout handler was delayed by -1.6427481059999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
W20260709 12:18:22.754204 31963 rpcz_store.cc:269] 0709 12:18:20.918839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.752720 (+1833881us) service_pool.cc:224] Handling call
0709 12:18:22.753827 (+  1107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:22.745726 31968 rpcz_store.cc:269] 0709 12:18:20.920213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.728249 (+1808036us) service_pool.cc:224] Handling call
0709 12:18:22.745060 (+ 16811us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:22.766934 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4716339729998253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3509s for a response
I20260709 12:18:22.275055  2805 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:22.682961 32651 rpcz_store.cc:276] 0709 12:18:21.108974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.109184 (+   210us) service_pool.cc:224] Handling call
0709 12:18:21.111954 (+  2770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:21.282549 (+170595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:21.282593 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:21.545091 (+262498us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.121248 (+576157us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:22.121302 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.121397 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:22.635961 (+514564us) raft_consensus.cc:1685] finished
0709 12:18:22.636008 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.636306 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":430140,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"thread_start_us":705611,"threads_started":2,"wal-append.queue_time_us":609385}
I20260709 12:18:22.720618 32040 rpcz_store.cc:276] 0709 12:18:20.953609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.604468 (+650859us) service_pool.cc:224] Handling call
0709 12:18:21.604928 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.064791 (+459863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.064834 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.065374 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.528158 (+462784us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:22.528887 (+   729us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.529284 (+   397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.529446 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:22.716868 (+187422us) raft_consensus.cc:1685] finished
0709 12:18:22.716887 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.717115 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":459371,"prepare.queue_time_us":129005,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":605317,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":383,"thread_start_us":461939,"threads_started":1,"wal-append.queue_time_us":642066}
I20260709 12:18:22.775251  2732 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:22.775507  2826 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.748361 32116 rpcz_store.cc:276] 0709 12:18:20.657017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.813000 (+155983us) service_pool.cc:224] Handling call
0709 12:18:20.813280 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:20.828220 (+ 14940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.828263 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.114641 (+1286378us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:18:22.114984 (+   343us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.746616 (+631632us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:22.746747 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.746927 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:22.747024 (+    97us) raft_consensus.cc:1685] finished
0709 12:18:22.747044 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.747481 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14538,"raft.queue_time_us":22497,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":5360,"spinlock_wait_cycles":5156736,"thread_start_us":613383,"threads_started":1,"wal-append.queue_time_us":562996}
I20260709 12:18:22.779021 32116 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:22.779675 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2025) took 1896 ms. Trace:
I20260709 12:18:22.780023 32116 rpcz_store.cc:276] 0709 12:18:20.883774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.778241 (+1894467us) service_pool.cc:224] Handling call
0709 12:18:22.778726 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.779417 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.779589 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.784485  2745 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:22.787092 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:22.788005 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 373) took 3097 ms. Trace:
W20260709 12:18:22.789796 32117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2010) took 2178 ms (client timeout 2222 ms). Trace:
W20260709 12:18:22.790235 32117 rpcz_store.cc:269] 0709 12:18:20.611141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.667183 (+ 56042us) service_pool.cc:224] Handling call
0709 12:18:22.789386 (+2122203us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b540043a3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:22.789623 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25210752}
I20260709 12:18:22.790956  2696 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:22.791617  2696 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:22.792307 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.793149 31972 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:22.742995  2709 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:22.797780  2709 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:22.788374 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1707) took 1824 ms (client timeout 2222 ms). Trace:
W20260709 12:18:22.799292 31970 rpcz_store.cc:269] 0709 12:18:20.964172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.771105 (+1806933us) service_pool.cc:224] Handling call
0709 12:18:22.788290 (+ 17185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.794195 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:22.809742 32041 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:22.788559 32118 rpcz_store.cc:276] 0709 12:18:19.690488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.691262 (+   774us) service_pool.cc:224] Handling call
0709 12:18:21.402964 (+1711702us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:18:21.403246 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.680026 (+1276780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.787618 (+107592us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002b57d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:22.787905 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2936023552}
I20260709 12:18:22.781347  2909 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:22.819397 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:22.793519  2696 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:22.828608 31972 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:22.829216 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:22.838398  2678 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:22.894593 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1998) took 1975 ms. Trace:
I20260709 12:18:22.895104 31969 rpcz_store.cc:276] 0709 12:18:20.919537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.818766 (+1899229us) service_pool.cc:224] Handling call
0709 12:18:22.894526 (+ 75760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:22.814780  2858 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:22.884387 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1698) took 2902 ms. Trace:
I20260709 12:18:22.906013 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.920866 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1852055340000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 12:18:22.924533 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.908401 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 79450.1us
I20260709 12:18:22.928257 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:22.931704 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1450481060001039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2791s for a response
W20260709 12:18:22.932200 32010 connection.cc:256] RPC call timeout handler was delayed by -1.0214754120001999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
I20260709 12:18:22.922109 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:22.958742 31972 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:22.964622  2742 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:23.110499 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.907614 31971 rpcz_store.cc:276] 0709 12:18:19.910032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.755570 (+845538us) service_pool.cc:224] Handling call
0709 12:18:20.780291 (+ 24721us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:20.789014 (+  8723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.789056 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:20.793787 (+  4731us) log.cc:844] Serialized 52 byte log entry
0709 12:18:20.857412 (+ 63625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:21.004769 (+147357us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.735422 (+1730653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.770630 (+ 35208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:22.810220 (+ 39590us) raft_consensus.cc:1685] finished
0709 12:18:22.810240 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.811921 (+  1681us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1587399,"prepare.queue_time_us":104283,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":1700167,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":22854,"thread_start_us":271149,"threads_started":3,"wal-append.queue_time_us":54313,"wal-append.run_cpu_time_us":3248,"wal-append.run_wall_time_us":1308153}
I20260709 12:18:23.145766 32039 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
W20260709 12:18:23.143265 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 86719us
I20260709 12:18:23.144107 31970 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:23.223923 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1997) took 2304 ms (client timeout 2907 ms). Trace:
W20260709 12:18:23.224526 31970 rpcz_store.cc:269] 0709 12:18:20.919300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.821828 (+1902528us) service_pool.cc:224] Handling call
0709 12:18:23.223849 (+402021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.200008 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1796) took 2251 ms. Trace:
I20260709 12:18:23.199837 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:23.056819 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1706) took 2116 ms (client timeout 2323 ms). Trace:
W20260709 12:18:23.241734 31966 rpcz_store.cc:269] 0709 12:18:20.930926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.802646 (+1871720us) service_pool.cc:224] Handling call
0709 12:18:23.046335 (+243689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.257179 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:23.259037 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.259971 31966 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:23.260553 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2004) took 2332 ms. Trace:
I20260709 12:18:23.260924 31966 rpcz_store.cc:276] 0709 12:18:20.928103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.258463 (+2330360us) service_pool.cc:224] Handling call
0709 12:18:23.260465 (+  2002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.010241  2840 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.202387  2696 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.964782  2549 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.272212 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.196134  2913 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:23.266857 32084 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.512s (SENT) [suppressed 39 similar messages]
I20260709 12:18:22.920593  2888 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.347s
I20260709 12:18:23.272115 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.228909 32038 rpcz_store.cc:276] 0709 12:18:20.948373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.594543 (+646170us) service_pool.cc:224] Handling call
0709 12:18:21.606548 (+ 12005us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.525225 (+918677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.525324 (+    99us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.526731 (+  1407us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.721940 (+195209us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:22.721982 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.723472 (+  1490us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.199517 (+476045us) raft_consensus.cc:1685] finished
0709 12:18:23.199544 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.199797 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":458129,"raft.queue_time_us":459146,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":71,"thread_start_us":654248,"threads_started":2,"wal-append.queue_time_us":484706}
I20260709 12:18:23.275049  2913 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.280733  2913 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:23.295481 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.296096 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.299731 31938 connection.cc:256] RPC call timeout handler was delayed by -1.3245702750000419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 12:18:23.300289 31938 connection.cc:256] RPC call timeout handler was delayed by -1.3002702750000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
I20260709 12:18:23.300355 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:23.300738 31938 connection.cc:256] RPC call timeout handler was delayed by -1.1805702750000364s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 12:18:23.301187 32654 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:23.301367  2742 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.301976 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:23.301187 31938 connection.cc:256] RPC call timeout handler was delayed by -1.0500702750000528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
I20260709 12:18:22.957458  2678 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.274456  2909 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:23.300356 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 2.32959e+06us
I20260709 12:18:22.991266  2732 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.307s
I20260709 12:18:23.262010 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:23.321933 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:23.315026 32046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.010s	sys 0.003s
W20260709 12:18:23.323271 32046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.010s	sys 0.003s
I20260709 12:18:23.322999 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:23.324416 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68186.9us
I20260709 12:18:23.325287 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:23.326049 32113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.002s	sys 0.018s
W20260709 12:18:23.326400 32113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.002s	sys 0.018s
I20260709 12:18:23.326695 31970 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:23.271672  2840 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:23.332404 32045 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.004s
I20260709 12:18:23.332886 32045 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:23.330415 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1994) took 2411 ms. Trace:
I20260709 12:18:23.338721 31970 rpcz_store.cc:276] 0709 12:18:20.918558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.265497 (+2346939us) service_pool.cc:224] Handling call
0709 12:18:23.330343 (+ 64846us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.341240 32660 raft_consensus.cc:2410] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:23.337275 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2003) took 2409 ms. Trace:
I20260709 12:18:23.341853 31969 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:23.341809 31971 rpcz_store.cc:276] 0709 12:18:20.927517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.263090 (+2335573us) service_pool.cc:224] Handling call
0709 12:18:23.337220 (+ 74130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.342527 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2002) took 2422 ms. Trace:
I20260709 12:18:23.342887 31969 rpcz_store.cc:276] 0709 12:18:20.920444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.140945 (+2220501us) service_pool.cc:224] Handling call
0709 12:18:23.342462 (+201517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.011073  2911 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.341262  2709 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.365757 31965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.007s	sys 0.032s
W20260709 12:18:23.366910 31965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.008s	sys 0.032s
I20260709 12:18:23.365216 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:23.367800 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.366729 32651 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:18:23.367391 31966 raft_consensus.cc:2376] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10. Re-sending same reply.
I20260709 12:18:23.366935 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.372964 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.386901  2709 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:23.389024 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.389750  2709 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:23.390127 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.390766 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:23.391135 31966 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:23.391669 32657 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 4: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:23.392036 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.392788 32042 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:23.393261 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.393944 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:23.369982 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:23.394973 32038 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.366127  2800 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.394039 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2006) took 2465 ms. Trace:
I20260709 12:18:23.394523 31971 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:23.397977 31966 rpcz_store.cc:276] 0709 12:18:20.928733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.392971 (+2464238us) service_pool.cc:224] Handling call
0709 12:18:23.393983 (+  1012us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:23.377784  2740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.015s	sys 0.028s
I20260709 12:18:23.425323 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:23.425539 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:23.426465 32039 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:23.426771 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:23.427614 32658 raft_consensus.cc:2410] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:23.427469 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:23.428385 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:23.428547 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:23.429076 32044 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:23.430222 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:23.431006  2855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.006s	sys 0.017s
W20260709 12:18:23.431427  2855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.006s	sys 0.017s
W20260709 12:18:23.434027 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.000s	sys 0.014s
W20260709 12:18:23.434399 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.001s	sys 0.014s
W20260709 12:18:23.443248 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53484.1us
I20260709 12:18:23.366437  2549 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.374344  2946 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.435631 32120 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.004s	sys 0.008s
W20260709 12:18:23.458470 32120 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.004s	sys 0.009s
I20260709 12:18:23.460575  2712 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:23.461266  2712 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:23.462371  2807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.015s	sys 0.010s
W20260709 12:18:23.462801 32112 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.007s	sys 0.011s
I20260709 12:18:23.462749 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1702) took 2641 ms. Trace:
I20260709 12:18:23.457283 32659 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:23.463094  2805 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:23.463747  2712 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:23.434732  2803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.003s	sys 0.010s
I20260709 12:18:23.464776 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1701) took 2657 ms. Trace:
W20260709 12:18:23.465171  2803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.003s	sys 0.010s
I20260709 12:18:23.465230 31970 rpcz_store.cc:276] 0709 12:18:20.807638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.398610 (+2590972us) service_pool.cc:224] Handling call
0709 12:18:23.464164 (+ 65554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.464406 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.464433 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.464495 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:23.464529 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.464591 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.464703 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.466457 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.466718 32040 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:18:23.466919 32047 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.759s	user 0.007s	sys 0.004s
I20260709 12:18:23.467325 32047 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:18:23.462775  2807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.015s	sys 0.010s
I20260709 12:18:23.469359 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.379914  2911 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.470945 31966 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:23.471525 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1709) took 1918 ms. Trace:
I20260709 12:18:23.471793 31966 rpcz_store.cc:276] 0709 12:18:21.553484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.421139 (+1867655us) service_pool.cc:224] Handling call
0709 12:18:23.470650 (+ 49511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.471318 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.471480 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.472764  2911 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.460583 32042 raft_consensus.cc:2376] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8. Re-sending same reply.
W20260709 12:18:23.473093 32121 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.001s
W20260709 12:18:23.463253 32112 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.008s	sys 0.012s
I20260709 12:18:23.476227 32121 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:18:23.463284 31971 rpcz_store.cc:276] 0709 12:18:20.821570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.432456 (+2610886us) service_pool.cc:224] Handling call
0709 12:18:23.462078 (+ 29622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.462391 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.462424 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.462476 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:23.462511 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.462567 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.462676 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.473672 31969 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
W20260709 12:18:23.433487  2740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.016s	sys 0.028s
I20260709 12:18:23.474411 32045 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.476392 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:23.477488 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81337.8us
I20260709 12:18:23.479890  2946 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.483081  2711 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.133s
I20260709 12:18:23.472947  2732 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:23.488090 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:23.487428 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:23.494836  2549 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:23.497110 31971 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:23.502825 32043 raft_consensus.cc:2368] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13.
I20260709 12:18:23.505199  2712 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:23.496958 32658 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
W20260709 12:18:23.506484 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62137.5us
I20260709 12:18:23.506764  2732 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:23.507243 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1875) took 4422 ms. Trace:
I20260709 12:18:23.507315 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:23.508538  2732 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:23.508670 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:23.509049  2732 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:23.510617  2881 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:23.511400  2881 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.515885  2954 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.506505  2689 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:18:23.521795 31972 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:23.505869  2712 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.522614 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1710) took 1952 ms. Trace:
I20260709 12:18:23.499281  2740 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.527747 31972 rpcz_store.cc:276] 0709 12:18:21.570496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.454888 (+1884392us) service_pool.cc:224] Handling call
0709 12:18:23.492346 (+ 37458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.522296 (+ 29950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.522550 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.522891  2630 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.543922  2962 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 12:18:23.546181  2712 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.546890  2712 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.547282  2712 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:23.547690  2712 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.550472  2630 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.553925  2630 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.559285  2952 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:23.566455 32659 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:23.573833 31970 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:23.574894 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1713) took 1907 ms. Trace:
I20260709 12:18:23.575445 31970 rpcz_store.cc:276] 0709 12:18:21.667483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.477381 (+1809898us) service_pool.cc:224] Handling call
0709 12:18:23.528754 (+ 51373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.574568 (+ 45814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.574824 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.535543  2953 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.541633  2800 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.611840  2840 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:23.513648  2732 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.601910 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:23.676635 32118 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.598775  2954 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.594022  2712 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.592710  2953 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.604122 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1794) took 1044 ms. Trace:
W20260709 12:18:23.616148 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.689317 32041 rpcz_store.cc:276] 0709 12:18:22.559848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.680341 (+120493us) service_pool.cc:224] Handling call
0709 12:18:23.603878 (+923537us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001a63d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:23.604050 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262530944}
I20260709 12:18:23.609886 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:23.602869 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63929.8us
I20260709 12:18:23.614733  2742 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.698895  2946 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.699996  2954 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:23.700548 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 90885.8us
W20260709 12:18:23.701947  2961 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.117s	user 0.004s	sys 0.000s
I20260709 12:18:23.703081  2740 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:23.703467 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.701889  2712 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.701869 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:23.709936 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1791) took 1160 ms. Trace:
I20260709 12:18:23.692899  2953 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:23.711040 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 200721us
I20260709 12:18:23.710381 32046 rpcz_store.cc:276] 0709 12:18:22.546727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.711020 (+164293us) service_pool.cc:224] Handling call
0709 12:18:22.711491 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.711954 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:22.712255 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.706730 (+994475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.706784 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.706834 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.706911 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.707022 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7878,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":33468,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":14414,"thread_start_us":988037,"threads_started":2,"wal-append.queue_time_us":744683}
I20260709 12:18:23.708570  2732 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:23.725140 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 348829us
W20260709 12:18:23.732336 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.718458  2946 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:23.740708 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:23.725939  2712 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.744745 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:23.745214 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:23.743979 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:23.748636 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:23.750667 32659 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:23.748566 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:23.759032 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 289789us
I20260709 12:18:23.759501 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:23.759881 32046 raft_consensus.cc:2376] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10. Re-sending same reply.
W20260709 12:18:23.760869 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.751468 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:23.763073 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44479.9us
I20260709 12:18:23.763994  2540 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:23.764631 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.748917  2946 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:23.765532  2946 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.765911  2946 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.765969 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:23.766749 31968 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.767719  2973 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:23.768165  2973 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:23.768546  2973 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.762327  2689 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 12:18:23.749310  2967 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:18:23.743552  2740 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:23.775353 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 266765us
I20260709 12:18:23.764644  2540 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.776943 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.778460 32659 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:23.779331  2344 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.018s	user 0.009s	sys 0.004s
I20260709 12:18:23.777081 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:23.779699  2344 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.780439 31963 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:23.780745 31964 raft_consensus.cc:2368] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:23.743664  2549 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.782974 32660 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:23.772908 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1891) took 3108 ms. Trace:
I20260709 12:18:23.788352 32118 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:23.790470  2742 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:23.791548 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.74558e+06us
I20260709 12:18:23.791666  2968 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:23.792223 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.792142  2968 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:23.793301  2968 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.794694  2968 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:23.795111  2712 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:23.797277 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57968.4us
I20260709 12:18:23.802515 32045 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:23.802017 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2022) took 3005 ms. Trace:
I20260709 12:18:23.805625 31968 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:23.809468 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.813567  2803 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.814141  2803 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.814515  2803 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.817384  2979 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.811815  2630 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:23.813630  2711 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.818603  2956 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.180s	user 0.001s	sys 0.001s
I20260709 12:18:23.818455  2978 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:18:23.818954  2711 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.820024  2978 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:23.820987  2711 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:23.821141  2678 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:23.821755  2977 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:23.822144  2977 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:23.822220  2800 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:23.806694  2807 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:23.823084  2977 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.817943  2979 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.825814  2979 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.826572 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:23.827453  2909 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.511s	user 0.005s	sys 0.005s
I20260709 12:18:23.827823 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:23.828207 32114 raft_consensus.cc:2410] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:23.828552 32118 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:23.829367 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.830084 32118 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:23.830137 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.830646  2549 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.831516 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:23.800720  2740 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.834928 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.838433  2540 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:23.785203 32120 rpcz_store.cc:276] 0709 12:18:20.664394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.827425 (+163031us) service_pool.cc:224] Handling call
0709 12:18:20.838215 (+ 10790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.838484 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:23.473885 (+2635401us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.734500 (+260615us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.740202 (+  5702us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:23.743158 (+  2956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.745372 (+  2214us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.772852 (+ 27480us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1855019,"prepare.queue_time_us":1862516,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":82,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":164,"thread_start_us":978522,"threads_started":2}
W20260709 12:18:23.853091 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 384) took 1846 ms (client timeout 1900 ms). Trace:
W20260709 12:18:23.853482 32121 rpcz_store.cc:269] 0709 12:18:22.007165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.126061 (+118896us) service_pool.cc:224] Handling call
0709 12:18:23.852830 (+1726769us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b540010ead4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:23.853009 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936407808}
W20260709 12:18:23.858974 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 118837us
I20260709 12:18:23.862190 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:23.862778  2888 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:23.864529 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:23.865056  2888 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:23.866427  2909 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.866802  2909 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.867142  2909 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.870489  2888 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:23.871558  2888 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.875257  2855 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:23.876401  2807 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:23.876958  2807 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:23.878216  2807 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.879611  2807 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.894812  2712 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.896090 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2032) took 1779 ms. Trace:
I20260709 12:18:23.896600 32114 rpcz_store.cc:276] 0709 12:18:22.117146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.713971 (+596825us) service_pool.cc:224] Handling call
0709 12:18:23.895900 (+1181929us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400439ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:23.896022 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540471552}
I20260709 12:18:23.850172  2858 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:23.901911  2549 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:23.903355  2740 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:23.904304 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 117670us
I20260709 12:18:23.905236 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.906466 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:23.901942  2858 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:23.907570 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61273us
I20260709 12:18:23.910727 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:23.911326 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.911881  2678 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:23.912122 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.912581  2678 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.805079 32112 rpcz_store.cc:276] 0709 12:18:20.796939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.985057 (+1188118us) service_pool.cc:224] Handling call
0709 12:18:21.985347 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:21.985564 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:23.479880 (+1494316us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.751274 (+271394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.774207 (+ 22933us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:23.788689 (+ 14482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.788773 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.801947 (+ 13174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":676275,"prepare.queue_time_us":715090,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":71,"raft.queue_time_us":30173,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":147,"thread_start_us":1039704,"threads_started":2,"wal-append.queue_time_us":322058}
I20260709 12:18:23.858641 32117 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
W20260709 12:18:23.846832  2696 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.001s
I20260709 12:18:23.914291  2973 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:23.933492  2973 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:23.935500  2800 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.936120  2800 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:23.939200 31964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 1996) took 3020 ms (client timeout 1829 ms). Trace:
W20260709 12:18:23.939718 31964 rpcz_store.cc:269] 0709 12:18:20.919081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.566546 (+1647465us) service_pool.cc:224] Handling call
0709 12:18:23.938949 (+1372403us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001ea0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:23.939102 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501077760}
I20260709 12:18:23.915174 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.941677 32117 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:23.942065 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1896) took 1048 ms. Trace:
I20260709 12:18:23.942365 32117 rpcz_store.cc:276] 0709 12:18:22.893528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.914926 (+1021398us) service_pool.cc:224] Handling call
0709 12:18:23.941996 (+ 27070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.920655  2979 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:23.943374 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:23.943468  2979 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:23.896029 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:23.944584  2979 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:23.945113 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49180us
I20260709 12:18:23.945556  2344 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:23.946127  2689 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.946938 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:23.950412  2740 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:23.897917 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:23.939260 31966 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:23.930884 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74594.5us
I20260709 12:18:23.924917 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.914358 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.918406 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.958074 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1898) took 1008 ms. Trace:
I20260709 12:18:23.958473 32121 rpcz_store.cc:276] 0709 12:18:22.950045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.918113 (+968068us) service_pool.cc:224] Handling call
0709 12:18:23.958017 (+ 39904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.925275 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:23.882241  2909 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:23.922776 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:23.957623  2913 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.611s	user 0.004s	sys 0.004s
I20260709 12:18:23.994786 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.997622 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:23.507702 32115 rpcz_store.cc:276] 0709 12:18:19.085199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.086872 (+  1673us) service_pool.cc:224] Handling call
0709 12:18:19.087186 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.087364 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:19.087877 (+   513us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.591224 (+503347us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.643968 (+3052744us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b1800225d00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:18:22.644510 (+   542us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.481810 (+837300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.492712 (+ 10902us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:23.506795 (+ 14083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.506908 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.507161 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3116267,"prepare.queue_time_us":3112063,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":114,"raft.queue_time_us":176147,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":11061,"spinlock_wait_cycles":16438400,"thread_start_us":1133963,"threads_started":3,"wal-append.queue_time_us":1316474,"wal-append.run_cpu_time_us":1323,"wal-append.run_wall_time_us":1001373}
I20260709 12:18:24.000571 32046 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:18:24.003197 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.003350 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:23.996872 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:24.010776 32114 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:24.012816 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:24.014143  2969 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.257s	user 0.003s	sys 0.001s
W20260709 12:18:24.015113 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 150787us
W20260709 12:18:24.015645 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67381.9us
I20260709 12:18:24.016306 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:24.022923 32042 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:18:24.023239 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.023871 32047 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:23.946245  2344 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:18:24.032181 32117 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:23.918064  2946 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.030563 32043 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:24.031147 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2030) took 1914 ms. Trace:
W20260709 12:18:24.040974 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 115776us
I20260709 12:18:24.042375 32121 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:24.033952 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:24.029995 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 107294us
I20260709 12:18:24.044934 32038 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:24.016937  2973 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.051457 32115 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:24.041392 32116 rpcz_store.cc:276] 0709 12:18:22.116330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.780913 (+664583us) service_pool.cc:224] Handling call
0709 12:18:22.781394 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:22.908106 (+126712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.908149 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:23.194234 (+286085us) log.cc:844] Serialized 52 byte log entry
0709 12:18:23.532532 (+338298us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:23.532573 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.532634 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.938151 (+405517us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b24001f6508. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f228912109d 00007f228bd5310e 00007f228bd6a88c 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221
0709 12:18:24.017603 (+ 79452us) raft_consensus.cc:1685] finished
0709 12:18:24.017623 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.031067 (+ 13444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63401,"raft.queue_time_us":103170,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":104626,"spinlock_wait_cycles":3618560,"thread_start_us":233541,"threads_started":1,"wal-append.queue_time_us":420442}
I20260709 12:18:24.030828  2740 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.058142 32117 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:24.058593 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 1897) took 1149 ms. Trace:
I20260709 12:18:23.991618  2953 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.058945 32115 rpcz_store.cc:276] 0709 12:18:22.909624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.003894 (+1094270us) service_pool.cc:224] Handling call
0709 12:18:24.058539 (+ 54645us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.032260  2696 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.061767 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1899) took 1073 ms. Trace:
I20260709 12:18:24.063295 32117 rpcz_store.cc:276] 0709 12:18:22.985942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.050753 (+1064811us) service_pool.cc:224] Handling call
0709 12:18:24.057901 (+  7148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.058564 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.058730 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:24.063977 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.999625  2913 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.065506 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.065922  2913 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:24.032413 32120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:24.066985 32120 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:24.062098  2858 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.067934 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37262 (request call id 11) took 1057 ms. Trace:
I20260709 12:18:24.068387 32120 rpcz_store.cc:276] 0709 12:18:23.039570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.000463 (+960893us) service_pool.cc:224] Handling call
0709 12:18:24.067839 (+ 67376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.069260  2858 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:24.069751 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.069979 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.071717  2953 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:24.072732 32038 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:24.066069 32117 raft_consensus.cc:2376] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4. Re-sending same reply.
I20260709 12:18:24.072656 32118 raft_consensus.cc:2368] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:24.025708  2689 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:18:24.093446 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2024) took 3249 ms. Trace:
I20260709 12:18:24.093585 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.094013  2689 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:24.094483 32114 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:18:24.051126  2826 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:24.094738 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.095084  2826 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.095295 32046 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.095541  2826 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:24.096046 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.096568 32117 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:24.098549  2946 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:24.055753  2711 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.099673  2800 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:24.100785  2803 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:24.101362 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50481.9us
I20260709 12:18:24.102958 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:24.105012 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:24.105566 32654 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.093837 32119 rpcz_store.cc:276] 0709 12:18:20.843704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.854299 (+ 10595us) service_pool.cc:224] Handling call
0709 12:18:24.086366 (+3232067us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002b52d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:24.093272 (+  6906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2796745600}
I20260709 12:18:24.106783 32046 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:24.107735 32119 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:24.109153 32119 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:24.092579 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 76377.7us
I20260709 12:18:24.103039  2803 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:24.111104 32120 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:18:24.115643 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.109088 32047 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:24.051549  2968 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:18:24.129907  2740 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.141078 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.141455 32119 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:18:24.143632 32047 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:24.057057  2973 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.145323 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.148486 32120 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.128842 32115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47766
I20260709 12:18:24.154587 32115 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:24.154942 32115 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.084722  2696 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.160171 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1716) took 1596 ms. Trace:
I20260709 12:18:24.117762  2913 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:18:24.162849 31969 rpcz_store.cc:276] 0709 12:18:22.564546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.347081 (+782535us) service_pool.cc:224] Handling call
0709 12:18:24.160007 (+812926us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:24.160112 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232224640}
I20260709 12:18:24.163937 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.141815  2711 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:24.146083  2946 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:24.182402  2740 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.184728  2740 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:24.185782  2983 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:24.187841  2977 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:24.188822 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.189635  2709 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.190569 32115 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.192446 32115 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:24.192931 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:47766 (request call id 1903) took 1185 ms. Trace:
I20260709 12:18:24.193222 32115 rpcz_store.cc:276] 0709 12:18:23.007071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.059629 (+1052558us) service_pool.cc:224] Handling call
0709 12:18:24.192823 (+133194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.195041 32117 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:24.196179 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.174798 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:24.176344 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73559.6us
I20260709 12:18:24.178560 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:24.179615  2711 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:24.207770 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.215927 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.218431 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.230593 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.231068 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.231720 32047 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:18:24.232558 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.232924 32115 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:24.233107 32047 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.234227 32115 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:24.234678 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.237618 32115 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:24.237690 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.234803  2689 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.235735  2993 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:24.236256 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.239059  2689 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.239180  2993 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.235859 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.240288 32044 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:24.241169 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.241461  2689 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:24.238565 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.207544  2946 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.235402 32040 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:24.198945 31963 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11. Re-sending same reply.
I20260709 12:18:24.241945 32044 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:24.238300 32043 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.246820 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:24.247345 32660 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.207122 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:24.234145  2958 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.769s	user 0.006s	sys 0.000s
W20260709 12:18:24.255314 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 208401us
W20260709 12:18:24.256167 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.256529 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.257740 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.259728 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:24.260614 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.261623 32117 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:24.262295 32047 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:24.262609 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:24.263182 31970 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.264231 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:24.267122 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:24.268311 32038 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:24.269310 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:24.270512 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.252813 32120 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:24.272819 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.164835  2840 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:18:24.190155  2709 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.276275 32047 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:24.275661 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:24.276747 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:24.283558 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 116283us
I20260709 12:18:24.283977 31963 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:24.276953 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.273403 32040 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:24.241492  2993 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:24.242604  2946 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:24.286743 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.290409 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.291674 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.286396 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:24.294553 32656 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.302778 32038 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:24.304733 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:24.265431  2909 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:24.306943 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 162440us
I20260709 12:18:24.312495 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:24.277356  2732 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.322887 32040 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:24.326833 32120 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:24.327406 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.327752 32039 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:24.314502  2678 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.338498 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:24.346391 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.354462  2689 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:24.362788  2976 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:24.355230 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:24.361907 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.330576  2953 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:24.377461 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.380213  2630 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:24.380722 32040 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.380776  2630 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.382807  2952 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:24.383204  2952 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:24.383513  2709 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:24.384130  2952 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.367815 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:24.391559  2946 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:24.392004  2952 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.395641 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1895) took 1512 ms. Trace:
I20260709 12:18:24.395993 32114 rpcz_store.cc:276] 0709 12:18:22.883728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.011599 (+1127871us) service_pool.cc:224] Handling call
0709 12:18:24.094147 (+ 82548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:24.094909 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.395428 (+300519us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b54002b52d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:24.395563 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188752256}
I20260709 12:18:24.397135 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.397658 32114 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.401014 32656 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:24.403148 32040 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:24.404276 32659 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:24.360488  2909 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:24.346800  2971 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:24.366770 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.350868  2732 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.362444  2897 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:24.355959  2973 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:24.338148 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.359920 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.362775  2881 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:24.361141  2805 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:18:24.376214 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 126598us
I20260709 12:18:24.378633  2994 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:24.379451  2976 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.379256  2826 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.488579 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1712) took 2882 ms. Trace:
I20260709 12:18:24.388929 32660 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:24.388659  2630 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.488972 31966 rpcz_store.cc:276] 0709 12:18:21.602834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.472520 (+1869686us) service_pool.cc:224] Handling call
0709 12:18:23.939082 (+466562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.953751 (+ 14669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.485249 (+531498us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54001ea0d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:24.485387 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072142208}
I20260709 12:18:24.380172  2997 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:24.325539  2993 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:24.380690  2953 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.391702 32653 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:24.355407 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1691) took 4567 ms. Trace:
I20260709 12:18:24.419414 32655 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:24.420074 32658 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:24.420526 32654 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:24.422890 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.450109 32043 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:24.470607 32120 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:18:24.479406 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 119533us
I20260709 12:18:24.481918 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:24.481420 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 158043us
I20260709 12:18:24.487725 32038 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 12 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:24.379859 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:24.490388 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:24.563398 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 250946us
W20260709 12:18:24.563609 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 73375.8us
I20260709 12:18:24.552414 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:24.538236  3001 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 12:18:24.547545 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.567363 32045 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:24.515522 31970 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:24.539317 32040 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:24.497504  2696 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:24.538841 32657 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 12:18:24.547631 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:24.571059 32656 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:24.571347 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.489496  2973 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:24.547217 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:24.486133  2994 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:24.496155 31965 rpcz_store.cc:276] 0709 12:18:19.788265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.623159 (+834894us) service_pool.cc:224] Handling call
0709 12:18:20.623428 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:20.629157 (+  5729us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:22.581148 (+1951991us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b18003dada0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:18:22.666350 (+ 85202us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.459785 (+793435us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.486415 (+ 26630us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.512390 (+ 25975us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.512547 (+   157us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.512585 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.512699 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.184014 (+671315us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:24.355151 (+171137us) raft_consensus.cc:1685] finished
0709 12:18:24.355170 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.355323 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88128,"prepare.queue_time_us":330144,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":4686,"raft.queue_time_us":39443,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":51319,"spinlock_wait_cycles":66869888,"thread_start_us":785265,"threads_started":2,"wal-append.queue_time_us":797839}
W20260709 12:18:24.565869 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 84100.6us
I20260709 12:18:24.491894  2997 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:24.497735 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:24.546203  2953 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:24.609522 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:24.610140 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.611786 31967 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:24.612826 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.617094 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:24.618780  3010 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:24.622296  2826 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:24.622713 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 125037us
I20260709 12:18:24.623548 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.625742 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 380) took 3761 ms. Trace:
I20260709 12:18:24.626811  2807 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:24.626118 32113 rpcz_store.cc:276] 0709 12:18:20.864525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.738580 (+1874055us) service_pool.cc:224] Handling call
0709 12:18:22.739115 (+   535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:22.741163 (+  2048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.741192 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:22.741740 (+   548us) log.cc:844] Serialized 52 byte log entry
0709 12:18:23.326776 (+585036us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.326826 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.326899 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.624809 (+1297910us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b24001f6478. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f2289120fb0 00007f228bd97488 00007f228bd97be8 00007f228bd6f398 00007f228bd6a76a 00007f228bd69bb6
0709 12:18:24.625405 (+   596us) raft_consensus.cc:1685] finished
0709 12:18:24.625429 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.625682 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1341,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":237,"spinlock_wait_cycles":32812928,"thread_start_us":584599,"threads_started":1,"wal-append.queue_time_us":796328}
I20260709 12:18:24.627453 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.628168 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:24.629268 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:24.629703  2630 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:24.614806  2952 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:24.630043 31965 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:24.617973  2946 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:24.627552  2807 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:24.570529  2805 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.634579 32114 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
W20260709 12:18:24.632742 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 252962us
I20260709 12:18:24.639967 32654 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:24.573194  2971 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:24.649063 32116 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:24.564860  2993 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:18:24.653604 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2034) took 1047 ms. Trace:
I20260709 12:18:24.654098 32118 rpcz_store.cc:276] 0709 12:18:23.601928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.911853 (+309925us) service_pool.cc:224] Handling call
0709 12:18:24.648724 (+736871us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540043a3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:24.648907 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367394560}
I20260709 12:18:24.654959 31965 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:24.656487 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.657037 32121 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:24.653497 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.644294 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.659216 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.646872 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.572788  2973 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:24.670064 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.628417  2953 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.671545  2954 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:24.671931 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:24.673695  2994 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:24.674327 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 128114us
I20260709 12:18:24.674579  2994 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:24.675824 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:24.676343 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.677076  2994 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:24.677439  2994 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.677651 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:24.678499  2826 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.678582  2976 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:24.678990 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:24.679688  2826 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:24.680063  2826 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:24.680414  2826 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.680513  2976 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:24.681066 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.681497  2979 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:24.645524  2897 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:24.681922  2979 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:24.682197  2897 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.682636  2897 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.682893  2979 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.682901  2696 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:24.684499  2696 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.684908  2696 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.642483  3010 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:24.687273  3010 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.631170  2732 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:24.655604  2881 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:18:24.691004  2732 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:24.691591  2732 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.692055  2732 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.694337 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.695145  2689 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.695614  2689 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:24.696584 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.696521  2689 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.697314 32039 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:24.698194 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.677347 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.651687 32045 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:24.653494 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.651811  2805 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:24.667033  2630 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:24.667618 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:24.631825  2807 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.630290  3000 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 12:18:24.670526  2973 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:24.764794 32656 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:24.764917 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.630446  2952 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:24.670696  2911 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.677853  2994 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:24.763211  2888 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:24.703693 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.677079 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:24.643527 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 95902.6us
I20260709 12:18:24.656098  2971 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:24.761382  2954 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.761641 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.766943 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.706912  2630 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:24.768848 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.654222 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.778805 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.778961  2952 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.782781  2994 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:24.784551 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.766714  2911 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:24.640939  3001 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.787005 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:24.765750  3008 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:18:24.763696 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.770082  2977 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.006s
I20260709 12:18:24.801949 31971 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:24.767714 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.785521  2888 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.653545  2344 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 12:18:24.787529 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.842751 31972 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:24.802441 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:24.853942 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.854230 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.862030 32121 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:24.863389 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:24.864429 32121 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:18:24.868014 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 65635.1us
I20260709 12:18:24.868547 31972 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:24.766125  2826 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.866252 32121 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:24.871654  2993 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.878737 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.879662 32041 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:24.762043  2696 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:24.866950 32042 raft_consensus.cc:2376] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10. Re-sending same reply.
I20260709 12:18:24.872721 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:24.873095 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.834903  2630 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:24.868902 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:24.881394 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.844029  2911 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.844309  2549 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:24.842617  3000 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:24.884142 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.647931  2997 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.891453 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:25.060151  3000 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:25.061347 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:24.970875 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6378320949997942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 12:18:25.062201 32039 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:25.062433  2549 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:25.063799  2549 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.065812  2549 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.067847  2696 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.071122  2826 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.078948  2712 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:25.079430  2712 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:25.080480  2712 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.081702  2712 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:25.084748  2630 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.979296 31968 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:18:24.985517 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.965206 31970 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:25.089540 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:24.962612  2953 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:25.090847  2953 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:24.856484  2888 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:24.894999 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 213977us
W20260709 12:18:25.045534 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 159111us
I20260709 12:18:24.927100 32118 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:24.907832 32044 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:24.919196 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:25.091756 32114 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:25.107635 32041 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.091545 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:24.872756  3001 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:24.924137 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 137204us
W20260709 12:18:24.973081 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 296093us
I20260709 12:18:25.061971  3000 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.119462  3000 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:25.120239  3012 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:25.120765  2979 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.159161 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:25.169605 32045 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:25.170779 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:25.113546 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 279833us
I20260709 12:18:25.091418  2953 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:24.962411  3002 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
W20260709 12:18:25.175758 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.123965  2858 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.183983 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:24.962630  3006 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.387s
I20260709 12:18:25.186188  3001 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.189002  2997 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.120784  3012 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:25.190963  2911 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.193861 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:25.169809 32039 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:25.201819 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.201788 32623 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.547s (SENT)
I20260709 12:18:25.237277 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:25.238262  2826 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:25.239297 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:24.919610  2977 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:25.055459  2954 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:25.092432  2994 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.210435 32118 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:25.211310 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.963021  2971 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.124022  2968 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:18:25.040313  2897 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:24.962692  2805 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:25.274230 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 80446.1us
W20260709 12:18:25.239333 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68293.3us
I20260709 12:18:25.169991  3027 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:18:25.294706 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:25.295517 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:25.296320 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:25.301465 32119 raft_consensus.cc:2410] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:25.296990 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
W20260709 12:18:25.296808  2973 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.000s
W20260709 12:18:25.301337 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 109574us
I20260709 12:18:25.301698 31968 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:25.287166  3035 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:25.303098 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 240550us
I20260709 12:18:25.304105 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1914) took 1153 ms. Trace:
I20260709 12:18:25.306924 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:25.309693 31968 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.307310 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2047) took 1109 ms. Trace:
I20260709 12:18:25.310501 32119 rpcz_store.cc:276] 0709 12:18:24.197936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.230304 (+ 32368us) service_pool.cc:224] Handling call
0709 12:18:25.295383 (+1065079us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:18:25.307251 (+ 11868us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9269504}
I20260709 12:18:25.301992 31969 raft_consensus.cc:606] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:25.302626 32623 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.548s (SENT)
I20260709 12:18:25.295552 31972 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.291936  2858 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:25.290652  2977 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.224263  3025 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:25.307787 32013 leader_election.cc:400] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
W20260709 12:18:25.293828  2732 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.001s
I20260709 12:18:25.309496 32115 rpcz_store.cc:276] 0709 12:18:24.151085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.238416 (+ 87331us) service_pool.cc:224] Handling call
0709 12:18:24.238710 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.241850 (+  3140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.241881 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.242408 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.301532 (+ 59124us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:24.301568 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.301646 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.303828 (+1002182us) raft_consensus.cc:1685] finished
0709 12:18:25.303856 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.304028 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1187,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":17097,"thread_start_us":58740,"threads_started":1,"wal-append.queue_time_us":253511}
I20260709 12:18:25.274711  2630 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.318452 32013 leader_election.cc:403] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:25.317238 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:25.321765 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.324620 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.328639 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:25.329939 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3234600999999202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 12:18:25.305346  2689 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:25.330485 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1029600999997911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 12:18:25.330768 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0498982229999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 12:18:25.330868 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:25.331139 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0300982229998681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 12:18:25.335829  3012 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.329600 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:25.360843 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:25.361725 31964 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:18:25.361853 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.323700  2977 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:25.366055 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73412.4us
I20260709 12:18:25.323451  2344 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:25.308842  2973 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.367192 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:25.371640 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:25.373644 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:25.337996  2840 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:25.377056 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74466us
I20260709 12:18:25.378361 31968 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:25.381908 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.382457  3032 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.365523  3023 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:25.386688  3023 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:25.387506  3023 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.388041 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:25.388698  3023 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.316682  3035 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:25.390717 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.391373 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.572s (SENT)
I20260709 12:18:25.391803 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:25.392023  3035 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.393916  3028 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.346037  2689 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.394604  3028 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.408109 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:25.399827 32114 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:25.327085  2732 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.422251 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:25.401700 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:25.423112 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58671.7us
W20260709 12:18:25.415903 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.406059 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:25.423870 31967 raft_consensus.cc:606] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:25.424285 31967 raft_consensus.cc:614] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:25.406550 31965 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:25.368503  2344 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:25.425561 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:25.412979 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45859.5us
I20260709 12:18:25.429353 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1806) took 1311 ms. Trace:
I20260709 12:18:25.426287 31969 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:25.427058 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.407574  2840 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:25.440858 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:25.441469 32659 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:25.441772  2984 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:25.442909  3037 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:18:25.443665 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.445151 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.445410 32118 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:25.445905  2807 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:25.446499  2807 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:25.446686 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:25.447263 32119 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:25.447412  2807 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.448899  3006 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.449409  3006 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.450930 32653 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:25.451823 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:25.452109  2689 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.452642  3035 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:25.453718  3035 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:25.454092  3035 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.437956 31969 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:25.427690  2968 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:25.454645 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 127601us
I20260709 12:18:25.435292 32046 rpcz_store.cc:276] 0709 12:18:24.117546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.117685 (+   139us) service_pool.cc:224] Handling call
0709 12:18:24.118028 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:24.147956 (+ 29928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.148014 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:24.162437 (+ 14423us) log.cc:844] Serialized 52 byte log entry
0709 12:18:24.243146 (+ 80709us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:24.329951 (+ 86805us) log.cc:844] Serialized 14 byte log entry
0709 12:18:24.554754 (+224803us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.412003 (+857249us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800140ce0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14a733 00007f228c1cfdbf 00007f228c1ce83e
0709 12:18:25.428880 (+ 16877us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b540019bf20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14a733 00007f228c1cfdbf 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c
0709 12:18:25.428956 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.429056 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.429110 (+    54us) raft_consensus.cc:1685] finished
0709 12:18:25.429127 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.429278 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4620,"prepare.queue_time_us":128222,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":24782,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":35864,"spinlock_wait_cycles":37429120,"thread_start_us":341745,"threads_started":3,"wal-append.queue_time_us":147370}
I20260709 12:18:25.363320  2858 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:25.459553 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:25.468143 32656 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:25.468524 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77176.2us
I20260709 12:18:25.469744 32040 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:25.379943  3027 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.399565  2973 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:25.474406 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:25.479897 32010 leader_election.cc:400] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:25.425750  3010 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:25.467748  3035 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.462284  2997 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:18:25.333106  3025 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.535439 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:25.446501  2994 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:18:25.462164  2984 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:25.530529 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68360.5us
I20260709 12:18:25.532330  3028 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.471194  2709 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:25.483474 31969 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.600100  3010 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:25.627086 32041 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:25.526871  2344 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.4, Last appended by leader: 4, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:25.633013 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 161264us
I20260709 12:18:25.634815 32121 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:25.596230 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 113732us
I20260709 12:18:25.635891 31969 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.471864  2952 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 12:18:25.633061 32653 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:25.480356  2549 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.148s
I20260709 12:18:25.468092  2954 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.616852  2911 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.479436  3027 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.664502  2994 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:25.635421  2709 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:25.686486  3010 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.687902  2709 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.689191  2709 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.690750  2344 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.662675 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:25.677421  3045 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:18:25.677951  2984 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.680761  2953 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.684154 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:25.684736  3021 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:25.685186  3033 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
W20260709 12:18:25.685657 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.698515 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:25.709663 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1826) took 1412 ms. Trace:
I20260709 12:18:25.693789  2994 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:25.710096 32043 rpcz_store.cc:276] 0709 12:18:24.296758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.566264 (+269506us) service_pool.cc:224] Handling call
0709 12:18:24.608616 (+ 42352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.631434 (+ 22818us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:24.631744 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.302718 (+670974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.302769 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:25.302812 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.709428 (+406616us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b54001946d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:25.709472 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.709590 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11627,"prepare.queue_time_us":605,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":301197,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":33107,"spinlock_wait_cycles":51922816,"thread_start_us":670566,"threads_started":2,"wal-append.queue_time_us":328196}
I20260709 12:18:25.701676  2807 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:25.718788  2540 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:25.700562  2855 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:18:25.730073 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46043.4us
I20260709 12:18:25.731999 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:25.732614  2855 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.733093 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:25.734027  2855 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:25.734433  2855 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:25.734798  2855 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.707695  2897 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:25.720458  2968 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:25.723052  3025 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.743633 31969 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:25.728972 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 244826us
I20260709 12:18:25.729027 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:25.729431 31972 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:25.718170 32113 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:25.747411  2732 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.748479 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:25.751086 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:25.700121  2858 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.740958  2807 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.728905  2984 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:25.757546 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.233s (SENT)
I20260709 12:18:25.756372  2994 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:25.750798  2888 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.002s
I20260709 12:18:25.728731  3028 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 12:18:25.768707  2988 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.576s	user 0.021s	sys 0.014s
I20260709 12:18:25.768640  2968 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:25.769361  2968 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.770244  2994 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.771442  2968 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.763849  2888 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.776691  2888 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.778559  2953 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:25.778792 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:25.779383 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1733) took 1137 ms. Trace:
I20260709 12:18:25.779506  3033 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.779783 31964 rpcz_store.cc:276] 0709 12:18:24.641962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.642276 (+   314us) service_pool.cc:224] Handling call
0709 12:18:25.779208 (+1136932us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400313b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:25.779325 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214230784}
I20260709 12:18:25.779959  3033 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.780884  2968 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:25.781386 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.781760 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1902) took 2775 ms. Trace:
I20260709 12:18:25.781489  2968 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.780803 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:25.782629 32046 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.783712  2968 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.784224  2968 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:25.779417 32656 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.785892 32041 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [8.4-10.5]
I20260709 12:18:25.758733  2549 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:25.786825  2549 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:25.787489 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:25.788105 32659 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:25.788326 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:25.787735  2549 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.782125 32116 rpcz_store.cc:276] 0709 12:18:23.006770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.054266 (+1047496us) service_pool.cc:224] Handling call
0709 12:18:24.648883 (+594617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.653101 (+  4218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.781584 (+1128483us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b540043a3d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:25.781708 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292054272}
I20260709 12:18:25.796186 32657 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:25.760725  2807 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:25.801391 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.804301 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:25.802042 32046 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:25.805022 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52731.2us
W20260709 12:18:25.805645 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.805971 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.808069 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:25.808837 32112 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:25.807732 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:25.810714 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.765813  2840 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:25.816634 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:25.817436 32042 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.806979  2740 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.818851  2740 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.821995  2740 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.784695  2968 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:25.822786 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:25.829519 32112 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:25.823971 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:25.830456 32046 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:25.826525 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:25.835311 32656 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:25.837672 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.842958 32042 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:25.843856 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.847337 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:25.782153  3033 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:25.849639 32657 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:25.855470 32659 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:18:25.861454 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 113074us
I20260709 12:18:25.864877 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1804) took 1169 ms. Trace:
I20260709 12:18:25.865234 32040 rpcz_store.cc:276] 0709 12:18:24.695180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.761409 (+ 66229us) service_pool.cc:224] Handling call
0709 12:18:24.761749 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.470127 (+708378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.864638 (+394511us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54001225d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:25.864810 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1627698048}
I20260709 12:18:25.866544 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1730) took 1239 ms. Trace:
I20260709 12:18:25.867075 31970 rpcz_store.cc:276] 0709 12:18:24.627023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.643420 (+ 16397us) service_pool.cc:224] Handling call
0709 12:18:24.643725 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.089424 (+445699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.866289 (+776865us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54000f7654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:25.866484 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738520832}
I20260709 12:18:25.868332 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2051) took 1094 ms. Trace:
I20260709 12:18:25.868680 32114 rpcz_store.cc:276] 0709 12:18:24.773603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.177873 (+404270us) service_pool.cc:224] Handling call
0709 12:18:25.178196 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.420442 (+242246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.868138 (+447696us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:25.868279 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509240192}
W20260709 12:18:25.889132 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.895419 32046 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:25.902347 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:25.936388 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 169981us
I20260709 12:18:25.857749  3028 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:25.973374 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.859628  2993 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:25.860742  2897 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:25.984319 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:25.986688 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.989987 32046 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:18:25.990734 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.993331 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 91123.7us
I20260709 12:18:25.995734 32040 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:26.018891 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.023351  3005 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:25.935712  2977 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:25.941339  2800 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:18:25.986267  2993 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:26.049458  2993 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.050653  2993 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.992100  3028 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:26.062722 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.063422 32046 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.015151  2897 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:26.077306 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 126538us
I20260709 12:18:26.026721  3023 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:26.075614 32040 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [8.6-10.7]
W20260709 12:18:26.080008 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.70053e+06us
I20260709 12:18:26.044616  2999 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:26.090219 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71444.7us
I20260709 12:18:26.091254 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.097056  2711 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.103005 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.081305  3023 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:26.129669 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:26.133281  2953 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:26.133761 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:26.137233 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:26.137859 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.138418 32046 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:26.094462  2999 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:26.139854 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48684.8us
I20260709 12:18:26.130757  2800 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:26.143901  2630 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:26.144701 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:26.165227 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:26.136039  3023 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:26.147637  2977 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:26.148756  3035 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.001s
I20260709 12:18:26.149744  3027 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:26.134732  2997 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:26.102334  2968 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:26.155570 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.172174 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.162385  3020 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:26.133797  2953 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:26.145753  2689 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:26.154210  3005 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.130007  2994 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:18:26.138031  2952 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:18:26.164232  2630 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:26.168082  2800 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:26.197918 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.174563  2712 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:26.204769  2712 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:26.205148  3053 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:18:26.205354  2999 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.179387  3012 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:26.209172 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:26.193455  3021 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:26.202907  2994 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:26.197305  3020 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:26.152526  3028 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:26.227681 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55598.9us
I20260709 12:18:26.228673 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.172870  3035 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:26.199589  2971 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.021s	sys 0.008s
I20260709 12:18:26.205188  2712 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.231884  2800 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:26.232975 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.241477  2344 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.002s
I20260709 12:18:26.241081  2630 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.220916  2968 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:26.192129  2897 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.254880  3020 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.258544  2953 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.222827  2689 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:26.262408 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69138.2us
I20260709 12:18:26.211102  3027 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 12:18:26.286538 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 111160us
I20260709 12:18:26.328586 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:26.266981  2984 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.001s
W20260709 12:18:26.293329 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.264638  2911 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.317051 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45274.8us
I20260709 12:18:26.267552  3001 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:26.265700  2971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.021s	sys 0.009s
I20260709 12:18:26.267541  2997 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:26.331737 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.340394 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:26.331987  2978 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:26.353240  3013 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.253s	user 0.000s	sys 0.000s
I20260709 12:18:26.332017  2807 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:26.357162 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.359351  3035 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:26.339030  2630 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:26.361340  2855 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.626s	user 0.005s	sys 0.005s
I20260709 12:18:26.361800  2855 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.365127  3006 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:26.366288 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1822) took 2132 ms. Trace:
I20260709 12:18:26.341360 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:26.369294 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.343343 32084 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:26.344257  3057 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 12:18:26.346120 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1753) took 1046 ms. Trace:
I20260709 12:18:26.371513 31965 rpcz_store.cc:276] 0709 12:18:25.299691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.724455 (+424764us) service_pool.cc:224] Handling call
0709 12:18:25.724804 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.725191 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.725222 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.725704 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.191603 (+465899us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:26.200029 (+  8426us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.212972 (+ 12943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.213075 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.345893 (+132818us) raft_consensus.cc:1685] finished
0709 12:18:26.345914 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.346055 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17188,"prepare.queue_time_us":8514,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":23013,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":20154,"thread_start_us":465436,"threads_started":1,"wal-append.queue_time_us":537085}
I20260709 12:18:26.350975  2977 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.335142  2911 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.366686 32047 rpcz_store.cc:276] 0709 12:18:24.233651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.290496 (+ 56845us) service_pool.cc:224] Handling call
0709 12:18:24.316448 (+ 25952us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:24.559982 (+243534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.560021 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:24.569804 (+  9783us) log.cc:844] Serialized 52 byte log entry
0709 12:18:24.767645 (+197841us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:24.782572 (+ 14927us) log.cc:844] Serialized 14 byte log entry
0709 12:18:24.782780 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.782853 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.737470 (+954617us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b180043efa0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:26.100119 (+362649us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b180043efa0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:26.352605 (+252486us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:26.366026 (+ 13421us) raft_consensus.cc:1685] finished
0709 12:18:26.366057 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.366222 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17029,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":30964,"raft.queue_time_us":53672,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":167,"spinlock_wait_cycles":104882432,"thread_start_us":363728,"threads_started":2,"wal-append.queue_time_us":204851}
I20260709 12:18:26.333473  3001 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:26.377575  2977 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:26.334650  2689 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:26.379359  2711 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:26.386833 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43504.6us
I20260709 12:18:26.389045 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.393244  2689 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:26.397249 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66656.3us
W20260709 12:18:26.397397 32623 connection.cc:256] RPC call timeout handler was delayed by -1.16239936199986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 12:18:26.397984 32623 connection.cc:256] RPC call timeout handler was delayed by -1.0579993619999186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 12:18:26.383873  3001 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.384264  2993 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:26.400714 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.420048 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.421852 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.422466 32040 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:26.422534 32046 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.414069 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:26.414077 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:26.414470 32112 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:26.421432 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.426348 32041 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 12 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:18:26.429597 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.432436 32119 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:26.431553 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.434547 32653 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:26.444268 32113 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.398609  3006 raft_consensus.cc:2764] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:26.446949 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.445348 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:26.449981 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.450938 32114 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:26.398679  3021 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:26.456125 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.461622 32039 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:26.415099 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:26.457773 32043 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:26.462704 32041 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:26.459978 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:26.460716 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:26.463649 32658 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.463873 32657 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:26.400358  3010 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.465883 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:26.403553  2909 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.465096 32660 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:26.405352  3065 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:26.492997 32011 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.493319 32046 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:26.465417  2999 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:26.494587  2999 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:26.494699 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:26.495587  2999 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.496682 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:26.467173 32118 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:26.498584 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:26.499289 32118 raft_consensus.cc:2368] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11.
I20260709 12:18:26.500068 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:26.421695  3023 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.500905 32118 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:26.501472 32047 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:26.467485 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:26.415254  2709 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:26.502622  2709 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.503276  3023 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:26.504200 32653 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:26.505155  2709 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:26.466933  3035 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:18:26.507310  2984 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:26.507750  2984 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.508111  2984 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.494138  2968 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.465775  2971 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:26.514923  2971 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.515383  2971 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.516108 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.517493 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.478842  2911 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:26.478953 32119 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:26.518672  3019 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:26.446295  2344 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 12:18:26.479357 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 90388.3us
I20260709 12:18:26.480049 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.471727 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:26.483006 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65594.5us
I20260709 12:18:26.483911 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.484858 32112 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:26.485805  3010 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.451366  2997 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.487264  2689 raft_consensus.cc:993] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:18:26.488126 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:26.526074 32657 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.526151 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:26.489080 32656 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:26.474769  2909 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:26.489055 32113 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
W20260709 12:18:26.527410 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.456845  3066 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.397605  2712 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.465610  3007 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:18:26.538590 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1923) took 2218 ms. Trace:
I20260709 12:18:26.539042 32121 rpcz_store.cc:276] 0709 12:18:24.320239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.887188 (+566949us) service_pool.cc:224] Handling call
0709 12:18:24.887545 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.648553 (+761008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.538346 (+889793us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b540012b6d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:26.538524 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717897088}
I20260709 12:18:26.539572 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:26.525482 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:26.540984 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:26.412616  3005 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:18:26.527484 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:26.544234 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:26.531023  3065 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:26.537412 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:26.535135 32658 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:26.551091 32118 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:26.473315  2977 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.544021  2953 raft_consensus.cc:1081] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:18:26.537918  3066 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.547536  3028 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:26.570900 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:26.551733  2709 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:26.514219  2630 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:18:26.558419  3065 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:26.602452 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59979us
I20260709 12:18:26.594645 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.596482 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.562094  2711 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:26.573457  3000 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.586745  3028 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.574728  3030 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:18:26.602591  2630 consensus_queue.cc:1243] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:26.579118  2840 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:18:26.611670 32046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:18:26.613168 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.614503 32046 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:26.590588  2978 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:26.602373 31866 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 5 to 6, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 5 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:26.654035 32044 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:26.656020 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.656711 32040 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:26.653715 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57319.4us
I20260709 12:18:26.658006 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:26.611362  3028 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.660130  3010 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:26.660669 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85537.5us
I20260709 12:18:26.607280  2909 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.656544  2805 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.612506  3000 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.663173  3065 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.612397  2711 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:26.697669  2840 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:26.698165  2840 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.698647  2840 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:26.698432  2711 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.699777 32040 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:26.613821  3035 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:26.671554  3071 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:18:26.672397  3070 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:18:26.652575 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.705303 32039 raft_consensus.cc:2368] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:26.677590 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.681347 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.615024 32046 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.702597 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.694077 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:26.706336 32044 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.700860  3001 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.712347  2840 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.712966  3001 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:26.718760 32040 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:26.719550  3000 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:18:26.722545 32044 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:26.723794 32045 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:18:26.732169 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.731523 32657 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:26.733119 32046 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:26.738440 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80490.7us
I20260709 12:18:26.739938 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:26.748293 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45771.1us
I20260709 12:18:26.755050 32045 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:26.758826 32046 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:26.760211 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.760412 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.761106 32113 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.737805  3007 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.771613 32112 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:26.767077 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:26.772142 32010 leader_election.cc:400] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:26.772460 32010 leader_election.cc:403] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:26.751616  2840 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:26.750488  3000 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.761348  2909 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.745812  3023 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.741149  3012 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.770824 32046 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:26.782413 32046 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:26.702072  2805 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:26.775656 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.777534 32113 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:26.791123 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:26.784569  3005 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:26.795056  2709 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:26.789264  3012 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.798772  2855 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:26.800299  3056 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.665s	user 0.000s	sys 0.005s
W20260709 12:18:26.802826 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47010.1us
I20260709 12:18:26.804287 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1754) took 1504 ms. Trace:
I20260709 12:18:26.804689 31966 rpcz_store.cc:276] 0709 12:18:25.299982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.738401 (+438419us) service_pool.cc:224] Handling call
0709 12:18:25.738832 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:25.739273 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.739308 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:25.739954 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 12:18:25.854388 (+114434us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:25.866403 (+ 12015us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.940998 (+ 74595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.941097 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.144713 (+203616us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b18003dace0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:26.799597 (+654884us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:26.804056 (+  4459us) raft_consensus.cc:1685] finished
0709 12:18:26.804081 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.804225 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78956,"prepare.queue_time_us":133983,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":76615,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":25655,"spinlock_wait_cycles":21098496,"thread_start_us":114016,"threads_started":1,"wal-append.queue_time_us":331172}
I20260709 12:18:26.808465 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.795989  2805 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:26.809876 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:26.810187 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.807634  3066 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:26.807816 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.811131 31968 raft_consensus.cc:2368] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8.
I20260709 12:18:26.807533 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.806746 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.799808  2971 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.814617 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.807504 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.813064 31965 raft_consensus.cc:2368] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:26.815331 31964 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:26.814862  3025 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.813084 31963 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:26.798121  3012 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:26.818274 31970 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:26.793103  2968 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:26.820021 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.820130 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.820595 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:26.821084 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.822572 31972 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:26.823871 31967 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:26.818773  3000 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:26.825390  3000 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:26.826026 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:26.826359  3000 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:26.827178 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.830564  2971 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:26.831112 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.832016 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.832299 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.832630 31970 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:26.832973 31972 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:26.833761 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.834503 31965 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:26.834661 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.835256 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.835791 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.836265 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.836452 31972 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:26.836872 31966 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:26.835875 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.790755  2984 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.823675  3025 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.840077  2984 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:26.840463  3025 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:26.842024  3025 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.842594  3025 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:26.843528 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.846153 31972 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:26.846457 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.952071 32041 raft_consensus.cc:2368] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:18:26.848140 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.953313 32040 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:26.953347 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.855113 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.856458 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1837) took 1075 ms. Trace:
I20260709 12:18:26.954553 32041 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:26.955469 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:26.954453 32038 rpcz_store.cc:276] 0709 12:18:25.780708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.789511 (+  8803us) service_pool.cc:224] Handling call
0709 12:18:25.802244 (+ 12733us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:25.815497 (+ 13253us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:25.854897 (+ 39400us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.093027 (+238130us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.095788 (+  2761us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:26.153769 (+ 57981us) log.cc:844] Serialized 52 byte log entry
0709 12:18:26.158434 (+  4665us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.163652 (+  5218us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.163916 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.164011 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.853822 (+689811us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b54000c74a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c144dbe 00007f228c14f930 00007f228c14f8d8 00007f228c14f858 00007f228c14f813 00007f228c14ea9c 00007f229203a2f6 00007f2292034aa8 00007f228c18b7cb 00007f228c18e2c0
0709 12:18:26.856232 (+  2410us) raft_consensus.cc:1685] finished
0709 12:18:26.856258 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.856400 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6633,"prepare.queue_time_us":311008,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":109,"raft.queue_time_us":18340,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":4033,"spinlock_wait_cycles":2929408,"thread_start_us":183327,"threads_started":2,"wal-append.queue_time_us":202571}
I20260709 12:18:26.858893 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.956504 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.956782 31970 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:26.860586 31971 raft_consensus.cc:2368] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
W20260709 12:18:26.960824 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.961289 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.952631 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:26.968327 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:26.968644 31969 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:26.969131 31972 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:18:26.981559 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 124243us
I20260709 12:18:26.981884 31972 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:26.983866 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.985697 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:26.986795 32040 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:26.990741 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:26.991393 31970 raft_consensus.cc:2376] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11. Re-sending same reply.
I20260709 12:18:27.021683 32041 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:27.022441 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:27.023624 31969 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.024209 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:27.026099 32044 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:27.027370 32038 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: [5.8-8.11]
I20260709 12:18:26.988274  3065 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.032469 32040 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:27.038657 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:27.040822 31970 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:27.041137 31963 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:26.856985  3028 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.042716 31968 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:27.050019 31969 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:27.030892  2968 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.056178 31969 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:18:27.111498 31969 pending_rounds.cc:85] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 8
I20260709 12:18:27.111830 31969 pending_rounds.cc:107] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:18:27.112075 31969 raft_consensus.cc:2889] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:27.087283 32623 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.972s (SENT)
I20260709 12:18:27.113226 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:27.091107 31970 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:27.051975  3065 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.095281 31967 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:27.099404  3025 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.099429  2711 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:27.101214  3028 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:27.126394  2855 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:27.110073  2840 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.127420  3001 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:27.129165  2840 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.132699  2840 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.133055  3010 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.133867  3030 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:27.134249  3030 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:27.135375  3030 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.130347 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:27.139716 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52469.6us
W20260709 12:18:27.140410 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.140806 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.122365  2968 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.142117  3010 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.142267  2712 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.142642  3010 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.143620  2968 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.144673 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:27.145107  3010 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.145265 32120 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:27.137733  2855 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:18:27.146183  2805 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:27.147588 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:27.146757  2805 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:27.137099  3030 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:27.130196  2807 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:27.145360  3065 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:27.175875  2888 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:27.179282  2888 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:27.180249 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:27.180133  2888 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.180773  2977 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:27.181103 32045 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:27.181277  2888 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:27.182047 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:27.182660 32040 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:27.175330  2805 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.188249 32657 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:18:27.194284 32043 raft_consensus.cc:1217] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-11.6]   Dedup: 6.4->[8.5-11.6]
I20260709 12:18:27.194978 32043 pending_rounds.cc:85] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 4
I20260709 12:18:27.195266 32043 pending_rounds.cc:107] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:18:27.195556 32043 raft_consensus.cc:2889] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:27.193048 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:27.205202 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:27.205473 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:27.205857 32038 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:27.206387 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:27.207847  3035 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:27.208477  3035 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:27.208861  3035 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:27.203220 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:27.210112 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:27.210589  2709 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:27.210744 32040 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:27.211848  3025 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:27.212225  3025 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.212662  3025 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:27.215600 32045 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:27.216753 32039 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:27.209664  3035 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.217733 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:27.212165  2971 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:18:27.218294 32039 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:27.220206 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:27.229442 32038 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:27.218637 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:27.237918 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:27.236429 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:27.242651  2911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.031s	sys 0.028s
W20260709 12:18:27.247541  2911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.031s	sys 0.028s
I20260709 12:18:27.234623  3065 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.239610  2971 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.241110  3006 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:27.259025 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:27.268888 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.269632 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.371296 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.243274  3005 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:27.250349  2630 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:27.374822 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 115864us
W20260709 12:18:27.375352 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.414978 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.252390  3023 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:27.412045  2976 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:27.419071 31965 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:27.437322 31965 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:27.425124 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:27.441506 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:27.374500  3006 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:27.440644  2549 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:18:27.374200  2978 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.183s
I20260709 12:18:27.437083  2976 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.443478 31965 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.422137  3023 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.440739  2712 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:27.443528  3075 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:18:27.373358  2689 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:18:27.421581  3005 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:27.452232  3012 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:27.448732  2800 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:27.473022  3094 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 12:18:27.473786  3087 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:18:27.481307 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:27.412398  2630 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:27.512657 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 92658.9us
I20260709 12:18:27.390976  2971 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.390518  3065 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.512015  3023 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:27.508752  2976 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:27.550295  2800 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:27.552874  2630 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.542131  2712 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:27.591202 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.582036  2954 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.592460 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:26.659998 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:27.595273 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:27.599344 31866 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 6 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 8 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.603407  2971 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.604174  2800 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.607389 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 689) took 1860 ms. Trace:
I20260709 12:18:27.607784 32651 rpcz_store.cc:276] 0709 12:18:25.747259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.747437 (+   178us) service_pool.cc:224] Handling call
0709 12:18:25.747794 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.387551 (+639757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.387584 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:26.392223 (+  4639us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.387958 (+995735us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b18003dc0c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:18:27.491774 (+103816us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:27.491816 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.491883 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.607081 (+115198us) raft_consensus.cc:1685] finished
0709 12:18:27.607100 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.607316 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":637955,"raft.queue_time_us":641961,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5989,"spinlock_wait_cycles":15667072,"thread_start_us":103619,"threads_started":1,"wal-append.queue_time_us":1167461}
I20260709 12:18:27.612546 31966 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 12. Ops: []
I20260709 12:18:27.587390  3006 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:27.801968 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.772327 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 179939us
I20260709 12:18:27.772867 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:27.689666 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:27.809504 32651 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:27.810403 32657 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:27.740083 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:27.811301 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:27.785859 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.811913 32046 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:27.784993 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 189801us
W20260709 12:18:27.815740 31938 connection.cc:256] RPC call timeout handler was delayed by -1.2175251150001714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:18:27.811501 31971 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:27.818202 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.816504 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:27.809092 31964 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:27.817173 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:27.615595  3097 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 12:18:27.821209 32010 leader_election.cc:400] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 8.
W20260709 12:18:27.827924 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 370066us
I20260709 12:18:27.621510  3019 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:27.674094  3035 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:27.612828  3065 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.619664  2712 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:27.843134 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:27.843523 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.704564  3007 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:27.783370  2911 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.718137  3010 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.839882 31966 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:27.845814 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.624869  3025 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.820942  2984 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.850790 32655 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:27.855242 32042 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:27.834049  3006 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.833762  2840 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:27.852945 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:27.613539  2954 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.860347  3079 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.860900  2984 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.865453  2976 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.834095  2977 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:27.833938  3019 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:27.871730 31966 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:27.822086  3108 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:18:27.891239 32120 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:27.844767  3065 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:27.906551 32655 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:27.844970  3007 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:27.623270  3005 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:27.898813 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83625.8us
I20260709 12:18:27.908381 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:27.846764  3000 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:18:27.902638 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:27.909554 31963 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.909677 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:27.910454 31968 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:27.904279 31971 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:27.851261  3098 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:18:27.852488  3010 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.913745 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:27.908923  2977 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.919581  2954 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:27.920759 31963 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:27.926100 31968 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:27.886577  2911 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:27.907387  2711 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.909258  3065 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.919505  2976 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:27.909461  3007 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.929363  3098 consensus_queue.cc:1243] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:27.899453  3001 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:18:27.958948 31964 raft_consensus.cc:2410] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:27.949452 32654 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:27.899708  3079 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:27.920966  2984 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:27.957397 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:27.911705 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 95291.5us
I20260709 12:18:27.972483 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.000244 32120 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:28.002269  3119 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:28.002897  3119 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.003710 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:28.004258 32038 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.005087  3119 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.006660 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:28.007192 32044 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.008246 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:27.948391  3065 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:27.975078 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.947491 31968 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:18:27.978358 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.942720  2971 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.995014 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:28.035791 31963 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:28.028908 32044 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:28.029253 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:27.929648  3023 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.030594 32038 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:28.066011 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:28.067629 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:28.083261 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 242335us
I20260709 12:18:28.066942 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:28.085677 32623 leader_election.cc:336] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.672s (SENT)
I20260709 12:18:28.086151 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.113147 32114 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:28.131021 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64172.5us
I20260709 12:18:27.950162  2711 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:28.131794 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:28.135404 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:28.136464 31972 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 11 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:18:28.035327  3025 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:28.160000 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 132748us
W20260709 12:18:28.166487 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 126825us
I20260709 12:18:28.082377  2840 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:28.129312  3120 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.205740  3020 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:28.323206 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:28.344211 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:28.386379 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.239187  3025 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:28.404429 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 447080us
I20260709 12:18:28.404500 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.407409 32121 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
W20260709 12:18:28.415584 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.335670  3030 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.385840  3001 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:28.582844 31963 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.027s
I20260709 12:18:28.586273  3030 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.589007 31963 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:28.573199 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 441434us
I20260709 12:18:28.406138  2978 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:28.582001 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 496308us
W20260709 12:18:28.486995 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 319184us
I20260709 12:18:28.594640 31972 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:28.486692 32010 leader_election.cc:400] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:28.608392 32010 leader_election.cc:403] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:28.607226 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:28.609639 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.001s	sys 0.023s
W20260709 12:18:28.610749 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.001s	sys 0.023s
I20260709 12:18:28.611449 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:28.615111 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:28.422344  3118 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:18:28.618777 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:28.466681  3025 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:28.622303 32651 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.008s	sys 0.014s
I20260709 12:18:28.272502  3120 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.628841  3000 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 12:18:28.628460 32651 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.008s	sys 0.014s
I20260709 12:18:28.460340  3007 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:28.388607  3020 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.644907  3025 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.645432  3030 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:28.647475 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:28.650813 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:28.605907  3010 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:28.656977 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:28.675206 31963 raft_consensus.cc:2376] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8. Re-sending same reply.
I20260709 12:18:28.625695  3134 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:28.655196  2911 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:28.674844 31968 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:28.665910  3020 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:28.687997  3103 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.102s	user 0.005s	sys 0.018s
I20260709 12:18:28.688218  3023 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.689859  3007 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:28.690351  2807 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.652031  3079 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:28.625291  2840 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:28.588235  3001 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:28.720460 31972 raft_consensus.cc:2376] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9. Re-sending same reply.
I20260709 12:18:28.727491  3108 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.727948  3108 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:28.728340  3108 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:28.731055 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 570177us
I20260709 12:18:28.731329  3113 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:28.733810  3133 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.734287  3133 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.736423 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:28.736627  3023 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:28.736900  3133 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.736454  3135 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.737620  2971 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:28.737929  3135 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:28.736471 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 331251us
I20260709 12:18:28.741997 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.738152  2971 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.740500  3135 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:28.708873  2630 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.118s	user 0.005s	sys 0.005s
W20260709 12:18:28.747337 32655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.003s	sys 0.036s
I20260709 12:18:28.747370 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:28.748075 32655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.004s	sys 0.036s
I20260709 12:18:28.707293  2711 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:28.715744  2807 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:28.755753 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.758225 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.759378 32120 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:28.759596 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.693846  3151 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.708491  3120 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.765906 32117 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:28.763330 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:28.765509 32120 pending_rounds.cc:85] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
W20260709 12:18:28.771278 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51973.8us
I20260709 12:18:28.770931 32112 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.769359 32119 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.775086 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.775902  3124 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:28.778656 32115 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:28.779032 32115 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:28.771059 32120 pending_rounds.cc:107] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:28.779587 32120 raft_consensus.cc:2889] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:28.769807 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.750979  3023 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.782236 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.768664 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:28.785357 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.760169  3134 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:28.787640  2909 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:28.788043  2909 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:28.789073  3134 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:28.788882  2909 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:28.789520  3134 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:28.789920  2909 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:28.764742  2807 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.795506 32117 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:28.763988  3035 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.799888 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.763545  3135 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:28.802981 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.804495 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:28.805068 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.805277 32040 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:28.753634  2630 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.782517  3113 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.771821  3005 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:28.797744  2709 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:28.798552  2997 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.000s	sys 0.027s
I20260709 12:18:28.819594 32120 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:28.820055 32120 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:28.820139 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 205076us
I20260709 12:18:28.766510  3141 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 12:18:28.821753 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.809402  3023 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.824875  3131 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.842947 32119 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:28.827248  2971 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.829129  2630 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.815336  3135 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.830053 32120 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.824467 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:28.861017 32120 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.861151 32118 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:28.861502  3065 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.788s	user 0.006s	sys 0.003s
I20260709 12:18:28.862066 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.862922 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.863142 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:28.864084 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.864391 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.865106 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:28.865437 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.866220 32118 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:28.819412  2978 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:28.835722 32121 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:28.819487  2709 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:28.835618 32046 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.005s	sys 0.012s
W20260709 12:18:28.837145  3006 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.947s	user 0.007s	sys 0.003s
I20260709 12:18:28.837332 32117 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:28.837750 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.825160  3151 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:28.780217  3136 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 12:18:28.822127  2997 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.000s	sys 0.028s
I20260709 12:18:28.818526  3113 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:28.866282 32114 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:18:28.867960 32046 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.005s	sys 0.013s
I20260709 12:18:28.875324 32114 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:28.866601 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:28.843067 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:28.878599 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.818549 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:28.867880  3133 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.867341  2978 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.891309  3133 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.893337  3108 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.867280  2709 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.875720 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1885) took 1062 ms. Trace:
I20260709 12:18:28.895489  3108 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:28.898874 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 92496.3us
I20260709 12:18:28.898742 32046 rpcz_store.cc:276] 0709 12:18:27.814469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.825962 (+ 11493us) service_pool.cc:224] Handling call
0709 12:18:27.826289 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.826541 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:27.841477 (+ 14936us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.875410 (+1033933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.875458 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:28.875502 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.875563 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.875670 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2158,"prepare.queue_time_us":15180,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":706240,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":1033532,"threads_started":2,"wal-append.queue_time_us":813085}
I20260709 12:18:28.901911 32120 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:18:28.902261 32120 pending_rounds.cc:85] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
W20260709 12:18:28.902460 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59458.9us
I20260709 12:18:28.861958  3065 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.902498 32120 pending_rounds.cc:107] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:28.905297 32120 raft_consensus.cc:2889] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:28.901202 32114 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.898015  2807 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.906059 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:28.909685 32112 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
W20260709 12:18:28.963862 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59798us
I20260709 12:18:28.975216  3150 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.980130  3150 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.980803  2911 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:28.981302  2911 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.983317  2911 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.986474  2997 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.986595  3025 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.987005  2997 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.987095  3025 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.989097  2997 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.986474 31938 leader_election.cc:400] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11.
I20260709 12:18:28.989545 31938 leader_election.cc:403] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:28.989593  2997 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.990573 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:28.991515  3155 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11.
I20260709 12:18:28.991660 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:28.992192  3155 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.992920  3155 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.989118  3025 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:28.993551 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.995364  3113 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:28.998294 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:28.999493  2911 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:29.001868  3156 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.003166  3153 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:29.004313  2911 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:29.004889  2911 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:29.004474  3134 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.005923 32655 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:28.982255  3150 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.006727  2911 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.990087  2997 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.009733  2997 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.014289 31965 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:28.975075  3110 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:29.015664  3110 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:29.016543  3110 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.019549 32115 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:28.976883  3065 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:28.977360 32114 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.906176  3151 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:28.979156 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73565.2us
I20260709 12:18:29.020444 32115 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:29.024991 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 215011us
I20260709 12:18:29.025913 31965 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:29.023905  3113 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:29.028264  3012 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.025844 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.028751  3012 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.029408  3065 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:29.029825  3065 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.030620 32010 leader_election.cc:400] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:29.030925 32010 leader_election.cc:403] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:29.027949  3113 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.036284 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:29.038165 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:29.038938 32040 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:29.043151 32044 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:29.049384 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:29.050092 32039 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:29.030198  3065 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.056384 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.057350 32047 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:29.061072 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.063163 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:29.062769  2997 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:29.062909  3006 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11.
I20260709 12:18:29.066286  3113 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:29.067569 32114 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:29.071282  3134 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.072469  3144 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:29.073652 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:29.075536 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.076305 32043 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
W20260709 12:18:29.077814 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47368.2us
I20260709 12:18:29.079018 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:29.080868 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.082263 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:29.083390 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.084208 32041 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:29.085212 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.085882 32041 raft_consensus.cc:2376] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8. Re-sending same reply.
I20260709 12:18:29.087598 32045 raft_consensus.cc:2368] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:18:29.087464 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:29.088690 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.090128 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.090960 32041 raft_consensus.cc:2376] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:18:29.099215 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.100102 32047 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:29.103229 32115 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:29.103889 32115 pending_rounds.cc:85] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
I20260709 12:18:29.104250 32115 pending_rounds.cc:107] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:29.104611 32115 raft_consensus.cc:2889] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:29.107918 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.110363 32114 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:29.111192 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.111768 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:29.112677 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.112694 32040 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:29.111940 32039 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:29.115705 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.116487 32043 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:29.118068 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.119681 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.120584 32112 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:29.120889 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:29.121670 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.123648 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.124533 32047 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:29.125696 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.126477 32119 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:29.126756 32117 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:29.129601 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.130435 32121 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:29.132736 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:29.133844 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 112597us
I20260709 12:18:29.136085 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:29.144447 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.086525  2709 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.106391  3113 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:29.064488  2978 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:18:29.168085 32116 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:29.169375 32116 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:29.113996  3066 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:29.114193  3144 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:29.169904 32039 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:28.910334  3023 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.228s
I20260709 12:18:29.172175 32119 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:29.174420  3152 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 12:18:29.092233  2984 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
W20260709 12:18:29.145936  2909 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 12:18:29.188086 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52087.1us
I20260709 12:18:29.201807 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:29.188755 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.189841 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.132732  3153 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.197713 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.197955 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:29.148567  3151 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:29.198978 32038 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:29.199033 32039 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 12. (index mismatch)
I20260709 12:18:29.200695 31968 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:29.208168 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:29.195647 32116 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.208346 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.208992 32116 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.209462 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.209885 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:29.210060  3153 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.210500 31966 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:29.210742  2855 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:29.212260  3153 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.196237  2978 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.213076  2978 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.213606  2997 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:29.213845 32041 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: [5.8-8.11]
W20260709 12:18:29.215075  3154 consensus_queue.cc:1175] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f05e4741d35c4cf3a31358c56e65b30c" responder_term: 10 status { last_received { term: 9 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:29.210518 31972 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.219075 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.220088 32112 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:29.215185  2978 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.206720  3151 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:29.221346 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.221783  3151 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.222271 32119 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:29.206067  3135 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:29.227650 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:29.224697 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:29.228278 31969 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.230489 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.234230 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:29.201416  3113 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:29.236130 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.240463 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.241303 32045 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:29.218823  3131 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.241183 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:29.196352  3065 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.223246  3006 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.229694  2855 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.245400 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.249836 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.254182 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.254699 31972 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:29.239945  2978 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:18:29.256275  2984 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:29.256847  2984 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:29.257722  2984 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.4, Last appended by leader: 4, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.244665  3153 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:29.258792  3153 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.258936  2709 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.259208  3153 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.237190  3030 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:29.261377 31969 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:29.263017 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:29.265019 31971 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:29.273243  2953 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:29.275017  3163 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:29.275796  3163 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:29.276154  3164 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:29.276650  3164 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.277056  2855 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:29.277760  2855 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:29.276970  3163 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.249262  3131 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.267127 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:29.304502 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.304621 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.305145 32121 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:29.283214 32039 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:29.283541 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.307678 32044 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:29.248831  3144 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:29.287098 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.287590 32112 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:29.314846 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:29.290732 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1882) took 1480 ms. Trace:
I20260709 12:18:29.316237 32042 rpcz_store.cc:276] 0709 12:18:27.810549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.810717 (+   168us) service_pool.cc:224] Handling call
0709 12:18:27.821237 (+ 10520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.855637 (+ 34400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.290459 (+1434822us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b540037a854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:29.290656 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75788672}
I20260709 12:18:29.245127  3148 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:29.320312 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:29.321125 32039 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.330422 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:29.337990 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70948.8us
I20260709 12:18:29.235755  3135 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.341600  2984 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.262375  3158 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.264523  2807 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:29.346686 32042 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:29.260425  3006 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.349433 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:29.351418 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.419204  2984 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.355225 32039 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:29.289270  3065 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.290962  3153 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.318380  3144 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.293778  3012 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.360682 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 408) took 1321 ms. Trace:
I20260709 12:18:29.430236 32113 rpcz_store.cc:276] 0709 12:18:28.038811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.038935 (+   124us) service_pool.cc:224] Handling call
0709 12:18:28.405598 (+366663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:28.405783 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:28.562968 (+157185us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.300406 (+737438us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b180032cf80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:29.360096 (+ 59690us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.360144 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:29.360187 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.360247 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.360627 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2008,"prepare.queue_time_us":157017,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3281,"raft.queue_time_us":2101,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"spinlock_wait_cycles":21684096,"thread_start_us":57206,"threads_started":1,"wal-append.queue_time_us":861046}
I20260709 12:18:29.276986  3164 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.291802  3131 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.278712  2855 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.415867 31967 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.327316  3030 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:29.278905  2978 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:18:29.345813  3148 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:29.347481 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:29.432365 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80992.9us
I20260709 12:18:29.357895  3110 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:29.437639 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.360739  2971 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:29.414180 32047 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.412892  3023 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.443106  2709 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:29.452279 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.462172 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.345479  3113 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.453845 31968 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:29.453950 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.445498  2984 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:29.442512  2807 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:18:29.439824  3144 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.454037 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:29.459627  3173 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.460136  3153 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.458179  2971 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.490011  3173 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:29.490662  3135 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:29.491530  3135 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:29.491977  3135 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:29.492321  3173 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:29.492537  2971 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.357676  2968 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:18:29.492864  3135 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.464929 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.410518  2997 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:18:29.468252 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.519649 32112 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.470532  2709 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:29.476881 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:29.455641 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 108453us
I20260709 12:18:29.522083 32047 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:29.467630 32113 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:18:29.471215  3172 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:18:29.465658 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.529145 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.493644  2968 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:29.419016  2953 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:18:29.500212  3187 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:29.500538  3006 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:29.464576  3110 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.501508  3133 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:29.507494 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.464586  3156 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:29.511303 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.518729 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.519987 32116 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:18:29.527491 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73533.7us
W20260709 12:18:29.530550 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74126us
I20260709 12:18:29.464763  3023 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.531302  2709 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.523381 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:29.587255 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63960.9us
I20260709 12:18:29.517774  3019 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:29.538400  3171 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:18:29.510783  2807 consensus_queue.cc:1243] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:29.471854  2805 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 12:18:29.584360 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.545s (SENT) [suppressed 19 similar messages]
I20260709 12:18:29.584935 32114 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:29.542026  3144 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.543105  2968 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:29.542855  3153 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 12:18:29.653460 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.590888 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.653836 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.656083 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:29.664603 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:29.651285 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.590379 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.561352  2711 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.518s
I20260709 12:18:30.414814 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:30.418901 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:30.419457 32120 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:29.642661 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:30.412241 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:29.600451 32112 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:30.413952 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 383) took 1331 ms. Trace:
W20260709 12:18:30.104964  3164 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.657s	user 0.001s	sys 0.010s
I20260709 12:18:30.422297 32112 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:30.415185  2977 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:30.212101  3189 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.711s	user 0.005s	sys 0.000s
W20260709 12:18:30.310619 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.46897e+06us
I20260709 12:18:30.422330 32045 rpcz_store.cc:276] 0709 12:18:29.082050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.106781 (+ 24731us) service_pool.cc:224] Handling call
0709 12:18:29.107080 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.246463 (+139383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.413715 (+1167252us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400032ad4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:30.413891 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308207360}
I20260709 12:18:30.426084 32656 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:29.664290  3023 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:30.273603 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.49844e+06us
I20260709 12:18:30.317647 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:30.386989 31936 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.009s	sys 0.012s
W20260709 12:18:30.025768 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.449129 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:30.291198  3000 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.361660 32653 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:18:30.455513 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:30.456709 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:30.315708 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:30.240147 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.458184 32655 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:29.881264 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:30.459235 32040 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:30.459321 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:30.460135 32655 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:29.982900 31968 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:30.460752 32655 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:30.362093 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:30.423933 32112 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:30.463078 32112 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.464351 32112 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.465267 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:29.727689  3100 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:29.606818  2984 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:30.311311  3136 consensus_queue.cc:1243] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 9.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:18:30.473464 32112 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:29.983457 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.665529 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 584) took 1855 ms. Trace:
I20260709 12:18:29.696238 31967 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:29.655699  3001 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:30.316188  3124 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.193321 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:29.666179 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 341) took 1828 ms. Trace:
I20260709 12:18:29.988322  3144 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:30.212991 31965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2033) took 3378 ms (client timeout 1545 ms). Trace:
I20260709 12:18:29.654035 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:29.950834 31937 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.007s	sys 0.006s
W20260709 12:18:30.187947 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 205709us
W20260709 12:18:30.188189 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 641511us
I20260709 12:18:30.452595 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:30.453325 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:30.315802 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:30.449713 31936 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.009s	sys 0.013s
W20260709 12:18:30.101567  3151 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.879s	user 0.006s	sys 0.013s
I20260709 12:18:30.314616  2840 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.472990 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.666272  3110 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:30.428694 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:30.050326 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:29.607291  3148 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:29.654560  3174 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.877s
I20260709 12:18:30.547616 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1920) took 1265 ms. Trace:
I20260709 12:18:29.651507  3175 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:29.656402  3019 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.592348 32656 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:30.310281  3141 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:30.465945 32657 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:30.474483 32116 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:30.474675 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2046) took 1148 ms. Trace:
W20260709 12:18:30.535421 31965 rpcz_store.cc:269] 0709 12:18:26.834262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.835022 (+   760us) service_pool.cc:224] Handling call
0709 12:18:29.014127 (+2179105us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b580067bbd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c12b0aa 00007f228c1ad447 00007f228c1c5fe7 00007f228c1c77de 00007f228c1d327c 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c
0709 12:18:30.212761 (+1198634us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400079ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:30.212923 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4288751744}
I20260709 12:18:30.474237 32654 rpcz_store.cc:276] 0709 12:18:27.809962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.810429 (+   467us) service_pool.cc:224] Handling call
0709 12:18:27.821705 (+ 11276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.959479 (+137774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.665266 (+1705787us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400550ad4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:29.665460 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293324544}
I20260709 12:18:30.465212  3137 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:29.663734  3066 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:30.646886 31964 rpcz_store.cc:276] 0709 12:18:27.837658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.837773 (+   115us) service_pool.cc:224] Handling call
0709 12:18:29.665951 (+1828178us) spinlock_profiling.cc:243] Waited 73.3 ms on lock 0x7b5400315e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:29.666111 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168627072}
W20260709 12:18:30.646402 31937 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.016s	sys 0.030s
I20260709 12:18:30.592427 32652 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:30.630018 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:30.647300 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 833848us
I20260709 12:18:30.647089 32045 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.592473 32044 rpcz_store.cc:276] 0709 12:18:29.282416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.348054 (+ 65638us) service_pool.cc:224] Handling call
0709 12:18:29.348471 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.356532 (+  8061us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:29.421074 (+ 64542us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.482039 (+ 60965us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.590059 (+108020us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:30.416328 (+826269us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b2c0005c228. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:18:30.416758 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:18:30.416908 (+   150us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:30.416943 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.417007 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.547349 (+130342us) raft_consensus.cc:1685] finished
0709 12:18:30.547377 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.547543 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19488,"prepare.queue_time_us":71977,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":28010,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":53349,"spinlock_wait_cycles":25801088,"thread_start_us":136468,"threads_started":2,"wal-append.queue_time_us":87917}
I20260709 12:18:30.660619 32658 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:30.658010 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2085) took 1016 ms. Trace:
I20260709 12:18:30.661218 32116 rpcz_store.cc:276] 0709 12:18:29.641993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.983185 (+341192us) service_pool.cc:224] Handling call
0709 12:18:30.657932 (+674747us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:30.661089 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.07665e+06us
I20260709 12:18:30.657917 31967 rpcz_store.cc:276] 0709 12:18:29.325903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.326036 (+   133us) service_pool.cc:224] Handling call
0709 12:18:30.474620 (+1148584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.555737  3156 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:30.665045 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.01101e+06us
I20260709 12:18:30.666019 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:30.471468  3100 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:30.593505  2984 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:30.650621  2709 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:18:30.678159  3065 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.475373  3196 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.672528  2549 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.671766 32084 connection.cc:256] RPC call timeout handler was delayed by -2.3514601169999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
I20260709 12:18:30.679625 32659 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:30.461735  3164 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:30.684103 32084 connection.cc:256] RPC call timeout handler was delayed by -2.1912601169999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 12:18:30.684460 32084 connection.cc:256] RPC call timeout handler was delayed by -2.0607601169999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 12:18:30.684759 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9968601169998692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 12:18:30.684895 32038 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:30.685137 32084 connection.cc:256] RPC call timeout handler was delayed by -1.8960601169999336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2323s for a response
I20260709 12:18:30.685207 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:30.685604 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0428601169999183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3271s for a response
I20260709 12:18:30.466598  3023 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.204s
I20260709 12:18:30.661096  3066 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:30.686172 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:30.655952  3141 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:30.678758  3065 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.538254  2968 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.673112  3175 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:30.530085  3136 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.146s
I20260709 12:18:30.674459  3137 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:30.652627  2840 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.659557  3151 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.699288 32038 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:30.691816  3156 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:30.702064 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:30.690976  2549 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.702833 32118 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:30.703045  3158 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:30.703769  3158 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:30.704901  2549 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:30.702107  3100 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.11, Last appended by leader: 11, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.706393 32084 leader_election.cc:400] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:30.704748  3158 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.695621  3110 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:18:30.705312  3151 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:30.661196  3124 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.709378  3065 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:30.736575 32651 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:18:30.740737 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.714098 32084 leader_election.cc:403] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:30.689289  3196 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.769414  3151 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:30.772058 32660 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:18:30.769277 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63019.2us
I20260709 12:18:30.778635 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:30.780370  3148 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:30.711834  3001 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:18:30.768285  3100 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:30.790266 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.797797 31967 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:30.706283  3156 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.772111  3205 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:30.806747 31965 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:30.808566 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:30.700709  3141 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.814694 31966 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:30.810679 31972 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:30.811563  3030 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.025s	sys 0.029s
I20260709 12:18:30.818562 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.820137 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.820776 32121 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:30.821194 32121 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:30.822762 31966 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:18:30.822808 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 365246us
I20260709 12:18:30.824514 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.739566  3066 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.827893 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.737367  2984 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.828543 31970 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:30.827287  2840 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:30.822013 32121 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.831097 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.826500 31972 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:30.831642 31970 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:30.832190 31970 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 6: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:30.831996 32112 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:30.831593 32121 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.832623 32112 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:30.826812  3141 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:30.826995 31966 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:30.827315 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:30.827514 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:30.817847  3030 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.026s	sys 0.029s
I20260709 12:18:30.804739  3203 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:30.831580 31866 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 5 to 6, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 6 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.833806 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:30.837813 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 383859us
I20260709 12:18:30.837922 31972 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.824575 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:30.844003 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:30.847966 32116 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:30.844355 32045 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 4. Preceding OpId from leader: term: 13 index: 5. (index mismatch)
I20260709 12:18:30.848415 32116 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:30.850157 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.594s (SENT)
W20260709 12:18:30.845580 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.850627 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:30.845683 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:30.846302 31965 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:30.846513 32112 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.851491 32114 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:30.754994  3175 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.851748 32112 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:30.853123 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:30.853817 32044 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:30.854260 31970 raft_consensus.cc:1217] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Deduplicated request from leader. Original: 8.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 12:18:30.854941 31970 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:30.849380 32116 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.856134 32116 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.858920 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:30.859527 32044 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.832934  3066 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.864869 32112 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 5. (term mismatch)
I20260709 12:18:30.870538 32112 pending_rounds.cc:85] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:18:30.871412 32112 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 12.5
I20260709 12:18:30.871806 32112 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:30.871688 31972 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:30.878398 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.882769 32121 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:30.883064 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:30.883348 32121 pending_rounds.cc:85] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
I20260709 12:18:30.855440  2840 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:30.846796  3203 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:30.797196  3137 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:30.886632 31966 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.888145 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:30.899857 31938 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:30.905764 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55618.4us
I20260709 12:18:30.913717 32116 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:30.914251 32116 pending_rounds.cc:85] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
I20260709 12:18:30.914608 32116 pending_rounds.cc:107] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:30.917420 32116 raft_consensus.cc:2889] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:30.858119  3209 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:30.931555 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:30.927342 31970 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:30.889011 32121 pending_rounds.cc:107] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:30.948217 32121 raft_consensus.cc:2889] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:30.953035 32044 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:30.940557  3212 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.940646  2976 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:30.944936 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:30.873270  3000 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.853536  2952 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 12:18:30.968056 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.886945  3220 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.068940 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:31.069696 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 128486us
I20260709 12:18:30.888484  3164 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:31.068388 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 168609us
I20260709 12:18:30.817515  3205 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:30.891160  2840 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.124959 31970 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:30.871587  3178 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.148s
I20260709 12:18:31.137925  2840 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.130760  3001 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:31.139083 31970 raft_consensus.cc:1217] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Deduplicated request from leader. Original: 7.4->[8.5-10.6]   Dedup: 8.5->[10.6-10.6]
I20260709 12:18:31.176015 31970 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:18:31.140975 31964 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:30.884892  3120 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.188s
I20260709 12:18:31.146095 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:31.132047 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:31.168618  3205 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:31.172176  3208 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.286s	user 0.000s	sys 0.000s
W20260709 12:18:31.173039  3207 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:18:30.929451  3150 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.206s
I20260709 12:18:31.017513  3142 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.077922  3154 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:31.002135  2805 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.210s
I20260709 12:18:31.288906 31965 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:31.291152 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:31.293582 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.011s
I20260709 12:18:31.004179  3151 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.180s
I20260709 12:18:30.965119  3148 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.219s
W20260709 12:18:31.293919 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 162021us
W20260709 12:18:31.255450 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 109445us
I20260709 12:18:31.311589 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:31.312956 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:31.297564 32039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.007s	sys 0.004s
I20260709 12:18:30.890651  3211 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.371s
W20260709 12:18:31.307195 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.000s	sys 0.011s
I20260709 12:18:31.310065 32657 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:18:31.315320 32039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.007s	sys 0.004s
I20260709 12:18:31.314263  3154 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:31.323251  3154 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.327711  3151 consensus_queue.cc:1243] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:31.328280  3151 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:31.328606  3151 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:31.329377  3151 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:31.336107 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 429526us
W20260709 12:18:31.336942 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.337001 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.337405 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.306296  2976 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:31.338984 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:31.340271 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:31.314481  3148 consensus_queue.cc:1243] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:31.341848 32114 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:31.255627  3001 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:31.340600 32623 connection.cc:256] RPC call timeout handler was delayed by -1.6587157740002567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
I20260709 12:18:31.145511  3220 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:31.348794 32120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.008s	sys 0.010s
I20260709 12:18:31.348869 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2040) took 2141 ms. Trace:
W20260709 12:18:31.349241 32120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.008s	sys 0.011s
I20260709 12:18:31.349327 31971 rpcz_store.cc:276] 0709 12:18:29.207827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.208074 (+   247us) service_pool.cc:224] Handling call
0709 12:18:31.348632 (+2140558us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b54000b6654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:31.348811 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128211200}
I20260709 12:18:31.349972 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1984) took 1687 ms. Trace:
I20260709 12:18:31.350709 31967 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:31.345017 32623 connection.cc:256] RPC call timeout handler was delayed by -1.6569157740002538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 12:18:31.352687 32623 connection.cc:256] RPC call timeout handler was delayed by -1.6182157740001912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
W20260709 12:18:31.355652 32623 connection.cc:256] RPC call timeout handler was delayed by -1.5471157740000763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 12:18:31.356101 32623 connection.cc:256] RPC call timeout handler was delayed by -1.3284157740001774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
W20260709 12:18:31.339829  3019 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.003s
W20260709 12:18:31.361377 32047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 387) took 2186 ms (client timeout 2222 ms). Trace:
W20260709 12:18:31.364112 32047 rpcz_store.cc:269] 0709 12:18:29.175169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.197720 (+ 22551us) service_pool.cc:224] Handling call
0709 12:18:31.361152 (+2163432us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001941d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:31.361303 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359864704}
I20260709 12:18:31.361510 32041 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:31.339229  3021 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.338055  2549 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:31.393353 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:31.344219  3211 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:31.345099  3151 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:31.399719 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.351806  3079 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:31.351593  3196 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:31.348133  3142 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.252673  3203 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:31.385808  3000 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.448067  3019 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:31.450330 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1930) took 1009 ms. Trace:
I20260709 12:18:31.451174  3148 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.400885  3231 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:31.453359  3211 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:31.454586  3124 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:31.417166  3156 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:31.458446  3184 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:31.447078 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53075us
I20260709 12:18:31.471575  3001 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:31.387315  3121 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.357316  2976 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:31.397557  3209 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:31.401439  3021 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.517660 32039 rpcz_store.cc:276] 0709 12:18:30.440758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.440860 (+   102us) service_pool.cc:224] Handling call
0709 12:18:30.446599 (+  5739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.446948 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.446977 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:30.461164 (+ 14187us) log.cc:844] Serialized 26 byte log entry
0709 12:18:31.315777 (+854613us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:31.315815 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.315884 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.450050 (+134166us) raft_consensus.cc:1685] finished
0709 12:18:31.450077 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.450246 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8878,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":3871,"thread_start_us":805124,"threads_started":1,"wal-append.queue_time_us":834604}
I20260709 12:18:31.418426  2549 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:31.558418 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.559273 31966 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.670s	user 0.001s	sys 0.009s
W20260709 12:18:31.559231 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 245013us
I20260709 12:18:31.564968 31966 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:31.560119 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 247219us
I20260709 12:18:31.520417  3231 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:31.565847 32623 leader_election.cc:400] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:18:31.570019 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:31.529791  3203 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.521411  3000 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:31.576550 32121 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 5.
I20260709 12:18:31.579989 32116 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:31.527015  3148 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.571130  3231 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.576601  2805 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.597292  3240 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:31.598665  3000 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:31.598917  3021 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:31.603446 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 645) took 1291 ms. Trace:
I20260709 12:18:31.603839 32651 rpcz_store.cc:276] 0709 12:18:30.312047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.314760 (+  2713us) service_pool.cc:224] Handling call
0709 12:18:30.315030 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.748133 (+433103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.603217 (+855084us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54003a07d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:31.603388 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968420224}
I20260709 12:18:31.609194  3134 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:31.609591  3134 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:31.610538  3134 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:31.612675  3141 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.776s	user 0.006s	sys 0.004s
I20260709 12:18:31.613662 32658 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:31.616948 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 703) took 1303 ms. Trace:
I20260709 12:18:31.617353 32659 rpcz_store.cc:276] 0709 12:18:30.313918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.317369 (+  3451us) service_pool.cc:224] Handling call
0709 12:18:31.616731 (+1299362us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54005eb5d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:31.616888 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528718592}
I20260709 12:18:31.554716  3184 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.612793 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 648) took 1299 ms. Trace:
I20260709 12:18:31.624619 32660 rpcz_store.cc:276] 0709 12:18:30.312896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.361850 (+ 48954us) service_pool.cc:224] Handling call
0709 12:18:31.612571 (+1250721us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54006031d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:31.612712 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701289856}
I20260709 12:18:31.567391  3121 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:31.630378 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:31.606138  3066 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.731s	user 0.007s	sys 0.000s
I20260709 12:18:31.632354 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:31.635866 32653 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:31.642827 31968 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:31.628360  3134 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.644999  3154 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.651098  3229 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:31.651350  3238 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:18:31.648535  3219 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:31.657474  3224 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:31.657629  3245 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:18:31.657909  3230 raft_consensus.cc:476] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Not starting pre-election -- already a leader
I20260709 12:18:31.658658  3234 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:31.629688  3135 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:31.657976  3216 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.657234  3222 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:31.593240  2805 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.665755 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1929) took 1225 ms. Trace:
I20260709 12:18:31.669659 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:31.646798  3151 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.671007 32044 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:31.593928  3209 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:31.671007 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:31.673380  3148 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:31.675087  3205 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:31.679117  3156 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:31.678843  3205 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.679535  3156 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:31.679966  3156 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.680425  2805 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:31.681381  2549 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:31.658905  3154 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.585253  3211 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.678301 32045 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:31.702651 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1926) took 1752 ms. Trace:
I20260709 12:18:31.676824 32047 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:31.761830 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.013160 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.351011 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 718690us
I20260709 12:18:31.765793 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:31.663988  3229 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.451851 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:32.559084 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.761063 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.366809 32044 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 8: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:31.350342 32120 rpcz_store.cc:276] 0709 12:18:29.662852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.420170 (+757318us) service_pool.cc:224] Handling call
0709 12:18:30.420450 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.420650 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:30.687488 (+266838us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.349668 (+662180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:31.349720 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:31.349768 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.349829 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.349924 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121748,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":1741,"thread_start_us":928247,"threads_started":2}
W20260709 12:18:32.646593 32011 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.008s	sys 0.010s
I20260709 12:18:32.407356 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1933) took 1954 ms. Trace:
I20260709 12:18:32.352147 32042 rpcz_store.cc:276] 0709 12:18:29.950202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.454887 (+504685us) service_pool.cc:224] Handling call
0709 12:18:30.473798 (+ 18911us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 12:18:30.500931 (+ 27133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.500987 (+    56us) raft_consensus.cc:1539] Triggering prepare for 5 ops
0709 12:18:30.558436 (+ 57449us) log.cc:844] Serialized 130 byte log entry
0709 12:18:30.738521 (+180085us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:30.789291 (+ 50770us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.801850 (+ 12559us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.883836 (+ 81986us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.620142 (+736306us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b18001176a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:31.663908 (+ 43766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.702193 (+ 38285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.702276 (+    83us) raft_consensus.cc:1685] finished
0709 12:18:31.702295 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.702569 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":919944,"prepare.queue_time_us":682379,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":1548,"raft.queue_time_us":1829715,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":118001,"spinlock_wait_cycles":52026240,"thread_start_us":199968,"threads_started":4,"wal-append.queue_time_us":131423,"wal-append.run_cpu_time_us":5262,"wal-append.run_wall_time_us":1612631}
I20260709 12:18:32.384876 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:31.703681 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:32.404843 32047 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:31.757972  3100 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.960s	user 0.012s	sys 0.000s
W20260709 12:18:31.954057 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.647173 32011 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.008s	sys 0.010s
W20260709 12:18:32.465564  3203 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.889s	user 0.011s	sys 0.000s
I20260709 12:18:32.676904 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1833) took 1007 ms. Trace:
W20260709 12:18:31.827876  3241 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:18:32.454210 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:32.691438 32115 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:32.388109 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 593) took 2073 ms. Trace:
I20260709 12:18:32.411569 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2039) took 3204 ms. Trace:
I20260709 12:18:32.615453  3205 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.646986 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:32.686944 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 647970us
I20260709 12:18:31.760345  3210 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.687978 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.652532  2709 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:32.688308 32047 rpcz_store.cc:276] 0709 12:18:31.669301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.669416 (+   115us) service_pool.cc:224] Handling call
0709 12:18:32.676824 (+1007408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.683622 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1846) took 1010 ms. Trace:
I20260709 12:18:32.676302  3143 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:32.178591  3231 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.000s
W20260709 12:18:32.676573 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 972941us
I20260709 12:18:32.366809  3142 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:31.792915  3242 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:18:31.675626  3226 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.676622 32043 rpcz_store.cc:276] 0709 12:18:30.440494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.441507 (+  1013us) service_pool.cc:224] Handling call
0709 12:18:30.465614 (+ 24107us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:30.838548 (+372934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.838582 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:30.845758 (+  7176us) log.cc:844] Serialized 52 byte log entry
0709 12:18:30.999390 (+153632us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:31.541309 (+541919us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.665326 (+124017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.665440 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.665494 (+    54us) raft_consensus.cc:1685] finished
0709 12:18:31.665512 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.665681 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122511,"prepare.queue_time_us":282013,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":215610,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":5012,"thread_start_us":770441,"threads_started":3,"wal-append.queue_time_us":88692,"wal-append.run_cpu_time_us":3378,"wal-append.run_wall_time_us":1653522}
I20260709 12:18:32.690855 32656 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 11. Preceding OpId from leader: term: 9 index: 12. (index mismatch)
I20260709 12:18:32.453285 32113 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:32.692041 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:32.337437  3156 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.001s
I20260709 12:18:32.692840  3100 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:31.788673  3250 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:18:32.692261 31963 rpcz_store.cc:276] 0709 12:18:29.207562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.207984 (+   422us) service_pool.cc:224] Handling call
0709 12:18:30.212914 (+1004930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.213229 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.213259 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.213319 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.213359 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.213427 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.411362 (+2197935us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400079ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:32.411497 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":210437,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":2301575552}
I20260709 12:18:31.829263  3184 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:31.927999  2549 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.642225 32117 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
W20260709 12:18:32.694514 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47613.9us
I20260709 12:18:31.792670  3220 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.685411  3108 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:32.725450  3108 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.725880  3108 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.734555  3265 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:32.735406  3154 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:32.736137 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:32.735183  3265 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:31.669581  3219 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.696201 32042 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
W20260709 12:18:31.845129  3248 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:18:31.908262  2840 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.001s
I20260709 12:18:32.647955 32041 rpcz_store.cc:276] 0709 12:18:30.453066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.453174 (+   108us) service_pool.cc:224] Handling call
0709 12:18:31.361299 (+908125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.386754 (+ 25455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.407108 (+1020354us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54001941d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:32.407279 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087798656}
I20260709 12:18:32.742862 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1849) took 1068 ms. Trace:
I20260709 12:18:32.697163 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:32.743271 32042 rpcz_store.cc:276] 0709 12:18:31.673975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.663340 (+989365us) service_pool.cc:224] Handling call
0709 12:18:32.742805 (+ 79465us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:32.743773 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46700.5us
I20260709 12:18:32.714509  3253 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.744359 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:31.703187  3135 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.745182 32042 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:32.745571 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1843) took 1073 ms. Trace:
I20260709 12:18:32.699435 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.701320 32044 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:32.708616 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.745879 32042 rpcz_store.cc:276] 0709 12:18:31.672623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.744116 (+1071493us) service_pool.cc:224] Handling call
0709 12:18:32.745517 (+  1401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.692129 32658 rpcz_store.cc:276] 0709 12:18:30.314279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.665036 (+350757us) service_pool.cc:224] Handling call
0709 12:18:30.665297 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.613987 (+948690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.387864 (+773877us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b540053edd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:32.388046 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180208896}
W20260709 12:18:32.698740  3134 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.035s	user 0.011s	sys 0.000s
I20260709 12:18:32.711983 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:32.696588  3203 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:31.739895  3216 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.718842 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:31.676334  3224 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.697956  3205 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.696578 32045 rpcz_store.cc:276] 0709 12:18:31.673302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.384559 (+711257us) service_pool.cc:224] Handling call
0709 12:18:32.683536 (+298977us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.744437 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:32.758296 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:32.652863  3174 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:32.384227  3151 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:31.703300  3234 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.760139 32660 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:32.764081 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.765209 32041 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:32.698732  2709 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.782771 32043 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:32.783447 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.784327 32042 raft_consensus.cc:2410] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:32.787916 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.711357  3156 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.744648  3253 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.708725  3226 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.748421  3184 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.748240  3215 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.756620  2711 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.757125  3261 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.827416 32044 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:32.756695  3203 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.835973  3203 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.832955 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.699904  3231 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:32.764559  3209 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.030s	sys 0.039s
I20260709 12:18:32.712937  3100 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.757112  3154 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.700045  3210 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.759534  3257 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:32.842947 32117 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:18:32.843329 32117 pending_rounds.cc:85] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 7
I20260709 12:18:32.839543 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1936) took 1997 ms. Trace:
I20260709 12:18:32.742018  2549 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.756639  3220 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:18:32.843538 32117 pending_rounds.cc:107] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:18:32.853758 32117 raft_consensus.cc:2889] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:32.790947  3265 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:32.845270 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 100900us
W20260709 12:18:32.841712 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.844552 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1836) took 1169 ms. Trace:
I20260709 12:18:32.792047 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:32.854672 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.788414  3174 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.855196 32044 rpcz_store.cc:276] 0709 12:18:31.670966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.652770 (+981804us) service_pool.cc:224] Handling call
0709 12:18:32.840507 (+187737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.857326 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.859984 32044 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:32.748545  2840 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.857610 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:32.855729 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:32.843897 32046 rpcz_store.cc:276] 0709 12:18:30.842234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.858226 (+ 15992us) service_pool.cc:224] Handling call
0709 12:18:30.858580 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:31.252153 (+393573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:31.254693 (+  2540us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:31.282070 (+ 27377us) log.cc:844] Serialized 52 byte log entry
0709 12:18:31.512154 (+230084us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:31.512189 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.512263 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:32.839313 (+1327050us) raft_consensus.cc:1685] finished
0709 12:18:32.839345 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.839487 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40716,"raft.queue_time_us":776688,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5122,"thread_start_us":507452,"threads_started":2,"wal-append.queue_time_us":273387}
I20260709 12:18:32.782903  3205 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.699744  3143 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:32.853951  3209 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.030s	sys 0.041s
I20260709 12:18:32.880641 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:32.853282  3215 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:32.882372  3257 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.882805  3215 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.882874  3272 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:32.883651  3215 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.884009  3272 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.852702  3079 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.888352  3079 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.881096  3267 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.857791  3264 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.883154  3257 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.839954  2711 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.835064  3184 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.890328  3079 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:32.874079  3270 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.873505  3268 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:18:32.903328  3231 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.854629  3154 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:32.897570  3229 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:32.908978  2855 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:18:32.909240  3224 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.909380  3156 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.909569  3234 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.899964  3135 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:32.911058  3259 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:18:32.911672  3226 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.911897  3143 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.900269  3257 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:32.913025  3253 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.915778  3219 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.916712  3216 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.916919  3210 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.839869  3261 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.905714  3174 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.863804 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:32.976320 32047 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:32.979883 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 122180us
I20260709 12:18:33.040855 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:32.918800  3203 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.042992  3253 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.043864 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:32.980233 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.045471 32044 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:33.046329  3203 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:33.048444 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1837) took 1377 ms. Trace:
I20260709 12:18:33.049015 32044 rpcz_store.cc:276] 0709 12:18:31.671226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.046755 (+1375529us) service_pool.cc:224] Handling call
0709 12:18:33.047536 (+   781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.047947 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.047981 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.048056 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:33.048101 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.048190 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.048322 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.983808 32046 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:32.985095 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:33.050926 32657 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:33.045976  3184 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:33.051855 32046 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.052214 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:33.052533 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1839) took 1380 ms. Trace:
I20260709 12:18:33.053061 32657 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:33.052997 32046 rpcz_store.cc:276] 0709 12:18:31.671654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.051014 (+1379360us) service_pool.cc:224] Handling call
0709 12:18:33.051464 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.052295 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.052472 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.987841 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:32.903609  2549 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:32.899736  2709 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:32.990466 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:33.056066  3253 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:33.056771 32659 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:33.058244 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:33.059078 32046 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:32.993870  3151 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:32.995595  3154 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:32.992087  2840 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:32.930815  3251 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.190s	user 0.003s	sys 0.000s
W20260709 12:18:32.931633  3252 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.138s	user 0.000s	sys 0.001s
I20260709 12:18:33.060997 32044 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:32.926800  3257 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:32.929318  3261 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:32.898900  3264 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:33.067358 32653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 591) took 2753 ms (client timeout 2543 ms). Trace:
I20260709 12:18:32.899667  3270 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:33.067809 32653 rpcz_store.cc:269] 0709 12:18:30.313774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.456464 (+142690us) service_pool.cc:224] Handling call
0709 12:18:33.066949 (+2610485us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400154d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.067139 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704363776}
I20260709 12:18:32.926389  3272 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:32.977972 32010 connection.cc:256] RPC call timeout handler was delayed by -1.8751853579999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:18:33.072250 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.072392 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 708) took 2658 ms. Trace:
W20260709 12:18:33.028683 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 270303us
I20260709 12:18:33.029183  2711 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:32.990513 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:33.030519 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.066186 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2088) took 1727 ms (client timeout 2047 ms). Trace:
W20260709 12:18:33.079844 32121 rpcz_store.cc:269] 0709 12:18:31.338612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.338738 (+   126us) service_pool.cc:224] Handling call
0709 12:18:33.065749 (+1727011us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54002b5cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.065939 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544829696}
I20260709 12:18:33.021893  3030 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:33.053013 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:33.058961 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:33.082875  3030 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.087357 32652 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:32.924835  3205 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:33.059880 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1840) took 1388 ms. Trace:
I20260709 12:18:33.093123  3270 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:33.062016 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1838) took 1390 ms. Trace:
I20260709 12:18:33.094657 32044 rpcz_store.cc:276] 0709 12:18:31.671434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.050227 (+1378793us) service_pool.cc:224] Handling call
0709 12:18:33.060639 (+ 10412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.061697 (+  1058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.061944 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:33.072093  3264 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:33.097136 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 44518.2us
I20260709 12:18:33.069273  3253 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:33.098224 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:33.098546  3174 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:33.099115  3174 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.028342 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.086719  3030 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:33.053828  3215 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.090603  2711 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.072888 32656 rpcz_store.cc:276] 0709 12:18:30.413765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.672968 (+259203us) service_pool.cc:224] Handling call
0709 12:18:30.683602 (+ 10634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.700979 (+2017377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.071819 (+370840us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400544054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:33.072182 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4615463808}
I20260709 12:18:33.092773 32046 rpcz_store.cc:276] 0709 12:18:31.671913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.054052 (+1382139us) service_pool.cc:224] Handling call
0709 12:18:33.058711 (+  4659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.059547 (+   836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.059794 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:33.055472 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:33.096201 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.073369 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:33.102780 32044 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:33.103168 32659 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:33.102986 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.103755 32657 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:33.104035 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.105839 32044 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.106693  2709 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:33.107084  2709 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.107414  2709 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:33.097191 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:33.099599  3174 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:33.118840  3113 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:33.119401  3113 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.119551 32117 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:33.119906  3113 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.120042 32117 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:33.120441 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:33.099606 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:33.122977 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:33.123605 32116 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2089) took 1783 ms (client timeout 1651 ms). Trace:
I20260709 12:18:33.123935 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:33.124099 32116 rpcz_store.cc:269] 0709 12:18:31.339938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.340059 (+   121us) service_pool.cc:224] Handling call
0709 12:18:33.123156 (+1783097us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400138b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.123391 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549905664}
I20260709 12:18:33.124749 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.125566 32114 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:33.125442 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.126785 32116 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:33.126827 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 1.561e+06us
I20260709 12:18:31.822072 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:33.124051 32120 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:33.104036 32046 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:33.099527  3253 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.132087 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.132548 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.132994 32121 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:33.133900 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.135273  2630 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:33.136720 32115 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.136432  3066 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:33.137879 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1783) took 2262 ms. Trace:
I20260709 12:18:33.138096 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.121572 32117 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.143463 32657 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:33.144366 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1847) took 1467 ms. Trace:
I20260709 12:18:33.144739 32044 rpcz_store.cc:276] 0709 12:18:31.673535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.105047 (+1431512us) service_pool.cc:224] Handling call
0709 12:18:33.105463 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.106210 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.141147 (+ 34937us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:33.140316  2954 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:33.147228 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.147432 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.137888 31866 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:33.148068  3278 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:33.142405 31972 rpcz_store.cc:276] 0709 12:18:30.875207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.875459 (+   252us) service_pool.cc:224] Handling call
0709 12:18:30.875897 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.884770 (+  8873us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:31.076854 (+192084us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.541905 (+465051us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:31.559123 (+ 17218us) log.cc:844] Serialized 26 byte log entry
0709 12:18:31.559278 (+   155us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:31.559536 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.118058 (+1558522us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b18003da1a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:33.137151 (+ 19093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.137261 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:33.137432 (+   171us) raft_consensus.cc:1685] finished
0709 12:18:33.137449 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.137667 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10041,"prepare.queue_time_us":237618,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":141,"raft.queue_time_us":1585760,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":52882,"spinlock_wait_cycles":14452096,"thread_start_us":244320,"threads_started":1,"wal-append.queue_time_us":332654}
I20260709 12:18:33.187805  3128 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:33.196108 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.195796  3128 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.197139  3108 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.162823 32114 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:33.158617 32117 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:33.148319 32121 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:33.198813 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.156764 32113 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:33.199061 32046 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:33.156204 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:33.199136 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 69499us
I20260709 12:18:33.208400  3174 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.211753  3284 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.212487  3284 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.218487 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:33.218809 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:33.225708 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1853) took 1365 ms. Trace:
I20260709 12:18:33.226148 32047 rpcz_store.cc:276] 0709 12:18:31.860557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.699163 (+838606us) service_pool.cc:224] Handling call
0709 12:18:33.225469 (+526306us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54000573d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.225629 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526082304}
I20260709 12:18:33.226951  3216 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:33.227415 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.222424 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:33.227524  3216 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:33.229867 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 412) took 2672 ms. Trace:
I20260709 12:18:33.229629  3216 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.231029 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 650) took 2917 ms. Trace:
I20260709 12:18:33.231451 32654 rpcz_store.cc:276] 0709 12:18:30.313452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.663893 (+350441us) service_pool.cc:224] Handling call
0709 12:18:30.665438 (+  1545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.665628 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.665653 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.665690 (+    37us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:30.665719 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.665771 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.230632 (+2564861us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b54005ea1d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.230818 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2854144}
I20260709 12:18:33.232913  3163 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:33.233536  3163 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:33.235682  2954 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:33.223246  3284 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:33.149286 31866 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:33.230911 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:33.222903 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.228223 32047 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:33.252595 32044 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:33.230273 32118 rpcz_store.cc:276] 0709 12:18:30.557697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.586400 (+ 28703us) service_pool.cc:224] Handling call
0709 12:18:30.619731 (+ 33331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.703241 (+ 83510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.229379 (+2526138us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x7b54002fab54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:33.229663 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190695808}
I20260709 12:18:33.239886 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:33.200945  3278 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.218901 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.220304 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.164331  3215 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:33.210618  3203 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:33.261907 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.219529  2711 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.267405 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:33.276624 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45984.9us
I20260709 12:18:33.157186  3257 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.121s
I20260709 12:18:33.260947  3278 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.280431 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.165112  2978 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:18:33.282586 32044 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:33.279983 32115 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.282856 32112 raft_consensus.cc:2410] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:33.280110  3155 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:33.283857  3155 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:33.284138 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.284310  3144 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.284946  3155 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:33.286448 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.69859e+06us
W20260709 12:18:33.287056 32623 leader_election.cc:336] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.691s (SENT)
I20260709 12:18:33.287602 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:33.290618 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 99870.5us
I20260709 12:18:33.291527 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.292989 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.295305  3128 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:33.295950  3128 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.296759  3128 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.297384  3128 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.298938  3209 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:33.299332  2978 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.298862  3163 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.300091  2978 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:33.304309 32623 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.718s (SENT)
I20260709 12:18:33.300412  3108 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:33.264720  3203 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.294154 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.304824 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:33.318163 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51740.9us
I20260709 12:18:33.324048  3215 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.324671  3215 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.325341  3257 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:33.326742 31936 connection.cc:256] RPC call timeout handler was delayed by -1.2752937180000572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2638s for a response
I20260709 12:18:33.327155  3215 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:33.327167 31936 connection.cc:256] RPC call timeout handler was delayed by -1.0990119230000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 12:18:33.343521 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.308280  3203 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:33.354861 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46572.9us
I20260709 12:18:33.315191  3174 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:33.358742 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.357513 32046 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:33.358719 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71385us
I20260709 12:18:33.351054  3155 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:33.371809 31935 connection.cc:256] RPC call timeout handler was delayed by -1.8791217110001526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
I20260709 12:18:33.372409  2978 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:33.373072  2978 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.373648  2978 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.374140 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:33.374230  3066 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:33.375034 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.299365  3143 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:18:33.379422 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.356484  2711 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.416527  2711 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.416651  3113 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.370909 32045 raft_consensus.cc:2376] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10. Re-sending same reply.
W20260709 12:18:33.381719 32013 connection.cc:256] RPC call timeout handler was delayed by -1.6050347990001228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 12:18:33.419930 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.420266 31964 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.420718 31972 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:33.420825 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:33.421608 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.421916 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:33.438033 32043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1842) took 1765 ms (client timeout 2247 ms). Trace:
W20260709 12:18:33.438512 32043 rpcz_store.cc:269] 0709 12:18:31.672401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.708402 (+1036001us) service_pool.cc:224] Handling call
0709 12:18:33.437714 (+729312us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b540012fa54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.437880 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38293376}
W20260709 12:18:33.386605 31968 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1769) took 2578 ms (client timeout 1745 ms). Trace:
I20260709 12:18:33.366489  3134 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:33.386699  3261 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:33.390537 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.415859 31967 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:33.420003 32047 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:33.421708 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.386847  3030 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:33.422874 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.423367 31965 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:33.384850  3023 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.424677 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:33.388458  3257 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.425537 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 385) took 4311 ms. Trace:
I20260709 12:18:33.425293  3066 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.427386  2978 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:33.428092 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 123839us
I20260709 12:18:33.428027  3278 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:33.429361 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.431262  2711 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:33.432097 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.432570 31866 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 5 to 6, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 6 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:33.435088  3128 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:33.439246 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:33.523594 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.523367 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.524147  3257 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:33.526537 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 101751us
I20260709 12:18:33.521107 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:33.537007 32657 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:33.537002 31966 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:33.533460 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:33.534466 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 109942us
I20260709 12:18:33.535002  3215 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:33.534845 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.536118 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:33.537737 31866 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:33.531018 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45913.1us
I20260709 12:18:33.525817 32040 rpcz_store.cc:276] 0709 12:18:29.113917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.114201 (+   284us) service_pool.cc:224] Handling call
0709 12:18:29.121923 (+  7722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.459912 (+1337989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.425087 (+2965175us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:33.425359 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3074984704}
W20260709 12:18:33.455482 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.492610 32115 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:33.691478 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.848470 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.813403 32045 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:33.691915 31963 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:33.692520 32040 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:33.809018 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.864203 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.692786 31965 pending_rounds.cc:85] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 8
I20260709 12:18:33.741829 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:33.812773 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:33.872301 31965 pending_rounds.cc:107] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:18:33.877841 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.693077 31866 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 7 to 8, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:33.880678 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.878132 31965 raft_consensus.cc:2889] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:33.878683 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:33.878942  3264 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.771212 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:33.800895 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.774547 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:33.893734 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.814436 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 166432us
I20260709 12:18:33.694528 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:33.888615 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 75956.8us
I20260709 12:18:33.831128 31970 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:33.805745 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:33.439363 31968 rpcz_store.cc:269] 0709 12:18:30.807686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.808263 (+   577us) service_pool.cc:224] Handling call
0709 12:18:33.386146 (+2577883us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54004303d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:33.386354 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906421632}
W20260709 12:18:33.797729 31936 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.703s (SENT)
W20260709 12:18:33.896006 31867 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 229) took 21505 ms (client timeout 15000 ms). Trace:
W20260709 12:18:33.913239 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.567704  3215 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:33.906993 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.907573 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:33.878700 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 595) took 2174 ms. Trace:
I20260709 12:18:33.917418 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:33.918207 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:33.918396 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:33.921990 32117 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:33.918643 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69198us
I20260709 12:18:33.916229 32660 rpcz_store.cc:276] 0709 12:18:31.704595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.704800 (+   205us) service_pool.cc:224] Handling call
0709 12:18:32.351416 (+646616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.762032 (+410616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.534072 (+772040us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b540067ded4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:33.878532 (+344460us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938790272}
I20260709 12:18:33.924460 31972 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:33.895507  3174 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:33.922225 32043 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:18:33.925454 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73891.5us
I20260709 12:18:33.926266 32112 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:18:33.888168  2709 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.005s
I20260709 12:18:33.833680  3113 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:33.926700 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.773123  2840 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:33.740347  3261 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:33.927542 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.977748 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.962674 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:33.994799 32040 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:33.931102 31866 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 7 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 8 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:33.773798  3030 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:33.977365 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.007424 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.007634 32043 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:34.007807 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:33.930569  2800 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 12:18:33.955801  3174 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.008867 31963 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:34.008442  3264 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.009963  3272 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:18:34.010357 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.010757 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:34.022593 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.930263  3210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.034s	sys 0.079s
I20260709 12:18:34.110260 32658 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:34.074996 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1848) took 2357 ms. Trace:
I20260709 12:18:34.111757 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.111930 32043 rpcz_store.cc:276] 0709 12:18:31.673756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.007024 (+2333268us) service_pool.cc:224] Handling call
0709 12:18:34.007430 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.031205 (+ 23775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.031475 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:33.919909  3215 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.011252 31866 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:34.043097 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:33.860957  3023 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:33.923074  3257 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:34.134684 32116 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:34.172757 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.030975  2800 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:34.063906  3174 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:34.182614 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 174920us
I20260709 12:18:33.993713  3261 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:34.216378 31866 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 6 to 7, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 7 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:34.218541  3119 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:34.216378  3163 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:34.220192  3163 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.220510  3163 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:34.220384  3119 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:34.223498  3066 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:34.240371 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.999964  2709 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:34.218078  3216 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:34.271986  3216 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.272672  3216 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.272050 31866 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:34.270511 31972 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
W20260709 12:18:34.299630 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.304818 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 175062us
I20260709 12:18:34.222088  3119 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:18:34.315544 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.320528 32044 raft_consensus.cc:2368] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10.
I20260709 12:18:34.336687 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:34.349259 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 452731us
I20260709 12:18:34.313923  3191 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.359050 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:34.390093 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 695326us
I20260709 12:18:34.082154 32120 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
W20260709 12:18:34.375056 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 302070us
I20260709 12:18:34.397262 31963 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:34.389926  3210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.034s	sys 0.085s
I20260709 12:18:34.405900 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.403306 32043 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:34.406111 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:34.409473 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 191778us
I20260709 12:18:34.413980 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1852) took 2553 ms. Trace:
I20260709 12:18:34.414400 32043 rpcz_store.cc:276] 0709 12:18:31.860246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.313773 (+2453527us) service_pool.cc:224] Handling call
0709 12:18:34.398748 (+ 84975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.413703 (+ 14955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.413920 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:34.415555 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 768838us
W20260709 12:18:34.424008 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.406987  3278 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:34.403195  3113 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:34.412631  3203 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.051s	user 0.010s	sys 0.002s
I20260709 12:18:34.389592  3163 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:34.437430 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.451311 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.474582 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:34.476846 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.476975 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:34.478221 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48473.1us
W20260709 12:18:34.481400 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 144825us
W20260709 12:18:33.908250 31867 rpcz_store.cc:269] 0709 12:18:12.020202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.020505 (+   303us) service_pool.cc:224] Handling call
0709 12:18:12.132322 (+111817us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890ab0b2 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:18:33.525457 (+21393135us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:18.555673 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:19.062716 (+507043us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:19.062749 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:18:19.062767 (+    18us) tablet.cc:662] Decoding operations
0709 12:18:19.065041 (+  2274us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:19.065139 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:19.065180 (+    41us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:18:19.468655 (+403475us) tablet.cc:701] Row locks acquired
0709 12:18:19.468678 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:18:19.492446 (+ 23768us) write_op.cc:270] Start()
0709 12:18:19.507486 (+ 15040us) write_op.cc:276] Timestamp: P: 1783599499483282 usec, L: 0
0709 12:18:19.507536 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:18:19.652622 (+145086us) log.cc:844] Serialized 6614 byte log entry
0709 12:18:20.962895 (+1310273us) op_driver.cc:464] REPLICATION: finished
0709 12:18:21.535388 (+572493us) write_op.cc:301] APPLY: starting
0709 12:18:21.535493 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:21.535907 (+   414us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:21.536054 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:21.537895 (+  1841us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:21.548922 (+ 11027us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:18:21.552847 (+  3925us) write_op.cc:312] APPLY: finished
0709 12:18:21.561201 (+  8354us) log.cc:844] Serialized 119 byte log entry
0709 12:18:21.561404 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:21.561579 (+   175us) write_op.cc:454] Released schema lock
0709 12:18:21.561640 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:21.562090 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:21.562127 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":415814,"reported_tablets":49,"spinlock_wait_cycles":18022144,"child_traces":[["op",{"apply.queue_time_us":572223,"apply.run_cpu_time_us":4421,"apply.run_wall_time_us":26683,"num_ops":17,"prepare.queue_time_us":657354,"prepare.run_cpu_time_us":25410,"prepare.run_wall_time_us":1442960,"raft.queue_time_us":391807,"raft.run_cpu_time_us":14840,"raft.run_wall_time_us":147938,"replication_time_us":1453302,"thread_start_us":1069523,"threads_started":4,"wal-append.queue_time_us":638161,"wal-append.run_cpu_time_us":29496,"wal-append.run_wall_time_us":2356095}]]}
I20260709 12:18:34.410069  3191 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.520974 31963 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:34.520625 31972 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:34.479725  3030 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:34.536542 32119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2087) took 4893 ms (client timeout 2229 ms). Trace:
W20260709 12:18:34.537031 32119 rpcz_store.cc:269] 0709 12:18:29.643004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.190150 (+547146us) service_pool.cc:224] Handling call
0709 12:18:34.536089 (+4345939us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:34.536350 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142771584}
I20260709 12:18:34.537675  3191 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:34.537930  3163 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:34.520093  3264 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:34.519752  3272 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:34.539088  3272 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:34.541157  3155 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.132s	user 0.006s	sys 0.004s
I20260709 12:18:34.541499  3272 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:34.542027 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.544237 31971 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:34.544651 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.545691  3136 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:34.548494  3174 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:34.408365  3261 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.550958  3261 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:34.554677 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.555588 32656 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:34.556833 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2062) took 1139 ms. Trace:
I20260709 12:18:34.557250 31970 rpcz_store.cc:276] 0709 12:18:33.416602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.422485 (+  5883us) service_pool.cc:224] Handling call
0709 12:18:34.555489 (+1133004us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400315454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:34.555586 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419169152}
I20260709 12:18:34.555387 32047 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.558516 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:34.558816 32047 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:34.436501  3113 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.559700 32043 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:34.558706 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:34.567821 32047 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.569825 32658 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:34.453147  3203 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:34.554745  3301 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:18:34.571425 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1854) took 2705 ms. Trace:
I20260709 12:18:34.579710 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.579624 32043 rpcz_store.cc:276] 0709 12:18:31.860839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.415263 (+2554424us) service_pool.cc:224] Handling call
0709 12:18:34.473429 (+ 58166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.566450 (+ 93021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.566664 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:34.570668 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51790.8us
I20260709 12:18:34.529623 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:34.560235  3030 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.582643 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 609) took 1148 ms. Trace:
I20260709 12:18:34.584596 32658 rpcz_store.cc:276] 0709 12:18:33.426113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.470289 (+ 44176us) service_pool.cc:224] Handling call
0709 12:18:34.574297 (+1104008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:34.595435 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.583046 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:34.583815 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:34.569571  2709 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:34.601727 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.602252 32043 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:34.605046 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75892.1us
I20260709 12:18:34.606931 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1861) took 1511 ms. Trace:
I20260709 12:18:34.607389 32043 rpcz_store.cc:276] 0709 12:18:33.092849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.580431 (+1487582us) service_pool.cc:224] Handling call
0709 12:18:34.602017 (+ 21586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.603895 (+  1878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.604144 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:34.573308  3023 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:34.608914 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.583035  3030 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.615216 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:34.598413  3174 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:34.598706  3272 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:34.618822 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.621564 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.624493 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.631266 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:34.633471 32043 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:34.606151  3304 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:18:34.636188 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1862) took 1543 ms. Trace:
I20260709 12:18:34.636709 32043 rpcz_store.cc:276] 0709 12:18:33.093179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.608291 (+1515112us) service_pool.cc:224] Handling call
0709 12:18:34.623736 (+ 15445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.634043 (+ 10307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.636128 (+  2085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:34.598894  3302 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:18:34.641908 32652 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:34.641127 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.644310 32654 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:18:34.644234 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85643.5us
I20260709 12:18:34.625102  2709 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:34.648283 32652 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7.
I20260709 12:18:34.647210 32121 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:34.645355 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:34.650399 32043 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:34.650686 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 241495us
I20260709 12:18:34.653873 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1864) took 1560 ms. Trace:
I20260709 12:18:34.654295 32043 rpcz_store.cc:276] 0709 12:18:33.093993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.637796 (+1543803us) service_pool.cc:224] Handling call
0709 12:18:34.650207 (+ 12411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.651425 (+  1218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.653813 (+  2388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:34.655259  3119 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:34.658443 32651 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:34.659353 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:34.639940  3272 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:34.664179 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.665566 32654 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:34.665725 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:34.668763 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.649168  2709 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.672827 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.661103  3304 consensus_queue.cc:1243] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:18:34.679097 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.675794 31971 raft_consensus.cc:2376] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9. Re-sending same reply.
I20260709 12:18:34.687644 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:34.688112 31971 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:34.688114  3191 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:34.688719 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1850) took 3004 ms. Trace:
I20260709 12:18:34.688910  3191 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.690994  3191 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.694469  3122 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:34.694785 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:34.676594 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.248s (SENT) [suppressed 27 similar messages]
I20260709 12:18:34.695590 31972 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:34.680415 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:34.700049 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.701138 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:34.690495 32040 rpcz_store.cc:276] 0709 12:18:31.674198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.009204 (+2335006us) service_pool.cc:224] Handling call
0709 12:18:34.009563 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.009850 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:34.322481 (+312631us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.677931 (+355450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:34.677983 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:34.678023 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.678081 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.678198 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":450913,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":8649,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":8254,"thread_start_us":572789,"threads_started":2}
I20260709 12:18:34.697577 32656 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:34.697999 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.704949 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.704321 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:34.674980  2709 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.707049 32657 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:34.701488 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:34.695020  3122 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.710034 32654 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:34.715055 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:34.710067  3194 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.728397 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:34.714938  3122 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:34.733686 31967 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:34.737455 31970 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.739379 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.751659 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:34.727825  3297 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:18:34.758059 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:34.760176 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52486.4us
I20260709 12:18:34.745731  3191 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.716379  3304 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:34.727406  3113 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:34.767614 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 51298.9us
I20260709 12:18:34.751521  3203 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:34.771250 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.772228 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:34.766703  3278 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:34.796921 31971 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:34.798733 32040 pending_rounds.cc:85] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 8
I20260709 12:18:34.799192 32040 pending_rounds.cc:107] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:18:34.799508 32040 raft_consensus.cc:2889] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.808007 31971 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:18:34.810139 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 96740.4us
I20260709 12:18:34.817157 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:34.848316 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:34.847406 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:34.854384 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.856230 32118 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:34.855747 32121 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:34.860457 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:34.883690 31967 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:34.861652  2344 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:34.863224  3100 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:34.866122  3216 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.593s	user 0.014s	sys 0.000s
W20260709 12:18:34.874703 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.769972  3305 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.149s
I20260709 12:18:34.861244  3307 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.820736  3278 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:34.883052 32118 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:34.848100  3215 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.006s
I20260709 12:18:34.896797 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:34.922617 32115 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:34.897812 32118 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:34.898341 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2111) took 1049 ms. Trace:
W20260709 12:18:34.898706  3308 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:18:34.898870  2984 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:34.902420 32038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 1927) took 4357 ms (client timeout 1968 ms). Trace:
I20260709 12:18:34.902515 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1851) took 3042 ms. Trace:
I20260709 12:18:34.903115 31971 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:18:34.904230  3278 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.909070  3163 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.909277  3310 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:34.911773  2855 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:34.902487 32119 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:34.924278 32117 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
W20260709 12:18:34.924374 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 234014us
W20260709 12:18:34.926154 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 249565us
I20260709 12:18:34.853231  3203 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:34.927709 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1867) took 1820 ms. Trace:
I20260709 12:18:34.923259 32114 rpcz_store.cc:276] 0709 12:18:33.849084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.862922 (+ 13838us) service_pool.cc:224] Handling call
0709 12:18:33.879059 (+ 16137us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.325246 (+446187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.325290 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:34.389253 (+ 63963us) log.cc:844] Serialized 52 byte log entry
0709 12:18:34.645625 (+256372us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:34.645675 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.645763 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.898019 (+252256us) raft_consensus.cc:1685] finished
0709 12:18:34.898040 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.898235 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128099,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":650644,"threads_started":2,"wal-append.queue_time_us":387188}
I20260709 12:18:34.805428  3272 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:34.932853 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1793) took 1393 ms. Trace:
W20260709 12:18:34.930366 32038 rpcz_store.cc:269] 0709 12:18:30.428296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.428416 (+   120us) service_pool.cc:224] Handling call
0709 12:18:34.766964 (+4338548us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b54001a6dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:34.785604 (+ 18640us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83850624}
I20260709 12:18:34.931712 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.959971 31970 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
W20260709 12:18:34.961817 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 97813.9us
I20260709 12:18:34.930773 32046 rpcz_store.cc:276] 0709 12:18:31.859908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.129939 (+2270031us) service_pool.cc:224] Handling call
0709 12:18:34.217267 (+ 87328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.282881 (+ 65614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.282927 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:34.347899 (+ 64972us) log.cc:844] Serialized 52 byte log entry
0709 12:18:34.700185 (+352286us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:34.715766 (+ 15581us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.716191 (+   425us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.716397 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.716487 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.902205 (+185718us) raft_consensus.cc:1685] finished
0709 12:18:34.902235 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.902437 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9544,"prepare.queue_time_us":38846,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":7115,"raft.queue_time_us":41442,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":61970,"spinlock_wait_cycles":3577600,"thread_start_us":274337,"threads_started":1,"wal-append.queue_time_us":495912}
I20260709 12:18:34.963924 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:34.963850 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.966795 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.946991 32043 rpcz_store.cc:276] 0709 12:18:33.098162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.655131 (+1556969us) service_pool.cc:224] Handling call
0709 12:18:34.656179 (+  1048us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:18:34.658796 (+  2617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.658828 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:18:34.690026 (+ 31198us) log.cc:844] Serialized 104 byte log entry
0709 12:18:34.798702 (+108676us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:34.842488 (+ 43786us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.842810 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.842919 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.865259 (+ 22340us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b18003c4240. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:34.918707 (+ 53448us) raft_consensus.cc:1685] finished
0709 12:18:34.918790 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.918964 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43487,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":181,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":21448320,"thread_start_us":108301,"threads_started":1,"wal-append.queue_time_us":212460}
I20260709 12:18:34.969322 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:34.895351  3216 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.975059 32121 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:34.972481 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:34.941057  3310 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.972986 32046 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:34.951013 31966 rpcz_store.cc:276] 0709 12:18:33.531857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.774070 (+242213us) service_pool.cc:224] Handling call
0709 12:18:33.885564 (+111494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.962063 (+ 76499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.962112 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:34.141200 (+179088us) log.cc:844] Serialized 26 byte log entry
0709 12:18:34.756930 (+615730us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:34.756990 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.757171 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.919615 (+162444us) raft_consensus.cc:1685] finished
0709 12:18:34.919722 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.924833 (+  5111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62553,"raft.queue_time_us":9496,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":45110,"thread_start_us":450121,"threads_started":1,"wal-append.queue_time_us":590822}
I20260709 12:18:34.973944 32118 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:34.961102  2855 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:34.978399 32043 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:34.979104 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 390) took 1872 ms. Trace:
I20260709 12:18:34.980530 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.929371  3215 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.976195 32116 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:34.982720 32046 rpcz_store.cc:276] 0709 12:18:33.107020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.969462 (+1862442us) service_pool.cc:224] Handling call
0709 12:18:34.969836 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.978870 (+  9034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.979048 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:34.984315 32039 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:34.985544 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.987129 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:34.988173  3216 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:34.988631 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:34.989424 31967 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:34.990548 31971 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:34.992292 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1945) took 1696 ms. Trace:
I20260709 12:18:34.992609 32043 rpcz_store.cc:276] 0709 12:18:33.295816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.967877 (+1672061us) service_pool.cc:224] Handling call
0709 12:18:34.978213 (+ 10336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.992123 (+ 13910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.992254 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:34.990028 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1792) took 1474 ms. Trace:
I20260709 12:18:34.985591  2911 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:35.002818 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.003990 32115 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:35.008001 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.009673 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:34.977923  3310 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:35.016525 32043 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:34.951126  2984 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:35.020747 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71225.9us
I20260709 12:18:35.039832 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:34.972247  3203 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.993503 31965 rpcz_store.cc:276] 0709 12:18:33.515732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.691910 (+176178us) service_pool.cc:224] Handling call
0709 12:18:33.692405 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.245649 (+553244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.245682 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:34.280668 (+ 34986us) log.cc:844] Serialized 52 byte log entry
0709 12:18:34.634555 (+353887us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:34.708729 (+ 74174us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.709129 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.709344 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.709533 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.989838 (+280305us) raft_consensus.cc:1685] finished
0709 12:18:34.989855 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.989974 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101794,"prepare.queue_time_us":439244,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":128413,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":26385,"spinlock_wait_cycles":4902272,"thread_start_us":287354,"threads_started":2,"wal-append.queue_time_us":400913}
I20260709 12:18:34.962633  3307 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:34.979333  3163 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.052186 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:35.059778 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 91876.2us
I20260709 12:18:35.038004  3261 raft_consensus.cc:2764] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:35.039695  3191 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.065448 32659 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:35.065721 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:35.071909 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:35.078985 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.053969  3209 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:18:35.086925 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.087553 31970 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:35.071619  3135 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.090714 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.097237  3326 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.098143  3296 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.677s	user 0.001s	sys 0.001s
I20260709 12:18:35.097903  3326 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:35.100440 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.100572  3326 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.091853  3253 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:35.078181  3100 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:35.080470  2855 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:35.096587 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.103312 32118 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:18:35.110698 32118 pending_rounds.cc:85] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 9
I20260709 12:18:35.110972 32118 pending_rounds.cc:107] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:18:35.111229 32118 raft_consensus.cc:2889] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:35.105959 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:35.071724  3272 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:18:35.073645  3265 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:35.112366 32114 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.116917  3303 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:35.117168  3321 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:18:35.119050  3121 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.128571 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.129231 31970 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:18:35.107293  3253 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.131834 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:35.140626 32118 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.131908 32119 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 6.
I20260709 12:18:35.109939  3100 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.110229  2855 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.158466  2855 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.158972 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:35.160759 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:35.162425  2953 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.162474  3100 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.163143  2953 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.163972 32047 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.103634  3135 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.165954  3135 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.135462  3163 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:35.144600 32623 leader_election.cc:336] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.662s (SENT)
I20260709 12:18:35.145460 31967 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:35.146214 32115 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:18:35.146730 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.146836 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.149314 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:35.146355 32121 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:35.151485  3203 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.151948 32045 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.151499 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.127374  3303 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.127534  3264 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.155510  2711 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:35.157378 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.170818  2977 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.227527  2711 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:35.172084  2840 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:35.227762 31964 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.228189  2840 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.227835  2977 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.228524  3330 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:35.230012  3302 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:18:35.230682 31969 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:35.230943  3098 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:35.232666  3303 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:35.172336  3307 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.142383  3253 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:35.173054  3030 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.000s
I20260709 12:18:35.175370  3122 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.183470 31966 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:35.183827  2897 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.184389  3322 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:35.187029  3128 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:35.188045 32044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 400) took 2076 ms (client timeout 2001 ms). Trace:
W20260709 12:18:35.248190 32044 rpcz_store.cc:269] 0709 12:18:33.111839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.283865 (+172026us) service_pool.cc:224] Handling call
0709 12:18:35.187822 (+1903957us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000573d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:35.187955 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2359845632}
I20260709 12:18:35.190250 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.196753 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.199983 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.134130 32085 leader_election.cc:400] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7.
I20260709 12:18:35.249917 32085 leader_election.cc:403] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:35.208470 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.209031 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.223210 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1947) took 1926 ms. Trace:
W20260709 12:18:35.223964 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 79392.9us
I20260709 12:18:35.190287  2953 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.226861 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1832) took 3557 ms. Trace:
I20260709 12:18:35.227109 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1950) took 1929 ms. Trace:
I20260709 12:18:35.234285 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:35.236833 31964 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:35.254880 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2072) took 1133 ms. Trace:
I20260709 12:18:35.254150 32045 rpcz_store.cc:276] 0709 12:18:33.297429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.151483 (+1854054us) service_pool.cc:224] Handling call
0709 12:18:35.151783 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.226851 (+ 75068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.227054 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:35.256127 32623 leader_election.cc:336] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.001s (SENT)
I20260709 12:18:35.253453 32120 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:35.230966  3302 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:35.183506  3215 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.225582  2984 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:35.250852 32046 rpcz_store.cc:276] 0709 12:18:33.296447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.983468 (+1687021us) service_pool.cc:224] Handling call
0709 12:18:35.006852 (+ 23384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.015612 (+  8760us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:35.062080 (+ 46468us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.222383 (+160303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.222437 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.222477 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.223038 (+   561us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.223155 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9942,"prepare.queue_time_us":61263,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":785,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4078,"thread_start_us":130616,"threads_started":1}
I20260709 12:18:35.256625 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:35.249598 32657 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:35.254061 32039 rpcz_store.cc:276] 0709 12:18:31.668989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.669118 (+   129us) service_pool.cc:224] Handling call
0709 12:18:34.807036 (+3137918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.984882 (+177846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.226654 (+241772us) spinlock_profiling.cc:243] Waited 3.11 s on lock 0x7b54001a6dd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:35.226788 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7143820160}
I20260709 12:18:35.255234 31964 rpcz_store.cc:276] 0709 12:18:34.121815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.157060 (+1035245us) service_pool.cc:224] Handling call
0709 12:18:35.254825 (+ 97765us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:35.259975 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 138671us
I20260709 12:18:35.270228 32117 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:35.270673 32044 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:35.271916 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.260507  3264 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:35.260789 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1953) took 1962 ms. Trace:
I20260709 12:18:35.274427 32045 rpcz_store.cc:276] 0709 12:18:33.305537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.255685 (+1950148us) service_pool.cc:224] Handling call
0709 12:18:35.260250 (+  4565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.260450 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.260481 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.260524 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.260576 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.260630 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.260732 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.261253 32121 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:35.271431 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1951) took 1973 ms. Trace:
I20260709 12:18:35.275758 32044 rpcz_store.cc:276] 0709 12:18:33.297720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.249030 (+1951310us) service_pool.cc:224] Handling call
0709 12:18:35.270112 (+ 21082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.271137 (+  1025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.271369 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.276185 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:35.277911 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.250380  3322 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.228085  3326 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:18:35.248018  3128 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:35.290510  3262 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:35.285073 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 141728us
I20260709 12:18:35.276898 32120 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:35.294214 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.294818 31964 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:35.286720  2984 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:35.288285  2977 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.289707 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1956) took 1867 ms. Trace:
W20260709 12:18:35.274107  2709 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.000s
I20260709 12:18:35.299223 32039 rpcz_store.cc:276] 0709 12:18:33.422325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.288746 (+1866421us) service_pool.cc:224] Handling call
0709 12:18:35.289047 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.289351 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.289378 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.289444 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.289476 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.289528 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.289628 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.290241 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:35.304618 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 104712us
I20260709 12:18:35.305735 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.259262  3302 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:35.308426 31972 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:35.247767  3030 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:35.318629 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56444.8us
I20260709 12:18:35.291306  3128 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.299396  3307 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:35.328183 32045 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:35.331527 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 402) took 1964 ms. Trace:
I20260709 12:18:35.331941 32045 rpcz_store.cc:276] 0709 12:18:33.367067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.275450 (+1908383us) service_pool.cc:224] Handling call
0709 12:18:35.301165 (+ 25715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.331271 (+ 30106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.331462 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.298358  3180 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.329618 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.329736 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:35.300004  2984 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.330566 32039 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:35.284878  3322 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.336337 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1957) took 1913 ms. Trace:
I20260709 12:18:35.336802 32039 rpcz_store.cc:276] 0709 12:18:33.422702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.302484 (+1879782us) service_pool.cc:224] Handling call
0709 12:18:35.325982 (+ 23498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.335999 (+ 10017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.336265 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.339823 32121 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:35.351249 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.351720 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.324244  3163 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.357689 32045 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:35.358625 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1959) took 1935 ms. Trace:
I20260709 12:18:35.359943 32045 rpcz_store.cc:276] 0709 12:18:33.423355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.332821 (+1909466us) service_pool.cc:224] Handling call
0709 12:18:35.357459 (+ 24638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.358351 (+   892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.358563 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.309285  2709 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.338354  2984 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:35.373526 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43514.9us
W20260709 12:18:35.373638 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 67969.3us
I20260709 12:18:35.374603 32039 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:35.377187 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1960) took 1953 ms. Trace:
I20260709 12:18:35.375381 32047 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:35.377548 32039 rpcz_store.cc:276] 0709 12:18:33.423690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.337721 (+1914031us) service_pool.cc:224] Handling call
0709 12:18:35.360634 (+ 22913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.376864 (+ 16230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.377129 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:35.356580  3098 raft_consensus.cc:2764] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:35.301863  2711 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.379689 31936 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.326150  2977 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:18:35.385780 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.380024 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1860) took 2288 ms. Trace:
I20260709 12:18:35.379098 32118 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:35.383438 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.389217 31964 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:35.387250 32047 rpcz_store.cc:276] 0709 12:18:33.092412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.554913 (+1462501us) service_pool.cc:224] Handling call
0709 12:18:34.555243 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.163521 (+608278us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b5800811ad0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c12b0aa 00007f228c1ad447 00007f228c1c5fe7 00007f228c1c77de 00007f228c1d1b49 00007f228c1d2416 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4
0709 12:18:35.379677 (+216156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.379949 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4200192}
I20260709 12:18:35.389503 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:35.390292 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.390544 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.391580 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.392336 31964 raft_consensus.cc:1217] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Deduplicated request from leader. Original: 5.3->[8.4-11.5]   Dedup: 8.4->[11.5-11.5]
I20260709 12:18:35.392616 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:35.393046 31964 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [8.4-11.5]
I20260709 12:18:35.384723 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.380290 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.393841 31963 raft_consensus.cc:2376] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7. Re-sending same reply.
I20260709 12:18:35.393923 32039 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:35.395268 32039 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:35.396899 32039 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [8.4-11.5]
I20260709 12:18:35.393894 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.398392 32047 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.399509 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:35.406111 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:35.410383 32120 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.408804 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1946) took 2112 ms. Trace:
I20260709 12:18:35.412148 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:35.421515 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6144103720000658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
I20260709 12:18:35.384172  3128 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:35.424503  3337 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 12:18:35.426506 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.430164 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.356580  3180 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.359450  3100 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.413031  2977 consensus_queue.cc:1243] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:35.437412 32047 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
W20260709 12:18:35.437412 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 60750.9us
I20260709 12:18:35.436729  2800 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:35.448221 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:35.419051  2711 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.454229  2840 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.454596  3065 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.455305  3065 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.456655  2840 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.457397  2800 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:35.458541  2977 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:35.458380  2800 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.450830 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.469322 31963 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:35.474614 32114 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:35.475766 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:35.458061  3065 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:35.481828 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.466305  3135 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.471127  2689 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:35.413105  3302 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.499902  3302 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.505625 32116 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:18:35.506134  3346 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:18:35.491454  3122 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.515754 32116 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:35.511633 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:35.520460 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.520752  2709 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.534655 32120 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:35.538703 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:35.539315 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:35.543615 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44725.8us
I20260709 12:18:35.546640  3065 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:35.548810  2911 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:35.549388  2911 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:35.411962 32038 rpcz_store.cc:276] 0709 12:18:33.296123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.972658 (+1676535us) service_pool.cc:224] Handling call
0709 12:18:34.991731 (+ 19073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.049455 (+ 57724us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:35.304199 (+254744us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.408469 (+104270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.408526 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.408573 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.408644 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.408752 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28358,"prepare.queue_time_us":67219,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":22009,"thread_start_us":256735,"threads_started":2}
I20260709 12:18:35.534629  3030 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.550280  2911 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.508524  3317 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:18:35.496208  3135 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.555936  3329 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:35.559643 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1954) took 2204 ms. Trace:
I20260709 12:18:35.534049  3319 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:35.569918  3322 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.571177  3100 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:35.574411  3310 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.003s
I20260709 12:18:35.574944  3310 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.575583  3180 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:35.576821  3322 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.576997  3310 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.567142 32046 rpcz_store.cc:276] 0709 12:18:33.354936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.269819 (+1914883us) service_pool.cc:224] Handling call
0709 12:18:35.284923 (+ 15104us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:35.317921 (+ 32998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.317965 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:35.351352 (+ 33387us) log.cc:844] Serialized 78 byte log entry
0709 12:18:35.482283 (+130931us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:35.482375 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.482479 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.559383 (+ 76904us) raft_consensus.cc:1685] finished
0709 12:18:35.559404 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.559577 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4881,"raft.queue_time_us":19795,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10165,"thread_start_us":102334,"threads_started":1,"wal-append.queue_time_us":145522}
W20260709 12:18:35.582505 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.584103 32042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1844) took 3911 ms (client timeout 2965 ms). Trace:
W20260709 12:18:35.584658 32042 rpcz_store.cc:269] 0709 12:18:31.672861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.788084 (+1115223us) service_pool.cc:224] Handling call
0709 12:18:35.583699 (+2795615us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b54001a4fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:35.583876 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60183808}
I20260709 12:18:35.587507 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1948) took 2290 ms. Trace:
I20260709 12:18:35.587883 32043 rpcz_store.cc:276] 0709 12:18:33.296784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.993180 (+1696396us) service_pool.cc:224] Handling call
0709 12:18:35.011729 (+ 18549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.022607 (+ 10878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.587252 (+564645us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b540037a354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:35.587440 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10587520}
I20260709 12:18:35.589128  2344 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:35.589793  2344 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.590925  3264 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 6. Current term is 7.
I20260709 12:18:35.594895  3253 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.596130 32119 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:35.597119  3253 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:35.598928  3135 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:35.598182 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:35.603082 32118 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:35.609530 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 389) took 2509 ms. Trace:
W20260709 12:18:35.609364 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49466us
I20260709 12:18:35.567066  3329 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:35.609979 32040 rpcz_store.cc:276] 0709 12:18:33.106609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.708405 (+1601796us) service_pool.cc:224] Handling call
0709 12:18:34.734093 (+ 25688us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.871952 (+137859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.871996 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:34.872642 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 12:18:35.017554 (+144912us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:35.109036 (+ 91482us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.109456 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.109616 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.603648 (+494032us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b5400217d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:35.603680 (+    32us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.609281 (+  5601us) raft_consensus.cc:1685] finished
0709 12:18:35.609304 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.609454 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47939,"prepare.queue_time_us":84099,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":1450,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":23848,"spinlock_wait_cycles":133108608,"thread_start_us":176424,"threads_started":2,"wal-append.queue_time_us":276281}
I20260709 12:18:35.612594  3330 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:18:35.611984  3317 consensus_queue.cc:1243] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:35.614342  3307 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:35.649885 32112 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: [13.5-13.5]
W20260709 12:18:35.702831 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.694682  3174 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:35.725546 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.727269 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.727962 32121 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:18:35.724263  2344 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:35.730238 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 115154us
W20260709 12:18:35.731528 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.732049 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.734155 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:35.741616  2344 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:35.745554 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.716320  3174 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.755084 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:35.756052 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:35.756620 32651 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.744055  2855 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.581s	user 0.004s	sys 0.009s
I20260709 12:18:35.758327 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.760016  2344 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:35.764570 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:35.767848 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:35.768375 32658 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.770865 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:35.771538 32659 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.773391  2968 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:35.773200  3180 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:35.774387  3155 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:18:35.774843 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.775866  2689 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:35.776759  2689 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.777088  2689 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:35.778460  3258 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:35.779049  3258 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.779711 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.781505 32041 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:35.782208 32041 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:35.782572  3186 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.782783 32659 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:35.783103  3186 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:35.784039 32041 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1845) took 4111 ms (client timeout 2921 ms). Trace:
W20260709 12:18:35.784518 32041 rpcz_store.cc:269] 0709 12:18:31.673089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.786739 (+1113650us) service_pool.cc:224] Handling call
0709 12:18:35.783794 (+2997055us) spinlock_profiling.cc:243] Waited 2.99 s on lock 0x7b54001a5ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:35.783971 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6882731392}
I20260709 12:18:35.786064 32041 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:35.770980  3317 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.787431 32046 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:35.787883  3186 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:35.788645 32121 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.790087 32045 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:35.790619 32115 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:35.792265 32112 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:35.792769  2911 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:35.804629 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:35.805258 32045 raft_consensus.cc:2376] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12. Re-sending same reply.
I20260709 12:18:35.805806 32115 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:35.793069  3025 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:35.793696  2977 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 9.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:18:35.794560 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50948us
I20260709 12:18:35.808832 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:35.794564  3361 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:18:35.795809 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.811942 32047 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:35.729949  3329 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.813463 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:35.796834 32651 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:35.796900 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.797379  3066 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:35.797639 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.798672  3174 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:35.753450  3221 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:35.749963  3303 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.733251  2800 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:35.737562  2984 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.739521  2709 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:18:35.806913 32119 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:35.809718 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:35.795323 32654 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.809772 32045 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:35.811067 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.814476 32655 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 8: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:35.814942 32046 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:35.823277  3302 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:35.823660 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.824733  3098 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:35.769274  2855 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.824307 32046 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:35.802891  3065 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:35.826216 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:35.828054 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.829140 32047 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:35.827349  3098 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:35.824744  2709 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:35.830070  3098 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:35.777920  2689 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.836231 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:35.837116 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 357) took 1183 ms. Trace:
I20260709 12:18:35.837848 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.838178 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:35.838899 32654 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:35.837860 32655 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:35.835857 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:35.846544 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:35.848199 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:35.852880 32658 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:35.837493 31968 rpcz_store.cc:276] 0709 12:18:34.660569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.660700 (+   131us) service_pool.cc:224] Handling call
0709 12:18:34.663880 (+  3180us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:34.686770 (+ 22890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.686810 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:34.687388 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:18:35.560819 (+873431us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b180032bf60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:18:35.582332 (+ 21513us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.613872 (+ 31540us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.614140 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.614234 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.831585 (+217351us) raft_consensus.cc:1685] finished
0709 12:18:35.836923 (+  5338us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.837061 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29461,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":88,"raft.queue_time_us":2606,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":13217,"spinlock_wait_cycles":4946176,"thread_start_us":21317,"threads_started":1,"wal-append.queue_time_us":1075555}
I20260709 12:18:35.862000 31970 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:35.861992 32045 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:35.863876 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.872088 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:35.870223 32113 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: [7.4-11.6]
I20260709 12:18:35.872839 32047 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:35.831175  3065 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:35.831249  3221 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:35.874500 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2050) took 6210 ms (client timeout 1575 ms). Trace:
I20260709 12:18:35.864518 32041 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:35.877875 31969 rpcz_store.cc:269] 0709 12:18:29.652785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.655854 (+  3069us) service_pool.cc:224] Handling call
0709 12:18:35.839177 (+6183323us) spinlock_profiling.cc:243] Waited 5.25 s on lock 0x7b5400402ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:35.863172 (+ 23995us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12069100288}
I20260709 12:18:35.859054  3303 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:35.859629  3304 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:35.869968  2711 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:35.885644 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49476us
I20260709 12:18:35.888551 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.884619 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1955) took 2495 ms. Trace:
I20260709 12:18:35.881383  2689 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.890285 32044 rpcz_store.cc:276] 0709 12:18:33.389603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.276752 (+1887149us) service_pool.cc:224] Handling call
0709 12:18:35.301700 (+ 24948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.328784 (+ 27084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.328820 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.329676 (+   856us) log.cc:844] Serialized 26 byte log entry
0709 12:18:35.559529 (+229853us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:35.559566 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.559631 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.788909 (+229278us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b48000fe358. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:18:35.881792 (+ 92883us) raft_consensus.cc:1685] finished
0709 12:18:35.881815 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.884540 (+  2725us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16888,"raft.queue_time_us":17044,"raft.run_cpu_time_us":2888,"raft.run_wall_time_us":460220,"spinlock_wait_cycles":12116864,"thread_start_us":200588,"threads_started":1,"wal-append.queue_time_us":199386}
I20260709 12:18:35.896127 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:35.899379 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 107189us
I20260709 12:18:35.902716 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:35.903465 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4668909840002016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 12:18:35.903924 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4389909840001565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 12:18:35.903484  3373 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:35.904719  3373 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:35.906803  3380 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.906811  3378 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:35.908445  3378 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:35.904309 32010 connection.cc:256] RPC call timeout handler was delayed by -1.3597909840002558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260709 12:18:35.910161 32010 connection.cc:256] RPC call timeout handler was delayed by -1.250128327000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 12:18:35.910584 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1916283269999894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 12:18:35.911449  2344 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:35.911818 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1284909840001092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
W20260709 12:18:35.912753 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1167909840000902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 12:18:35.884487  3304 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:35.910661  3375 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.913321 32010 connection.cc:256] RPC call timeout handler was delayed by -1.1106283270000858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
I20260709 12:18:35.897001  3326 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:35.892238  3215 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:35.896796  3365 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.168s	user 0.002s	sys 0.001s
I20260709 12:18:35.900594  3135 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:35.893807  3303 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:35.918493 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:35.935122 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.931118 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.929178 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:35.915194  3326 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:35.938575  3326 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:35.938666 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.939736 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.880352  3098 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.942221  3098 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:35.943697  3362 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:18:35.945041  3369 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:35.936322 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:35.946094 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.915339  3304 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.950448 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.952147 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.955158 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.858253  3371 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
W20260709 12:18:35.961743  3368 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:18:35.963125  3370 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:35.963665  3370 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:35.925370  3221 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.964095  3370 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.884018  3379 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:18:35.967056  3194 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:35.967408  3194 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:35.975867 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.971984 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:35.978616  3317 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:35.978794 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.979195  3317 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:35.980055  3317 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:35.926999  3065 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.981921 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:35.946646  2711 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:36.024430 32042 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.954556  3210 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:36.027534 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:35.907472  3380 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:36.036334 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.037298 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65386.3us
I20260709 12:18:36.039624 32010 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10.
I20260709 12:18:36.039825 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:35.905889  3373 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.909552  3378 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.5, Last appended by leader: 5, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:35.922704  3375 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.044921 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:36.027041  3215 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:36.034816  2984 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:36.028607  3376 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:36.033730  2897 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:36.062253 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 253628us
I20260709 12:18:36.065714 31964 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:36.052929  3030 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:36.069509 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.074728 32114 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:36.075560 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.078003 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.078421 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.057564  2855 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:36.064284  2711 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:36.083618 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.070168  3030 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:36.088127  2800 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:36.088594  2800 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:36.088682  3388 raft_consensus.cc:2764] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:36.089401  2800 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.090768  2953 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:36.091135  2953 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:36.087172  2711 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:36.091696  2711 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.091933  2953 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.093257  2953 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:36.088188  3030 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.099493  3373 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.087958  3377 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:18:36.101533  3388 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:36.101925  3388 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:36.102802  3388 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.104452  2800 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.106137 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:36.106448  3100 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:36.106668 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.106978  3100 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.107362 32114 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:36.107661 32658 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:36.108386 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:36.109141  3100 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:36.109472 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.110636  3387 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:36.054971  3303 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:36.117633 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.118788 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.118438  3194 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.119674  3194 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.119939 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.121035 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.099501  3380 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:36.042249 32120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:36.123603 32120 raft_consensus.cc:606] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:36.123978 32120 raft_consensus.cc:614] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:36.125085 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.126170  3370 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.129925  3370 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:36.070604  3376 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:36.131441 32043 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:36.131953  3376 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.133049 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 1995) took 1127 ms. Trace:
I20260709 12:18:36.133539 32118 rpcz_store.cc:276] 0709 12:18:35.006066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.431986 (+425920us) service_pool.cc:224] Handling call
0709 12:18:35.602834 (+170848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.603678 (+   844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.132836 (+529158us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540010efd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:36.132979 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392110976}
I20260709 12:18:36.136070 32658 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:36.077662  3374 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.059s
W20260709 12:18:36.137251 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.137390  3374 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:36.138332 32655 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:36.106820 32652 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:36.121985 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:36.154763 32114 raft_consensus.cc:2410] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:18:36.161934 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83594us
I20260709 12:18:36.162901 32120 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:36.165833 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 180604us
I20260709 12:18:36.172405 32113 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:36.172055 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:36.174940 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:36.066434  2911 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:18:36.182175 32114 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:18:36.187992 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 104497us
I20260709 12:18:36.188884 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:36.178400 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:36.217056 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 54069.1us
I20260709 12:18:36.177001  3385 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:36.173659  3155 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:18:36.177620  2977 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:18:36.256347 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2126) took 1108 ms. Trace:
I20260709 12:18:36.177313  3326 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.185593  3066 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:18:36.256839 32117 rpcz_store.cc:276] 0709 12:18:35.148465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.151268 (+  2803us) service_pool.cc:224] Handling call
0709 12:18:36.256150 (+1104882us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b5400139a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:36.256282 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113079552}
I20260709 12:18:36.230445  3253 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:36.225311  3385 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.250608  3370 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:18:36.264424 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.234349  3384 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:18:36.272807  3258 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:36.275453  3378 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:36.189003  3215 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.280042  3210 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:36.280496  3210 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.282678  3163 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:36.283133  3163 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:36.284039  3163 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.284999  3317 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:36.285416  3163 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:36.288280  3066 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:18:36.288829 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.285411  3317 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:36.290868  3317 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.250455  3374 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:36.295465  3374 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:36.295646  3258 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:36.296152  3258 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.296402  3374 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.299516  3326 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:36.300179 32121 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:36.309562 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 136075us
I20260709 12:18:36.310774 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.315182  3379 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:36.315588  3307 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:36.315537  2711 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.316294  3379 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:36.316653  3386 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:36.317060  3386 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:36.317937  3386 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.312686 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.318262 32655 raft_consensus.cc:1217] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[9.8-12.10]   Dedup: 9.8->[10.9-12.10]
I20260709 12:18:36.319383 32118 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:36.321722 31970 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:36.322047 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.329088 31964 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:36.323195 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.257411  2977 consensus_queue.cc:1243] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:18:36.330796 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.336755 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.327091 32655 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: [9.8-12.10]
I20260709 12:18:36.321003 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.339236 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:36.344141 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.345141 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.346552 32117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:36.351222 31938 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.267306  3385 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.319130  3386 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.322208  3391 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:36.316327  3307 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:36.269384  3375 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:36.363585 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:36.399302 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:36.384416 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 120148us
I20260709 12:18:36.401249 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:36.404392 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 215583us
I20260709 12:18:36.414726 32119 raft_consensus.cc:1217] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[8.4-12.6]   Dedup: 8.4->[10.5-12.6]
I20260709 12:18:36.415418 32119 pending_rounds.cc:85] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:18:36.331951  2711 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:36.415666 32119 pending_rounds.cc:107] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 12:18:36.425747 32119 raft_consensus.cc:2889] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.407562 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:36.414572 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:36.355515  3379 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:36.359056  3258 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:36.358767  3303 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:36.360406  3066 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:36.396996  3307 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.397504  3391 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.452450 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:36.469357 31964 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
W20260709 12:18:36.452135 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52887.7us
I20260709 12:18:36.414666  3384 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:36.472596 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:36.476194 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.474049  3258 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:36.477448 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.477706  3258 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.478083  3258 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.479506 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.480247 31972 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:36.480170  3380 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:36.481056  3186 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:36.481777  3186 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:36.482445  3380 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:36.482617 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:36.482822  3380 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:36.482867 31963 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 12:18:36.483202 32041 raft_consensus.cc:1813] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Handling vote request from an unknown peer 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.483866 32041 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:36.483783  3380 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.485040  3380 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.486979 32047 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:36.488114 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:36.488911 32043 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:36.489799 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:36.490682 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:36.427732  3375 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:36.414956  3098 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:36.475199 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.359946  3025 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 12:18:36.473371 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.004s	sys 0.012s
I20260709 12:18:36.515024 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.544320 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:36.514961 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 113811us
I20260709 12:18:36.530898 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:36.542380 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.004s	sys 0.012s
I20260709 12:18:36.546942 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.553543 31967 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
W20260709 12:18:36.460790  2689 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.005s
W20260709 12:18:36.545495 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 641621us
I20260709 12:18:36.554982 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:36.557493 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:36.472946  2711 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:36.564363 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:36.564381 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:36.473876  3371 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:36.471051  3391 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.568133 32047 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.569713 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:36.578454 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 469712us
I20260709 12:18:36.581869 31965 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:36.516167  3384 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:36.449144  3379 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.475042  3155 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 12:18:36.492455  3370 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 12:18:36.511850  3373 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:18:36.571954  2711 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:36.610370 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 295792us
I20260709 12:18:36.559950  3098 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.579620  3186 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:36.584296  3371 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:36.631026 32047 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
W20260709 12:18:36.640937 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 54934.4us
I20260709 12:18:36.642222 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.573961  2855 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:18:36.626919  2711 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:36.634372  2800 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 12:18:36.592700  3302 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:18:36.691617 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49495us
W20260709 12:18:36.698324 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.709206 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 345020us
I20260709 12:18:36.654506  3396 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.723973 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.710032 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.725663  2800 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:36.703630  3194 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.001s
W20260709 12:18:36.711158  3304 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.003s
I20260709 12:18:36.696408  3303 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:36.648007  3186 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.726184  2800 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.750926  3128 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:36.751999  3135 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:36.756201  3145 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:36.756548  2709 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:36.756970  2344 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:36.806028  2344 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:36.767323  3030 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.003s
I20260709 12:18:36.768188 31971 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:18:36.770470 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.771114 31970 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 13 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 4.
I20260709 12:18:36.772004 32659 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:36.772163  3379 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:36.772672  3066 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:36.776170  3388 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.818634  3030 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.821647  3030 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:36.776249 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52478.9us
I20260709 12:18:36.824282 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.825201  3215 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:36.784015 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.803992  3145 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:36.812841 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 232721us
I20260709 12:18:36.814430  2344 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.804548  3135 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.793947  2800 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.768252  3329 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.825584 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:36.841320  3398 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:18:36.865818 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:36.866832 32041 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:36.792595  3194 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:36.734676  3396 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.851192  2344 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.802488  3378 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:36.809335  2709 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:36.818573 32013 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.670s (SENT)
I20260709 12:18:36.881033 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:36.869372 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43865.8us
I20260709 12:18:36.882141 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.749332  3304 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.870857 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:36.816290  3066 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:36.876348 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:36.852471 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 492778us
I20260709 12:18:36.903028 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:36.907217 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.908565 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:36.841807  3374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.001s	sys 0.016s
I20260709 12:18:36.847347  3098 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:36.807633  3128 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.896585 32047 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
W20260709 12:18:36.897816 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 79276.3us
I20260709 12:18:36.878161 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:36.910791 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.889446 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.845115  3174 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:18:37.009495  3174 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:37.010138  3174 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.010761  3066 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.012126  3066 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:36.920569 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:36.880849  3395 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:36.964728  3406 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:36.931515  2689 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:37.013024  3395 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:37.013526  2689 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:36.806130  3371 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.015048 32047 pending_rounds.cc:85] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 3
I20260709 12:18:37.015412 32047 pending_rounds.cc:107] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:18:37.015668 32047 raft_consensus.cc:2889] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.901722  3215 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:36.966154 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:37.019557 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:37.020664 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:36.966885 32113 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:37.022156  3065 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.965731 32121 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:18:37.022869 32121 pending_rounds.cc:85] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:18:37.023173 32121 pending_rounds.cc:107] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:37.023401 32121 raft_consensus.cc:2889] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.919741  3180 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:36.942760 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.935537 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.747929  3384 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.014061  3395 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:36.896581  2709 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.013028  3406 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:37.026566 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 148482us
I20260709 12:18:37.072233 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:36.919024  3210 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.003s
I20260709 12:18:37.027712 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:36.972185  3374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.001s	sys 0.017s
W20260709 12:18:37.073735 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46118.5us
W20260709 12:18:37.074092 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 153615us
I20260709 12:18:37.074606 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:37.074821 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:37.028368 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.028409 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 109863us
I20260709 12:18:36.979339 32042 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:37.077047 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.056911 31869 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 6 to 7, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:37.072413 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:36.919498  2977 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:36.920352  3303 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:36.922770  3326 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:36.963122  2984 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:36.881139  3194 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:36.970481  3317 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
W20260709 12:18:37.057024  3258 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.004s
I20260709 12:18:37.113049 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:37.078712 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:36.978400  3378 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:36.928573  3330 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.140s
I20260709 12:18:37.030274  3098 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:36.923118  2855 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.151s
I20260709 12:18:37.020581  3391 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.068709  3392 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:37.123307 32118 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:37.101658  3377 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:37.099064  3215 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:37.103356  2984 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:37.160156 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83168.5us
I20260709 12:18:36.924680  3145 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.166421 31869 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 8, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 8 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:37.121416  3174 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:37.143033  3370 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.004s	sys 0.024s
I20260709 12:18:37.174790 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1981) took 1262 ms. Trace:
I20260709 12:18:37.080073  3210 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.098906  2977 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.144851  3392 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:37.190436 31869 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:37.184473 32038 rpcz_store.cc:276] 0709 12:18:35.900812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.900997 (+   185us) service_pool.cc:224] Handling call
0709 12:18:35.917874 (+ 16877us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.056698 (+138824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.056727 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.072394 (+ 15667us) log.cc:844] Serialized 52 byte log entry
0709 12:18:36.416356 (+343962us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:36.499910 (+ 83554us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.500124 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.500216 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.162690 (+662474us) raft_consensus.cc:1685] finished
0709 12:18:37.162712 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.162944 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28931,"prepare.queue_time_us":83316,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":105995,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":22948,"thread_start_us":332947,"threads_started":2,"wal-append.queue_time_us":458915}
W20260709 12:18:37.184092  3370 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.004s	sys 0.025s
I20260709 12:18:37.117419  3317 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.176060  3186 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.138229  3378 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:37.205806 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.011s	sys 0.047s
W20260709 12:18:37.225062 31866 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 243) took 21030 ms (client timeout 15000 ms). Trace:
I20260709 12:18:37.202108 32120 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:37.152288  3098 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:37.206084 32116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.031s	sys 0.025s
I20260709 12:18:37.121104  3258 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.234975 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:37.185743  3400 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:37.178237  3414 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.200315 31869 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:37.225111 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.012s	sys 0.047s
I20260709 12:18:37.189744  2977 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.189590  3030 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:37.231246 32116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.032s	sys 0.025s
I20260709 12:18:37.244338 32113 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:37.211848  3145 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.235704 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:37.240939 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:37.242177 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.250484 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 894178us
I20260709 12:18:37.194348  3174 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 12:18:37.225715 31866 rpcz_store.cc:269] 0709 12:18:16.201531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.231741 (+ 30210us) service_pool.cc:224] Handling call
0709 12:18:37.224978 (+20993237us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:34.862176 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:34.862996 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:34.863050 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:18:34.863071 (+    21us) tablet.cc:662] Decoding operations
0709 12:18:34.864346 (+  1275us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:34.864455 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:34.864495 (+    40us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:18:34.981979 (+117484us) tablet.cc:701] Row locks acquired
0709 12:18:34.985773 (+  3794us) write_op.cc:260] PREPARE: finished
0709 12:18:34.985948 (+   175us) write_op.cc:270] Start()
0709 12:18:34.986114 (+   166us) write_op.cc:276] Timestamp: P: 1783599514985880 usec, L: 0
0709 12:18:34.986132 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:18:34.986890 (+   758us) log.cc:844] Serialized 4703 byte log entry
0709 12:18:35.446195 (+459305us) op_driver.cc:464] REPLICATION: finished
0709 12:18:35.573228 (+127033us) write_op.cc:301] APPLY: starting
0709 12:18:35.608823 (+ 35595us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:35.620215 (+ 11392us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:35.620382 (+   167us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:35.669477 (+ 49095us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:35.669841 (+   364us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:18:35.669888 (+    47us) write_op.cc:312] APPLY: finished
0709 12:18:35.670374 (+   486us) log.cc:844] Serialized 89 byte log entry
0709 12:18:35.670615 (+   241us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:35.670826 (+   211us) write_op.cc:454] Released schema lock
0709 12:18:35.670913 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:35.671439 (+   526us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:35.671471 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7639138,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":112283,"apply.run_cpu_time_us":5245,"apply.run_wall_time_us":112593,"num_ops":12,"prepare.queue_time_us":301803,"prepare.run_cpu_time_us":16914,"prepare.run_wall_time_us":212576,"raft.queue_time_us":218306,"raft.run_cpu_time_us":2614,"raft.run_wall_time_us":141999,"replication_time_us":447410,"thread_start_us":547264,"threads_started":4,"wal-append.queue_time_us":158728,"wal-append.run_cpu_time_us":24242,"wal-append.run_wall_time_us":1497653}]]}
I20260709 12:18:37.235690 32651 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:37.194536  3215 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:18:37.251941 32045 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:37.242033 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 431864us
W20260709 12:18:37.285960 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.198328  3418 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:18:37.244525  3304 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.304629 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:37.308295 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:37.286054  3396 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.287631  3400 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:37.287663  2953 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:18:37.252516  3030 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.299302  3128 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.306864  2689 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.289744  3420 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:18:37.246915  2984 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.317476  3423 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.340760 32044 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:37.342765 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.326256  3303 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.320325  3407 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:37.332051  3128 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.319870  3304 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:18:37.357981  2344 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.367527  2344 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.297371  3392 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.403257 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:37.404049 32659 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:37.314144  3417 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:37.412431  2344 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.412542 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:37.358645  2984 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:37.418093 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 343318us
I20260709 12:18:37.420691 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:37.423045 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 262020us
I20260709 12:18:37.345780  3194 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.431080 32038 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:37.431591 32038 pending_rounds.cc:85] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 5
I20260709 12:18:37.431854 32038 pending_rounds.cc:107] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:37.432140 32038 raft_consensus.cc:2889] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:37.365818  3180 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:37.361544  3374 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:37.424603 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 352231us
I20260709 12:18:37.365639  3387 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.454671 32045 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:37.353987  3423 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:37.492777 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.413816  3210 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:37.408964 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.416123  3385 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.023s	user 0.015s	sys 0.000s
W20260709 12:18:37.518500  3421 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:18:37.518972  3385 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.520845 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2025) took 1448 ms. Trace:
I20260709 12:18:37.521780  3385 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.477254  3434 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.525162  3434 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:37.525597  3386 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.123s	user 0.008s	sys 0.002s
W20260709 12:18:37.527673  3163 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.242s	user 0.012s	sys 0.000s
I20260709 12:18:37.528380  3025 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.528882  3025 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.531231  3423 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:18:37.521399 32115 rpcz_store.cc:276] 0709 12:18:36.072624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.145187 (+ 72563us) service_pool.cc:224] Handling call
0709 12:18:36.145481 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.179176 (+ 33695us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:36.222903 (+ 43727us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.834884 (+611981us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18002cdbe0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:37.200226 (+365342us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:37.212151 (+ 11925us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.212319 (+   168us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:37.212354 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.212642 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.520504 (+307862us) raft_consensus.cc:1685] finished
0709 12:18:37.520599 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.520773 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66066,"prepare.queue_time_us":70301,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":79145,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":31859,"spinlock_wait_cycles":4592896,"thread_start_us":344611,"threads_started":1,"wal-append.queue_time_us":1086518}
W20260709 12:18:37.535138  3419 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:18:37.538132  3258 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.539970 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:37.540730 32115 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:37.543963  3434 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.439355 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.366438  3422 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.415611  3391 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.619395  3422 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.620000  3422 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:37.505142 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.621569 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:37.425449 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:37.622803 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 336900us
W20260709 12:18:37.507877 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 82044.5us
I20260709 12:18:37.624300 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:37.508934  3413 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 12:18:37.629799 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 204466us
I20260709 12:18:37.629721 31966 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:37.631820 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.502974  3317 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.635015  3391 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:37.636047  3391 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:37.636449  3391 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:37.510880  3128 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:18:37.590458  3098 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.591095  3406 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:37.629529  3380 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.144s	user 0.010s	sys 0.005s
W20260709 12:18:37.592870  3416 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
W20260709 12:18:37.593343  3415 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.392s	user 0.001s	sys 0.000s
I20260709 12:18:37.637496  3391 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.472358  3411 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:37.447486  3400 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:37.480448  3307 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.005s	user 0.012s	sys 0.000s
I20260709 12:18:37.480692  3387 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.474231  3180 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.614684 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.674646 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.676770 32045 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:37.674124 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 234844us
I20260709 12:18:37.679365 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:37.629288  3135 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.000s
I20260709 12:18:37.661988 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:37.640328  3417 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.479251  3374 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.695626 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:37.663786 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1886) took 1192 ms. Trace:
I20260709 12:18:37.641139  3388 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:37.656493  3186 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:37.668286  2711 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.956s	user 0.013s	sys 0.000s
I20260709 12:18:37.707098 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:37.707728 31972 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:37.679662 32038 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:37.592370  3030 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.673736  3098 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:37.693849 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:37.639752  2897 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:18:37.697489 31970 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:18:37.699246  2800 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.000s
I20260709 12:18:37.700160 31971 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:37.699091 32039 rpcz_store.cc:276] 0709 12:18:36.471636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.482039 (+ 10403us) service_pool.cc:224] Handling call
0709 12:18:36.482342 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.499542 (+ 17200us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:36.499942 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.660388 (+1160446us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.662115 (+  1727us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.662360 (+   245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:37.662415 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:37.662480 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.662566 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.663711 (+  1145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25472,"prepare.queue_time_us":295738,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":17156,"raft.queue_time_us":89200,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10570,"spinlock_wait_cycles":15301376,"thread_start_us":1158284,"threads_started":3,"wal-append.queue_time_us":741071}
I20260709 12:18:37.701136 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:37.663533  3025 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.696328 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:37.682650  3155 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:37.705037 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2031) took 1364 ms. Trace:
I20260709 12:18:37.723018 32119 rpcz_store.cc:276] 0709 12:18:36.340678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.340828 (+   150us) service_pool.cc:224] Handling call
0709 12:18:36.365637 (+ 24809us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:36.436652 (+ 71015us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:36.436698 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.437417 (+   719us) log.cc:844] Serialized 52 byte log entry
0709 12:18:36.895747 (+458330us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:36.895787 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.895857 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.704830 (+808973us) raft_consensus.cc:1685] finished
0709 12:18:37.704849 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.704981 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45711,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":54604,"thread_start_us":333083,"threads_started":1,"wal-append.queue_time_us":711389}
I20260709 12:18:37.713650  3391 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.682833 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:37.736441 32043 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.711274 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.714454 31969 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.702616  3396 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:37.716109 32046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.007s	sys 0.009s
I20260709 12:18:37.720199 32044 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.663460  3437 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.740219  3409 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:37.740573  3437 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.741035  3437 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.740859  3409 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.703929  3417 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.742132  3417 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.742372  3376 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:37.742875  3376 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:37.709631  3128 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:37.734154  3066 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.722s	user 0.014s	sys 0.000s
I20260709 12:18:37.683379  3307 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.696832  3135 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.713855  3098 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:37.743902  3329 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.873s	user 0.006s	sys 0.007s
I20260709 12:18:37.674108  3411 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.682741  3428 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:18:37.674201  3406 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:18:37.709285  3386 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 12:18:37.739485 32046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.008s	sys 0.009s
I20260709 12:18:37.714530  3388 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:37.714144  2711 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.721082  2800 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.763023 31964 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:37.642706 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:37.775620 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 132955us
I20260709 12:18:37.764107  3307 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.764834 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:37.778631 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:37.764832 32087 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.000s	sys 0.024s
I20260709 12:18:37.770341 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.730229  3155 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.750510  3128 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:37.778198 32041 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:37.778538 32117 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:37.786640 32117 raft_consensus.cc:614] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:37.787331 32112 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:18:37.787752 32112 pending_rounds.cc:85] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 6
I20260709 12:18:37.788086 32112 pending_rounds.cc:107] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:18:37.788403 32112 raft_consensus.cc:2889] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:37.768153  3030 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:37.789309  3030 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.789778  3411 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:37.780952 32087 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.000s	sys 0.025s
W20260709 12:18:37.767715  2709 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.668s	user 0.006s	sys 0.007s
I20260709 12:18:37.785015 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.759374  3396 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:37.792233 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:37.822252 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52012.1us
I20260709 12:18:37.821375 31965 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:37.826809 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.828291 31967 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.775130  3317 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.774940  3330 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.773507  3186 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.833416 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:37.777715  3210 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:37.819412  3145 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.002s
I20260709 12:18:37.791287  3128 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:37.839660 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:37.791790  3326 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.793350  3436 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.840637 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.861254 32047 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:37.791975  2897 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.793085  3370 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:37.839527  3030 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.828328  2709 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.848611 32654 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:18:37.876812 32654 pending_rounds.cc:85] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 9
I20260709 12:18:37.877110 32654 pending_rounds.cc:107] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:18:37.850229  3440 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:37.859731 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:37.863248 31969 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:37.836993  3384 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.864909 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:37.872465 31967 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:18:37.840888  2977 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 12:18:37.848464  3180 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.877597 32654 raft_consensus.cc:2889] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:37.878196 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.894738 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:37.879084 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:37.881914 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 149372us
I20260709 12:18:37.825805  3411 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:37.883729 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:37.887358 31968 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:37.892724 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:37.895748 31963 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 7: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:37.861161  3374 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.873387  3210 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:37.922251 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:37.924129 31963 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: [9.8-12.10]
I20260709 12:18:37.836321  3317 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.840678  3186 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.940243 31963 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:18:37.881309  3258 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:18:37.896653  2977 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:37.941630  3399 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.180s
I20260709 12:18:37.942436  3427 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.942881  3427 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.840492  3330 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.943259  3427 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.943583  2977 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.944615 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:37.944895  3330 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.945489  3330 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:37.920063  3411 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.919420  3388 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:37.947091  3442 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:18:37.947404  2709 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:37.947968  3411 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.948477  3411 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.948549  2711 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:37.948770 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1987) took 1081 ms. Trace:
I20260709 12:18:37.866212  3326 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.941928  3453 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:37.950883 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:37.945837 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:37.984347 31963 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:37.987525 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:37.988708 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:37.989519 32119 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.949218 32040 rpcz_store.cc:276] 0709 12:18:36.867706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.879238 (+ 11532us) service_pool.cc:224] Handling call
0709 12:18:36.910602 (+ 31364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:37.236500 (+325898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.236546 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:37.245616 (+  9070us) log.cc:844] Serialized 52 byte log entry
0709 12:18:37.624282 (+378666us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:37.656353 (+ 32071us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.656584 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.656658 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.948568 (+291910us) raft_consensus.cc:1685] finished
0709 12:18:37.948587 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.948716 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":927,"prepare.queue_time_us":31862,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1757,"raft.queue_time_us":89445,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":3319,"spinlock_wait_cycles":984960,"thread_start_us":627130,"threads_started":2,"wal-append.queue_time_us":411941}
I20260709 12:18:37.839977  3098 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:37.994851 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 168106us
I20260709 12:18:37.996037 32047 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:18:37.923758  3387 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.001545 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.997946 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:37.939639  3135 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:38.028576  3449 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 12:18:37.990648  3396 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.876832  3436 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:37.949136  2711 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.945930  3330 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.009987 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:38.011979  3186 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:37.984194  2977 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.018744  3456 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:38.019410  3444 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 12:18:38.019706  3329 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:37.988881  3374 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:38.020740  3155 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.021903  3462 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.984959  3428 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.021834 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.057118  3428 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.057837 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.058791 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.059381  3428 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.059804 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:38.023312  3441 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:18:38.023412 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:38.027055  3452 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:18:38.060804 31964 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:38.027798  3446 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:18:37.937199  3210 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:38.061462  3465 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.062173  3465 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:38.062359  3210 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.063251  3462 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:38.062981  2977 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.064332 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:37.949018  3411 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.047361  3456 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.065642 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.066504  2855 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.066860  2855 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:38.067068  3456 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.067233  2855 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.067704  3456 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:38.048403 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2015) took 2861 ms. Trace:
I20260709 12:18:38.069087 32112 rpcz_store.cc:276] 0709 12:18:35.187064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.804500 (+617436us) service_pool.cc:224] Handling call
0709 12:18:35.816486 (+ 11986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.788821 (+1972335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.048113 (+259292us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b540012b1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:38.048324 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4531693312}
W20260709 12:18:38.076434  3408 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.771s	user 0.004s	sys 0.005s
I20260709 12:18:38.056242  3459 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.062770  3465 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:38.029696 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1988) took 1161 ms. Trace:
I20260709 12:18:38.032325 32044 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:38.086649 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:38.087296 32044 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:38.088047 32119 raft_consensus.cc:2410] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:38.065132  2977 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.067912 31972 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:38.049677 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 1991) took 1180 ms. Trace:
I20260709 12:18:38.044938  2711 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.084033 31965 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:18:38.084697 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.085057 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:38.088953 32043 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:38.104799 32040 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:38.107604 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 211813us
I20260709 12:18:38.102038 32041 rpcz_store.cc:276] 0709 12:18:36.869061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.033034 (+163973us) service_pool.cc:224] Handling call
0709 12:18:37.033504 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.791304 (+757800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.049366 (+258062us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54000569d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:38.049605 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712048000}
I20260709 12:18:38.085973 32046 rpcz_store.cc:276] 0709 12:18:36.868197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.879035 (+ 10838us) service_pool.cc:224] Handling call
0709 12:18:36.906830 (+ 27795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.916729 (+  9899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.916761 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.917376 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.772327 (+854951us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:37.772382 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.772473 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.029456 (+256983us) raft_consensus.cc:1685] finished
0709 12:18:38.029487 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.029636 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92619,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":728835,"threads_started":1,"wal-append.queue_time_us":775602}
I20260709 12:18:38.111135 31964 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:38.107789 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:38.111375 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:38.111737 32044 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.110397 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:38.080078  3434 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.111174 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.117414 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:38.118939 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:38.119104 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:38.121572 32040 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:38.085891  3155 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:38.086217  2800 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.065124  3462 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.105659  3374 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:38.130064 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.130347 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.129236  3465 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.105505  3428 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.131835 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.018253  3417 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.134970 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:38.135648 31972 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:38.135299 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89207.9us
I20260709 12:18:38.115445  3440 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:38.136767 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:38.141049 31965 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:38.142474 32121 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: [7.4-11.6]
I20260709 12:18:38.090497  3459 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.142059 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 624) took 1654 ms. Trace:
I20260709 12:18:38.145195 32044 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:38.145924 31972 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:38.146166 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.152204  3128 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.154917  3303 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:38.157301 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:38.158109 31965 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:38.144474 32652 rpcz_store.cc:276] 0709 12:18:36.482570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.486415 (+  3845us) service_pool.cc:224] Handling call
0709 12:18:36.486693 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.487037 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.487064 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.487509 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:18:36.905074 (+417565us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:36.905114 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.905180 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.075657 (+1170477us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:38.132121 (+ 56464us) raft_consensus.cc:1685] finished
0709 12:18:38.132142 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.137224 (+  5082us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1063,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":27888896,"thread_start_us":417193,"threads_started":1,"wal-append.queue_time_us":737465}
I20260709 12:18:38.136725  3428 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.130944  3155 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.158079 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.160769 32112 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:38.160988  2711 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:38.161798  3155 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.136336  3465 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.169042 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:38.199664 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.145643  3437 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.226518 32120 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.166571  3330 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:38.203107  3459 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.159960  3128 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.206760  3155 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.148777  3388 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:18:38.210945  3461 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:38.245487 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.146806  3186 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:38.214671 32114 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:38.215629  3440 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.216758  3468 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:38.218094  3370 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:38.218943  2855 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.222654  3417 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.223443 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.214776 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.263253 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:38.264351  2977 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:38.265026 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:38.245297 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.247229 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.267195 32659 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:38.247455 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.268601 31936 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.247944 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:38.134441  3374 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.275380  3437 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.276690  3374 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.252792 32653 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:38.166798  3395 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.254096 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.255702 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.278510 32113 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:38.257050 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:38.258921 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:38.271828  3386 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.260842  3186 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.254796  3155 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.272923  3465 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:38.273064  3391 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.002s
I20260709 12:18:38.252405  2800 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:18:38.268666 32651 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
W20260709 12:18:38.275022  3410 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.971s	user 0.007s	sys 0.000s
I20260709 12:18:38.280262 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.280275 32654 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.273782  3128 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:38.254472  3420 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:18:38.323496 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.284674  3374 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.302234  3377 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:38.302924 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 98881.9us
W20260709 12:18:38.308439 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57454.9us
I20260709 12:18:38.315478 32120 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:38.319240 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.254326  3307 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:18:38.334128 32653 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:38.341672 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.334936 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.340617 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:38.343641 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 169334us
I20260709 12:18:38.344558 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:38.258549  3317 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:38.304093  3468 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:38.340279 31969 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:38.348915 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.349550 32660 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:38.328050  3377 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:38.324100  3386 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:38.351743 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:38.329339  3374 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.328850  3478 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:38.350832  3128 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:38.350024  3458 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.349242  3391 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.363289 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:38.355641  3386 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.369006 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.360350  3471 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:38.361372  3374 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.375962 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:38.381443 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:38.383217 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.383208 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:38.369659  3386 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.367445  3391 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:38.374497  3471 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:38.387673  3471 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:38.387825  3391 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.388640  3253 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:38.389055  3253 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:38.378494  3458 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:38.387698  3467 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:18:38.393306  3326 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:38.370329  3377 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.392894 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:38.396510 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:38.398881 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.375782  3475 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:38.414775  2855 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:38.416172  3478 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:38.416507 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 826cffce8bb540c8beb56cc00a702782 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42276
I20260709 12:18:38.416790  3155 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.417348  3478 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:18:38.418279  3451 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.499s	user 0.000s	sys 0.002s
I20260709 12:18:38.420248  3066 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:38.420747  3066 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:38.422932  3427 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.424064 32041 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:38.421736  3066 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.426399  2977 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:38.426831  3066 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.428572 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37262 (request call id 13) took 2082 ms. Trace:
I20260709 12:18:38.428776  3307 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:38.429258  3307 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.429627  3186 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.429785  3307 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:38.431242  3464 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.287s	user 0.002s	sys 0.000s
I20260709 12:18:38.407285 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.428964 32117 rpcz_store.cc:276] 0709 12:18:36.346149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.346368 (+   219us) service_pool.cc:224] Handling call
0709 12:18:38.428369 (+2082001us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540012b1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1c9853 00007f229119760a 00007f228a71d7d4 00007f228a71d762 00007f228a71d691 00007f228a71d60c 00007f228a71c8c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:38.428488 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3291219840}
I20260709 12:18:38.444283 32654 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:38.445200 32659 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:38.448041 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.448848 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.389926  3253 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.464553 32113 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:38.464614  3253 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.467195  3427 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.460311  2977 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:38.471529  2977 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:38.471468 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:38.472520  2977 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.404157  3180 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:38.417366  3411 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:38.479045  3303 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:38.446456  3418 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.009s	sys 0.015s
I20260709 12:18:38.483206 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.433211  3475 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:38.485631  3418 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.178s	user 0.009s	sys 0.016s
I20260709 12:18:38.462672  3155 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:38.494626 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 121663us
I20260709 12:18:38.498582 32114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:38.499236 32114 raft_consensus.cc:606] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:38.499576 32114 raft_consensus.cc:614] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:38.501600  3487 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.495369  3420 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:18:38.502171  3487 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.505393 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2146) took 1218 ms. Trace:
I20260709 12:18:38.512605 32114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:38.505821 32120 rpcz_store.cc:276] 0709 12:18:37.286877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.287038 (+   161us) service_pool.cc:224] Handling call
0709 12:18:38.505191 (+1218153us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b540010e5d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:38.505319 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118426496}
I20260709 12:18:38.506742 32115 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:38.515347 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.376808  3468 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.496322 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:38.520023  3489 tablet_replica.cc:333] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:18:38.523504 32114 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:38.463456  3417 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:18:38.523979 32114 raft_consensus.cc:614] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:38.529965 32654 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:38.526858 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:38.488830  3475 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:38.523227  3380 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:38.523801  3066 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:38.538743 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:38.537298 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.524680  3420 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.524768  3155 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:38.533017  3186 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:38.550843 32654 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:38.531891  3303 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:18:38.552984 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2003) took 3371 ms. Trace:
I20260709 12:18:38.525413  3486 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:18:38.556111 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.531498  3128 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:18:38.559666 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.553362 32116 rpcz_store.cc:276] 0709 12:18:35.181902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.554231 (+372329us) service_pool.cc:224] Handling call
0709 12:18:36.257354 (+703123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.257604 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:36.583413 (+325809us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.287575 (+704162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:37.294880 (+  7305us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:37.294947 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.295012 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.025680 (+730668us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b48002301d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:18:38.552820 (+527140us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400139a50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:38.552927 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23478,"prepare.queue_time_us":102153,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":7982,"raft.queue_time_us":41040,"raft.run_cpu_time_us":18702,"raft.run_wall_time_us":734517,"spinlock_wait_cycles":1621183616,"thread_start_us":933875,"threads_started":2,"wal-append.queue_time_us":724802,"wal-append.run_cpu_time_us":2136,"wal-append.run_wall_time_us":1201592}
I20260709 12:18:38.538172  3487 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.578799  3396 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:38.560868  3066 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:38.579453  3378 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.580278  3066 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.580515  3155 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.583369 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:38.583645  3386 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.584272 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:38.584892 31972 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:38.585114  3253 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.585537  3253 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.585791  3386 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.585939  3253 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.587337 32114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:38.587838  3411 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:38.588476  3411 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.587930 32114 raft_consensus.cc:606] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:38.592099 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:38.596166 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:38.597945 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:38.592689 32654 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.587196 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:38.601271 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.625710 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:38.626959 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.630807 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:38.631879 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 111184us
I20260709 12:18:38.574568  3495 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:38.611557 31859 maintenance_manager.cc:419] P a0887d9cdc8044caa86af7acdb454104: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033353
W20260709 12:18:38.652805 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.616472  3386 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:38.577194  3471 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.657392 32654 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:18:38.660784 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:38.634513  3128 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:38.634281  3495 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:38.667891 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:38.668741 32116 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.651264  3411 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.579485  3396 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.671066 32114 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:38.659993  3422 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.036s	user 0.014s	sys 0.000s
I20260709 12:18:38.678829  3253 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.679620  3307 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:38.681093  3253 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:38.683080 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:38.683651  3307 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:38.683862 31964 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:38.660807  3386 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:38.705541 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:38.706341 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:38.712564 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.713254 32117 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 4.
I20260709 12:18:38.715498 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:38.693090  3380 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:38.860544 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.869768 32121 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:38.921334 32120 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:38.922713 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:38.928592 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:38.929414 32117 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:38.931443 31964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:38.933162 31964 raft_consensus.cc:606] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:38.934831 31964 raft_consensus.cc:614] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:38.691847  3422 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:38.941159 31964 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:18:39.000680 32655 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:18:38.953687  3416 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:18:39.044874 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.012s	sys 0.073s
W20260709 12:18:39.059749 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.012s	sys 0.073s
I20260709 12:18:39.044952 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 405) took 1192 ms. Trace:
I20260709 12:18:39.060577 32047 rpcz_store.cc:276] 0709 12:18:37.852199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.921926 (+ 69727us) service_pool.cc:224] Handling call
0709 12:18:39.044703 (+1122777us) spinlock_profiling.cc:243] Waited 70.8 ms on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:39.044884 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162897664}
I20260709 12:18:39.061465 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:39.062676 32117 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:39.063105 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.952821  3307 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:39.057466  2800 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.035s	sys 0.034s
I20260709 12:18:39.000399  3497 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.207844 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:38.862089  3495 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:39.227157 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.014s	sys 0.050s
W20260709 12:18:39.227936 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.014s	sys 0.050s
W20260709 12:18:39.169250  2800 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.035s	sys 0.034s
W20260709 12:18:39.201853  3391 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.813s	user 0.012s	sys 0.000s
W20260709 12:18:39.228223 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 696507us
W20260709 12:18:39.227077  3416 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:39.297201 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:39.169875  3307 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:39.138979  3386 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.424187  3180 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.425168  3180 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.245942  3497 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:39.405912  3374 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.022s	user 0.009s	sys 0.002s
I20260709 12:18:39.298259  3391 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.225616  3380 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:39.416256  3475 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.878s	user 0.011s	sys 0.000s
I20260709 12:18:39.441805 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.442759 32047 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:18:39.430186  3155 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.444550 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.445370 32045 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:39.447233 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.448177 32045 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:39.455719 32116 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.004s
I20260709 12:18:39.456336 32116 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:39.459666 31963 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:39.463032 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:39.465049 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:39.466631 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:39.462215  3155 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.438620  3374 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.462292  3418 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:39.473582  3502 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:18:39.440834  3489 raft_consensus.cc:2243] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:39.483342 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:39.512892  3497 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:39.515888 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:39.442751  3396 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:39.551484  3490 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.018s	sys 0.061s
I20260709 12:18:39.525638 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2118) took 1383 ms. Trace:
W20260709 12:18:39.590174  3490 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.018s	sys 0.061s
I20260709 12:18:39.526489 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.591162  3514 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:39.591508 32047 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:39.592003  3514 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:39.440054  3475 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:39.535207 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.535142 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 238036us
I20260709 12:18:39.596446  3475 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:39.598451  3180 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:39.548794  3303 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:39.533677 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.601871 32121 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:39.605526 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:39.543313 32113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.161s	sys 0.188s
W20260709 12:18:39.606379 32113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.161s	sys 0.188s
W20260709 12:18:39.546444 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 388988us
W20260709 12:18:39.548739 32120 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.015s	sys 0.003s
I20260709 12:18:39.549674 32117 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:39.555974 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.630897 32120 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.015s	sys 0.003s
I20260709 12:18:39.538797  3507 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:39.575577 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:39.653370 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 77896.6us
W20260709 12:18:39.556020  3440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.010s	sys 0.074s
I20260709 12:18:39.588579 31967 raft_consensus.cc:2368] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11.
I20260709 12:18:39.536325 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:39.590250 31965 rpcz_store.cc:276] 0709 12:18:38.142233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.158797 (+ 16564us) service_pool.cc:224] Handling call
0709 12:18:38.159109 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.276406 (+117297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.276451 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.277043 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:18:38.343467 (+ 66424us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:38.349664 (+  6197us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.350000 (+   336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.568554 (+218554us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b180028ae00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419
0709 12:18:39.525163 (+956609us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540042fed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:39.525244 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.525308 (+    64us) raft_consensus.cc:1685] finished
0709 12:18:39.525330 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.525554 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9142,"raft.queue_time_us":80,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4707,"spinlock_wait_cycles":284711808,"thread_start_us":65935,"threads_started":1,"wal-append.queue_time_us":86901,"wal-append.run_cpu_time_us":2177,"wal-append.run_wall_time_us":1062055}
I20260709 12:18:39.551249  3484 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:39.629019 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.617261 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:39.645521 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.651146  3489 raft_consensus.cc:2272] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:39.652401 32038 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:39.531925  3438 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.632073 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:39.654089 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:39.655859 32117 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:18:39.617841  3505 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 12:18:39.664017 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:39.655495  3440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.010s	sys 0.075s
I20260709 12:18:39.666522 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:39.632623  3422 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:39.575331  3418 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.629700  3307 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:39.667824 31972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.031s	sys 0.030s
I20260709 12:18:39.668273 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:39.668352 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.608711  3303 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:39.705755 32043 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:39.710839 32038 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:39.667174  3065 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.713902 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:39.686983  3514 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:39.715394 31972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.032s	sys 0.031s
W20260709 12:18:39.705461  3378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.029s	sys 0.041s
I20260709 12:18:39.719226 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:39.733201 32119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 440) took 1604 ms (client timeout 1954 ms). Trace:
I20260709 12:18:39.733098 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:39.733673 32119 rpcz_store.cc:269] 0709 12:18:38.129290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.129811 (+   521us) service_pool.cc:224] Handling call
0709 12:18:39.719055 (+1589244us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b58000bfd98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:18:39.733135 (+ 14080us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6765440}
W20260709 12:18:39.682385  3375 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.022s	sys 0.095s
I20260709 12:18:39.665616  3180 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:39.735116 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.671047  3391 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:39.706422  3307 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:39.663472  3484 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:39.719430 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:39.669741  3496 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:18:39.667277  3438 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:39.736137  3378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.029s	sys 0.041s
I20260709 12:18:39.734884  3406 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:39.736716  3186 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:39.738937  3065 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:39.774107 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:39.780633 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61289.2us
W20260709 12:18:39.787828 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 133826us
I20260709 12:18:39.788110 32038 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:39.737201  3135 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:39.736783  3422 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:39.736650  3511 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:39.804227 32626 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.3:35255 timed out after 1.954s (SENT) [suppressed 15 similar messages]
I20260709 12:18:39.807310 32043 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:39.783743  3375 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.788298  3496 consensus_queue.cc:1243] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:39.752764  3307 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:39.792378 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:39.818905 32042 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:39.792543 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:39.820000 32045 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.817317  3374 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:39.785324 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 249098us
W20260709 12:18:39.831000  3510 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:39.819588  3307 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:39.788635 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:39.797145 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 194187us
W20260709 12:18:39.855708  3509 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:18:39.815589  3511 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:39.792335  3422 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:40.014441 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:40.014973 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 226427us
I20260709 12:18:40.015393 32041 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:40.000727 32038 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:40.015738 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:40.005328 32045 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:18:40.012521 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.017347 32042 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:40.017776 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:40.022357 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:40.027894 32011 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:40.031046  3303 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.033167 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.033577 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:40.010846  3378 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:40.022643  3516 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.361s	user 0.000s	sys 0.010s
I20260709 12:18:40.012517  3386 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.043596  3386 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.043912  3530 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:40.025805  3380 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:40.026732  3486 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:40.032300  3513 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.210s	user 0.003s	sys 0.006s
I20260709 12:18:40.019043  3422 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.044106  3425 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.045197 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2043) took 1413 ms. Trace:
I20260709 12:18:40.048099 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.051865  3440 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:40.055454  3511 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.045922  3378 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.061630  3378 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.063862 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 687) took 1740 ms. Trace:
I20260709 12:18:40.064097  3531 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.064282 32659 rpcz_store.cc:276] 0709 12:18:38.323514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.325418 (+  1904us) service_pool.cc:224] Handling call
0709 12:18:38.444913 (+119495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.445639 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.063661 (+1618022us) spinlock_profiling.cc:243] Waited 99.3 ms on lock 0x7b54003efb50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.063799 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228466560}
I20260709 12:18:40.064635  3531 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.067997  3155 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.068501  3489 ts_tablet_manager.cc:1916] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:18:40.026335  3479 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:18:40.068411  3532 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:40.029487  3186 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:18:40.073341 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.074134 32042 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.049646  3380 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.051892 32113 rpcz_store.cc:276] 0709 12:18:38.631820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.632585 (+   765us) service_pool.cc:224] Handling call
0709 12:18:38.632966 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.634207 (+  1241us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:38.659073 (+ 24866us) log.cc:844] Serialized 14 byte log entry
0709 12:18:40.000448 (+1341375us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.000946 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.001079 (+   133us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:40.001112 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.001176 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.044939 (+ 43763us) raft_consensus.cc:1685] finished
0709 12:18:40.044966 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.045126 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796,"prepare.queue_time_us":15885,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2823,"raft.queue_time_us":424241,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":21201,"thread_start_us":1147416,"threads_started":2,"wal-append.queue_time_us":1040643}
I20260709 12:18:40.086303  3428 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.077306  3532 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.090410  3532 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.090822 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2048) took 1416 ms. Trace:
I20260709 12:18:40.091200 32114 rpcz_store.cc:276] 0709 12:18:38.674823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.675041 (+   218us) service_pool.cc:224] Handling call
0709 12:18:38.702702 (+ 27661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.707457 (+  4755us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:38.859621 (+152164us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.353078 (+493457us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.353696 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.353852 (+   156us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:39.353893 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.353965 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.090610 (+736645us) raft_consensus.cc:1685] finished
0709 12:18:40.090632 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.090764 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11736,"prepare.queue_time_us":152933,"raft.queue_time_us":11582,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":20244,"spinlock_wait_cycles":134409984,"thread_start_us":295467,"threads_started":1,"wal-append.queue_time_us":635046}
I20260709 12:18:40.094933  3395 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:40.096925  3395 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:40.097458  3528 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:40.097939  3528 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:40.098371  3528 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.101161  3538 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:40.101704  3538 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.032908  3525 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:18:40.115159 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2121) took 1460 ms. Trace:
I20260709 12:18:40.115774 31972 rpcz_store.cc:276] 0709 12:18:38.654394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.660574 (+  6180us) service_pool.cc:224] Handling call
0709 12:18:38.660982 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.670605 (+  9623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.670634 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.684066 (+ 13432us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.732691 (+1048625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:39.732753 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.732848 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.114907 (+382059us) raft_consensus.cc:1685] finished
0709 12:18:40.114928 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.115083 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1951,"raft.queue_time_us":22249,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":1048207,"threads_started":1,"wal-append.queue_time_us":1091284}
I20260709 12:18:40.069733  3418 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.071079  3438 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:40.122476 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 382) took 1743 ms (client timeout 2215 ms). Trace:
W20260709 12:18:40.123114 31963 rpcz_store.cc:269] 0709 12:18:38.378588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.392547 (+ 13959us) service_pool.cc:224] Handling call
0709 12:18:40.122154 (+1729607us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400068454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.122389 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448545664}
I20260709 12:18:40.128403 32044 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.129307 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.129789 32044 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.130510 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.131408 32038 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:40.134147  3186 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.134780  3186 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.079988  3380 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:40.225893  3458 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:40.223913  3535 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.255013  3458 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:40.231099  3531 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.232645  3538 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.098141  3395 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.228011  3380 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.225400  3409 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.257102  3395 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.259496  3380 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.211190 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.263336 32041 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:40.255285  3535 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.111502  3307 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.268031  3422 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.270849  3535 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.256003  3458 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.280378 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.297794 32046 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:18:40.309077 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99668.4us
W20260709 12:18:40.309486 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.310171  3541 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.310425  3539 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:40.310913  3541 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.311174  3539 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.313630  3541 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.313812  3539 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.314692  3539 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.315285  3539 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.317727  3539 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.324530  3528 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.326613  3386 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.337989 32114 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:40.345399  3499 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:40.347451  3540 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:40.348094  3540 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:40.348912 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2103) took 2469 ms (client timeout 1871 ms). Trace:
W20260709 12:18:40.349327 31969 rpcz_store.cc:269] 0709 12:18:37.879326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.883483 (+  4157us) service_pool.cc:224] Handling call
0709 12:18:40.348623 (+2465140us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54000b5c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.348833 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961138048}
W20260709 12:18:40.355551 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2090) took 4418 ms (client timeout 1700 ms). Trace:
I20260709 12:18:40.355980 31968 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:40.350262  3540 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.355206  3471 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:40.359906 31966 rpcz_store.cc:269] 0709 12:18:35.937197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.937308 (+   111us) service_pool.cc:224] Handling call
0709 12:18:40.355368 (+4418060us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54001ea0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.355485 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3144284416}
W20260709 12:18:40.358807 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 330982us
I20260709 12:18:40.356560 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:40.361068  3422 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.363222 32653 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:40.360042  3376 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.388806  3436 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:40.393414 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.397284 32118 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.400250 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2034) took 2266 ms. Trace:
I20260709 12:18:40.400705 32118 rpcz_store.cc:276] 0709 12:18:38.133613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.133720 (+   107us) service_pool.cc:224] Handling call
0709 12:18:40.397028 (+2263308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.397841 (+   813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.399984 (+  2143us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b5400434ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.400172 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5204352384}
W20260709 12:18:40.402355 32112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 441) took 2272 ms (client timeout 1527 ms). Trace:
W20260709 12:18:40.402843 32112 rpcz_store.cc:269] 0709 12:18:38.129632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.129811 (+   179us) service_pool.cc:224] Handling call
0709 12:18:40.396878 (+2267067us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.402233 (+  5355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68917120}
I20260709 12:18:40.405406 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:40.411909  3380 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.412596  3380 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.414475  3135 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.415019  3135 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.417207  3135 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.422328  3380 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.419425 31964 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:40.423413 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2124) took 1758 ms. Trace:
I20260709 12:18:40.423507  3186 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.423909 31964 rpcz_store.cc:276] 0709 12:18:38.665236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.017648 (+352412us) service_pool.cc:224] Handling call
0709 12:18:40.418734 (+1401086us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:18:40.418998 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.423151 (+  4153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.423352 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17450368}
I20260709 12:18:40.425563 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2033) took 2292 ms. Trace:
I20260709 12:18:40.425913 32115 rpcz_store.cc:276] 0709 12:18:38.132952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.133055 (+   103us) service_pool.cc:224] Handling call
0709 12:18:38.506565 (+373510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.516468 (+  9903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.425370 (+1908902us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b540010e5d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.425506 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858351360}
W20260709 12:18:40.430392 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.430830 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.431893  3380 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.432039  3428 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:40.436291  3428 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.438409  3386 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.439608 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.440276  3428 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.438998  3386 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.441613 32042 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:40.432559  3380 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.434569  3378 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.434706  3540 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:40.435415 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0119119170001341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 12:18:40.435734  3539 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.442638  3386 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.461691  3540 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.454238 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:40.462914 32651 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:18:40.464133 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0119119170001341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 12:18:40.463886  3539 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.465229  3409 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.465683  3409 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.456384 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.467211  3378 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.453809 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:40.467633  3409 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.468146 32657 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.449239  3098 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.479405  3538 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 12:18:40.483412 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 449898us
I20260709 12:18:40.483594  3538 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:40.484025  3538 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:40.484823  3538 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.486016  3538 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.459260 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.487663 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.472566 31963 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:40.488688 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:40.470503 32120 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.007s	sys 0.004s
I20260709 12:18:40.489755 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2055) took 1568 ms. Trace:
I20260709 12:18:40.453327 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.490412 32654 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:40.490128 32120 rpcz_store.cc:276] 0709 12:18:38.921029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.923828 (+  2799us) service_pool.cc:224] Handling call
0709 12:18:38.924153 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.924418 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:39.631300 (+706882us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.804814 (+173514us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.938688 (+133874us) log.cc:844] Serialized 14 byte log entry
0709 12:18:40.489439 (+550751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:40.489487 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:40.489534 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.489603 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.489705 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":877461,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":2582,"raft.queue_time_us":392200,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"thread_start_us":1528701,"threads_started":4,"wal-append.queue_time_us":451430}
I20260709 12:18:40.491719 32654 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:40.492684  3535 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.493312 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.493219  3535 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.496069 32115 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:40.497102  3535 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.497906 32115 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:40.481557  3376 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.442749  3135 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.499629  3135 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.499846  3409 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:40.501225  3531 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:40.490564 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:18:40.501597  3531 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:40.460459  3521 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:40.453001  3544 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:40.472759  3386 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.482759  3539 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:40.512681 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.503489 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.503909 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:40.515946 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:40.516423 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.516898 32038 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:40.517163 32115 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:40.515486 32660 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.517678 32115 pending_rounds.cc:85] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 6
I20260709 12:18:40.518230 32115 pending_rounds.cc:107] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:40.518278 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.518814 32115 raft_consensus.cc:2889] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.511632 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.520138 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 689) took 2180 ms. Trace:
I20260709 12:18:40.512612 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:40.520870 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.511952 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.521939 32043 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:40.514842 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.522948 32041 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:40.527678 32651 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:40.528913 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.529199 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.512255 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.502434  3531 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.530990 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.531574 32116 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 6.
I20260709 12:18:40.504257  3392 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.531939  3531 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.532639 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.533260 31972 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:40.534691  3377 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.534899 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:40.535810  3520 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.536324  3520 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.536653  3495 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.534698 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.504221  3521 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:40.538347 32113 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:40.539609  3521 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:40.540596 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.540987  3521 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.506067  3484 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.542073  3423 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:40.520495 32655 rpcz_store.cc:276] 0709 12:18:38.340196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.340336 (+   140us) service_pool.cc:224] Handling call
0709 12:18:40.519991 (+2179655us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54006031d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:40.520089 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511330688}
I20260709 12:18:40.542479  3423 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:40.542632 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.543196 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.543907 32046 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:40.545966 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.478205 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:40.551333 32657 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:40.552017 32654 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.553985 32046 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:40.555560 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67911.4us
I20260709 12:18:40.555842 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.556563 32046 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:18:40.556545 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:40.559489 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.564361 32113 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.563879 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:40.567518 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89394.2us
I20260709 12:18:40.569278 32042 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:40.576187 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.576489 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:40.577086 32113 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:40.577186 32120 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:40.584935 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:40.528939 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:40.698585 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.699245 32113 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:18:40.699750 32113 pending_rounds.cc:85] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 7
I20260709 12:18:40.699991 32113 pending_rounds.cc:107] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:18:40.700179 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.700222 32113 raft_consensus.cc:2889] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.609766 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.610648 32657 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:40.701463 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.611268 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.702734 31966 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:40.610911 32118 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:40.704149 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:40.705127 31966 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.610721 32655 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:40.615557 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46623.7us
I20260709 12:18:40.614753 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.708892 32041 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:40.709971 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.645664 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:40.632270 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:40.614089 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:40.713681 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.714604 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.714979 32116 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:40.720739 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:40.731343 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:40.732815 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:40.733388 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.482517  3380 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.502269  3135 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.743610 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.738642 32120 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:40.745708 32120 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:40.747291 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:40.741912 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.748335 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.747387 32120 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:40.739672 31970 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:18:40.749402 32120 pending_rounds.cc:85] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:18:40.742125 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:40.751271 32118 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:40.754103 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.749845 32120 pending_rounds.cc:107] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:40.759717 32120 raft_consensus.cc:2889] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.761072 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:40.761083 32114 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:40.609392  3128 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.777566 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:40.700472  3526 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.752770 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.818257 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.543896  2689 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.819330 32120 raft_consensus.cc:2368] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:18:40.818564 32118 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:40.821653 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.752406  3539 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.775409 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.506768  3540 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:40.698498  3520 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:40.609597  3478 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.610107  3456 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.698657  3417 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.698709  2800 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:40.816497 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 451085us
I20260709 12:18:40.755158 31970 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:41.011387 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:40.828956 32658 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.770442 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:40.829387 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89569.2us
I20260709 12:18:41.015321 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:40.831041 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 185510us
I20260709 12:18:40.508133  3386 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.832751 32119 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:40.505324  3544 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.775409  3484 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:40.827188  3258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.440s	user 0.067s	sys 0.032s
I20260709 12:18:40.940557 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.741806  3396 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:18:40.716971  3523 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 12:18:40.761322  3399 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:40.944293 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:40.821501  3378 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:40.938934 32653 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.031s	sys 0.024s
W20260709 12:18:41.114428 32653 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.032s	sys 0.024s
I20260709 12:18:41.109558 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:40.940685 32659 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.013s	sys 0.044s
W20260709 12:18:41.115412 32659 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.013s	sys 0.044s
I20260709 12:18:41.115566 31970 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 5: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:40.829903  3382 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.827044  3135 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:40.950541 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.121398 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.010586  3420 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:18:40.543462  3423 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:40.846462  3427 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:40.966784 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 50704.8us
I20260709 12:18:41.005532 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:41.129722 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 124282us
I20260709 12:18:40.814671  3128 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:40.966784  3388 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.009s	sys 0.031s
I20260709 12:18:40.829259  2689 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.029639 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.142212 32658 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
W20260709 12:18:41.021356  2911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.020s	sys 0.058s
W20260709 12:18:41.021317  3494 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.009s	sys 0.035s
I20260709 12:18:41.149986 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:40.846575  3538 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.187s
I20260709 12:18:41.091866 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.090895  3543 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:18:41.044101 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:40.968403  3303 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:41.032577  3258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.645s	user 0.068s	sys 0.032s
W20260709 12:18:41.148322  2911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.020s	sys 0.058s
I20260709 12:18:41.111059 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.180853 32118 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:41.149746  3494 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.009s	sys 0.036s
I20260709 12:18:41.020745  3520 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.021107  3428 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.029888  3526 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:40.946300 32657 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:41.147306  3420 consensus_queue.cc:1243] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:41.025580  3540 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
W20260709 12:18:40.965377 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.188318 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:40.830668  3380 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.119398 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.111234  3399 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:41.121582 32120 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:41.006717  3478 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.106987  3558 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.124801  3382 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.209523 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.026013  2800 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.153137 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.057229  3484 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.159366 32119 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:41.100455 32656 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:41.159575 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:41.142246  3388 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.009s	sys 0.031s
I20260709 12:18:41.109525  3514 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:18:41.128943  3396 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:41.169917 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.128505  3427 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.034027 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:41.179802 31964 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:41.111059 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.249811 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.021885  3417 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.188954 32114 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:41.158978  3066 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.129312  3135 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.193893 32043 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:41.195442 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.030023  3409 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 12:18:41.261533 32118 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:41.260600 31971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.012s	sys 0.012s
W20260709 12:18:41.214665 32046 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.001s
I20260709 12:18:41.179332  3303 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.188078  3526 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:41.212199  3377 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.006s
I20260709 12:18:41.218055 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.242573 31970 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:41.199683  3399 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.210189  3507 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:41.250803 32041 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:41.250805 31966 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:18:41.237787  3098 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.754s	user 0.005s	sys 0.005s
I20260709 12:18:41.187886  3428 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.263106 32046 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:18:41.356310 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.192713  3520 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.242110  3396 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:41.203688  3380 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.283034 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.314783  3494 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.255437  3066 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.302745 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 379) took 2927 ms (client timeout 1789 ms). Trace:
I20260709 12:18:41.304306 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.227453  3558 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.249568  3530 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:18:41.269052  3258 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.283936  3526 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.253726  3420 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:18:41.312733 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.316668 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:41.417906 31970 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:41.417977 31966 raft_consensus.cc:2376] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11. Re-sending same reply.
I20260709 12:18:41.316879  3386 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.317282 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2107) took 3436 ms. Trace:
I20260709 12:18:41.307556  3507 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.319278 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:41.319703  3395 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.062s	user 0.011s	sys 0.000s
W20260709 12:18:41.310178 32660 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.766s	user 0.006s	sys 0.002s
I20260709 12:18:41.422740 32660 raft_consensus.cc:2410] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:18:41.320624 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:41.423380 32042 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.321790 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:41.324323  3562 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:18:41.324469  3508 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 2.007s	user 0.000s	sys 0.011s
I20260709 12:18:41.288148  3525 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.285564  3543 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:41.307054  3540 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:41.295996 32038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.009s	sys 0.019s
I20260709 12:18:41.308257  3475 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.253420  3427 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:41.363462 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:41.436688 32046 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:41.437775 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.438546 32046 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
W20260709 12:18:41.353734  3564 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:18:41.348727  2897 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:18:41.368575 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 598195us
I20260709 12:18:41.325919  3544 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:41.408449 31967 rpcz_store.cc:269] 0709 12:18:38.375549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.375709 (+   160us) service_pool.cc:224] Handling call
0709 12:18:41.300216 (+2924507us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000f6254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.302666 (+  2450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2772950784}
I20260709 12:18:41.313447  3481 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:18:41.417357  3258 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:41.416014 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.419217 31968 rpcz_store.cc:276] 0709 12:18:37.880482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.988687 (+108205us) service_pool.cc:224] Handling call
0709 12:18:40.355530 (+2366843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.359980 (+  4450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.317074 (+957094us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b54001ea0d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.317220 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5393621248}
I20260709 12:18:41.424496 32044 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:41.412274  3066 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.362568  3128 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:41.428241 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 297467us
W20260709 12:18:41.430626 32038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.009s	sys 0.020s
I20260709 12:18:41.430371  3476 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.440541 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.496356 31970 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:41.424500  3494 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:18:41.430570  3543 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:41.482239 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:41.482770 32043 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:41.483983 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.443810 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.432723 32042 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:41.442688  2897 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:41.497325 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 242749us
I20260709 12:18:41.493690 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 639) took 1118 ms. Trace:
I20260709 12:18:41.513971 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:41.478248  3258 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.517855 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.493721  3468 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.517293 31968 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.516753 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:41.480532  3098 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.517340 31964 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.481089  3544 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:41.493852  3484 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.481581  3478 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.497669  3382 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:41.544726  3565 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:18:41.546981 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 693) took 1085 ms. Trace:
I20260709 12:18:41.547349 32654 rpcz_store.cc:276] 0709 12:18:40.461856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.492647 (+ 30791us) service_pool.cc:224] Handling call
0709 12:18:40.551852 (+ 59205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.552382 (+   530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.546796 (+994414us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b54005eb5d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.546926 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135542528}
W20260709 12:18:41.548588  3568 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 12:18:41.549343  3557 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:18:41.551894 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 746) took 1097 ms. Trace:
I20260709 12:18:41.552265 32656 rpcz_store.cc:276] 0709 12:18:40.454694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.456078 (+  1384us) service_pool.cc:224] Handling call
0709 12:18:41.551693 (+1095615us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540053edd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.551827 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455790464}
W20260709 12:18:41.558805  3561 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 12:18:41.559723  3559 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:18:41.458272  3396 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.563031  3396 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.573297  3258 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.519933 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 638) took 1155 ms. Trace:
I20260709 12:18:41.496466  2800 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.533911  3098 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.536971  3135 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.594278 31964 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:41.594543  3423 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:41.638396 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.517132 32659 rpcz_store.cc:276] 0709 12:18:40.374923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.375074 (+   151us) service_pool.cc:224] Handling call
0709 12:18:40.375351 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.375561 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:41.115861 (+740300us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.493314 (+377453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.493372 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:41.493423 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.493499 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.493627 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3134,"prepare.queue_time_us":773928,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":3140,"raft.run_cpu_time_us":2411,"raft.run_wall_time_us":20287,"thread_start_us":1117072,"threads_started":2}
I20260709 12:18:41.640092 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2131) took 1028 ms. Trace:
I20260709 12:18:41.641052  3399 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.643285  3135 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.643827  3135 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.645856  3135 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.647567  3541 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:41.648473 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.650005  2689 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.651646 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.652465 32659 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:41.651126 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.656235 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:41.657132  3135 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:41.657579  3135 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.658007  3135 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.640434 31964 rpcz_store.cc:276] 0709 12:18:40.612001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.639335 (+1027334us) service_pool.cc:224] Handling call
0709 12:18:41.639605 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.639778 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.639801 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.639840 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:41.639879 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.639944 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.640040 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:41.662573 32651 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:41.664202 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 697) took 1166 ms. Trace:
I20260709 12:18:41.664548 32651 rpcz_store.cc:276] 0709 12:18:40.497526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.528583 (+ 31057us) service_pool.cc:224] Handling call
0709 12:18:41.664034 (+1135451us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400543654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.664147 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458666368}
I20260709 12:18:41.666961  3558 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.668378  3307 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.669661  3520 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.596215 32653 rpcz_store.cc:276] 0709 12:18:40.365023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.365170 (+   147us) service_pool.cc:224] Handling call
0709 12:18:40.365461 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.365681 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:41.114856 (+749175us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.498239 (+383383us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.519572 (+ 21333us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:41.519648 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.519730 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.519869 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":783569,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":25851,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":1131877,"threads_started":2}
I20260709 12:18:41.596855  3489 ts_tablet_manager.cc:1929] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.5
I20260709 12:18:41.599570  3098 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.672215  3489 log.cc:1199] T 826cffce8bb540c8beb56cc00a702782 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/826cffce8bb540c8beb56cc00a702782
I20260709 12:18:41.600620 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1903) took 1995 ms. Trace:
I20260709 12:18:41.673099 32040 rpcz_store.cc:276] 0709 12:18:39.605072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.605536 (+   464us) service_pool.cc:224] Handling call
0709 12:18:39.605872 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.654773 (+ 48901us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:40.767154 (+1112381us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.198868 (+431714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.198931 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:41.198982 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.600316 (+401334us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b540003a054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:41.600381 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.600543 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":725786,"prepare.queue_time_us":730965,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":24,"raft.queue_time_us":656,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"spinlock_wait_cycles":111831296,"thread_start_us":791969,"threads_started":2,"wal-append.queue_time_us":584434}
W20260709 12:18:41.604797 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 87988us
W20260709 12:18:41.585812  3521 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.044s	user 0.007s	sys 0.003s
I20260709 12:18:41.607388 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:41.675683 31969 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:41.676007 32013 leader_election.cc:400] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:18:41.676306 32013 leader_election.cc:403] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:41.615974  3377 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:18:41.676994 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:41.436271  3525 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.604853 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2023) took 1044 ms. Trace:
I20260709 12:18:41.538650 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.678833 32038 rpcz_store.cc:276] 0709 12:18:40.560352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.560478 (+   126us) service_pool.cc:224] Handling call
0709 12:18:40.560806 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.561057 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:40.775652 (+214595us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.604136 (+828484us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b180020d1c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:41.604392 (+   256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.604466 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:41.604535 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.604630 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.604782 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194891,"prepare.queue_time_us":379338,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3708,"raft.queue_time_us":111456,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"spinlock_wait_cycles":2980224,"thread_start_us":717033,"threads_started":1,"wal-append.queue_time_us":720211}
I20260709 12:18:41.617506  3436 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:41.542357  2709 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:41.681092  3436 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:41.681244  2709 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:41.542761  2897 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:41.626821  3556 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
I20260709 12:18:41.682493  3484 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:41.683203  2897 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.629793  3458 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.683851  3484 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:18:41.684551  3484 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.684882  3484 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:41.685220  3484 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:41.539984  3517 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.719s	user 0.013s	sys 0.004s
W20260709 12:18:41.630522  3569 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:18:41.616904 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2004) took 3872 ms. Trace:
I20260709 12:18:41.682247  2709 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.688582  2709 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:41.687455 32039 rpcz_store.cc:276] 0709 12:18:37.744051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.758334 (+ 14283us) service_pool.cc:224] Handling call
0709 12:18:37.758654 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.029977 (+271323us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:38.042968 (+ 12991us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.581994 (+539026us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b1800315a60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:40.369446 (+1787452us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b1800315a60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:18:40.409650 (+ 40204us) log.cc:844] Serialized 14 byte log entry
0709 12:18:40.941650 (+532000us) log.cc:844] Serialized 14 byte log entry
0709 12:18:40.948212 (+  6562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:40.948267 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:40.948310 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.616614 (+668304us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540013d3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:41.616673 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.616834 (+   161us) inbound_call.cc:177] Queueing success response
0709 12:18:41.626265 (+  9431us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
Metrics: {"mutex_wait_us":2399165,"prepare.queue_time_us":2462097,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":4449,"raft.queue_time_us":101942,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":64425,"spinlock_wait_cycles":755000704,"thread_start_us":437577,"threads_started":1,"wal-append.queue_time_us":855629}
I20260709 12:18:41.631199  3380 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.678149 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:41.576968  3543 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.634106 31867 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 13, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 13 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.690990  3543 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.691977 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:41.691529 31867 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:41.692673  3514 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.693040  3514 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.635318 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.693405  3514 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.443070  3475 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.601286  3528 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:41.579800  3570 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:18:41.581498  3567 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:18:41.696628 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2127) took 1085 ms. Trace:
I20260709 12:18:41.697646  3475 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.696943 31971 rpcz_store.cc:276] 0709 12:18:40.610966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.612935 (+  1969us) service_pool.cc:224] Handling call
0709 12:18:40.613213 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:40.712744 (+ 99531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:40.712777 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.744259 (+ 31482us) log.cc:844] Serialized 26 byte log entry
0709 12:18:41.262705 (+518446us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:41.262744 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.262811 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:41.696428 (+433617us) raft_consensus.cc:1685] finished
0709 12:18:41.696447 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.696573 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99848,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":21514,"thread_start_us":518107,"threads_started":1,"wal-append.queue_time_us":713370}
I20260709 12:18:41.698467 32652 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:41.699060 31971 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:18:41.699820 32652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 732) took 3455 ms (client timeout 1672 ms). Trace:
W20260709 12:18:41.700302 32652 rpcz_store.cc:269] 0709 12:18:38.244700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.244916 (+   216us) service_pool.cc:224] Handling call
0709 12:18:41.699610 (+3454694us) spinlock_profiling.cc:243] Waited 3.43 s on lock 0x7b5400603bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.699737 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7892088064}
W20260709 12:18:41.573786  3560 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:18:41.575690  3544 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.589267 31968 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:41.517709 31966 raft_consensus.cc:2368] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:18:41.590827  3428 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.595973  3258 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:41.610589 31869 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 149) took 28869 ms (client timeout 15000 ms). Trace:
I20260709 12:18:41.703277 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:41.704386 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:41.705262  3377 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.705835  3377 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.580840  3495 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.044s	user 0.009s	sys 0.002s
I20260709 12:18:41.680502 31965 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:41.681926  3436 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.691071  3380 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.679867  3525 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.527734  3468 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:41.543916  3376 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.045s	user 0.005s	sys 0.005s
W20260709 12:18:41.703469 31869 rpcz_store.cc:269] 0709 12:18:12.741664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.778955 (+ 37291us) service_pool.cc:224] Handling call
0709 12:18:14.602452 (+1823497us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1d7f3a 00007f2291f8de9c 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:18:15.071985 (+469533us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890ab640 00007f2291f82846 00007f229208c90e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:18:41.610493 (+26538508us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:37.788368 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:37.789012 (+   644us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:37.789051 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:18:37.789071 (+    20us) tablet.cc:662] Decoding operations
0709 12:18:37.789708 (+   637us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:37.789775 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:37.789806 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:18:37.790209 (+   403us) tablet.cc:701] Row locks acquired
0709 12:18:37.790228 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:18:37.790425 (+   197us) write_op.cc:270] Start()
0709 12:18:37.790539 (+   114us) write_op.cc:276] Timestamp: P: 1783599517790345 usec, L: 0
0709 12:18:37.790564 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:18:37.884327 (+ 93763us) log.cc:844] Serialized 1662 byte log entry
0709 12:18:38.394827 (+510500us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b1800028d80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2557 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:38.608477 (+213650us) op_driver.cc:464] REPLICATION: finished
0709 12:18:39.736365 (+1127888us) write_op.cc:301] APPLY: starting
0709 12:18:39.753088 (+ 16723us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:39.832106 (+ 79018us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:39.832142 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:40.031072 (+198930us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:40.031302 (+   230us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:18:40.031364 (+    62us) write_op.cc:312] APPLY: finished
0709 12:18:40.031623 (+   259us) log.cc:844] Serialized 41 byte log entry
0709 12:18:40.351949 (+320326us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:40.352058 (+   109us) write_op.cc:454] Released schema lock
0709 12:18:40.352103 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:40.352562 (+   459us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:40.352592 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28473,"reported_tablets":25,"spinlock_wait_cycles":3360556544,"child_traces":[["op",{"apply.queue_time_us":1097552,"apply.run_cpu_time_us":63052,"apply.run_wall_time_us":623502,"num_ops":4,"prepare.queue_time_us":337928,"prepare.run_cpu_time_us":35796,"prepare.run_wall_time_us":337682,"raft.queue_time_us":130768,"raft.run_cpu_time_us":80474,"raft.run_wall_time_us":1020525,"replication_time_us":805088,"spinlock_wait_cycles":8910080,"thread_start_us":1820394,"threads_started":5,"wal-append.queue_time_us":474184,"wal-append.run_cpu_time_us":20149,"wal-append.run_wall_time_us":2418741}]]}
I20260709 12:18:41.673964  3489 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42276 (request call id 0) took 3258 ms. Trace:
I20260709 12:18:41.666090  3135 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.522585 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:41.701285 31971 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:41.693040 31867 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 10 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.719144 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:41.741791 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 707788us
I20260709 12:18:41.743310 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:41.747328 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.028s	sys 0.024s
W20260709 12:18:41.747849 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.028s	sys 0.024s
I20260709 12:18:41.749488 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:41.748486 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 704436us
W20260709 12:18:41.633231  3531 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.101s	user 0.010s	sys 0.000s
I20260709 12:18:41.734972 31967 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:41.707616 31963 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:41.706271 32047 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:18:41.753270 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.025s	sys 0.030s
W20260709 12:18:41.779047 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.026s	sys 0.030s
I20260709 12:18:41.739001 31867 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 10 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.729496 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:41.752362 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:41.780772 32652 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.780417 31867 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 8 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:41.732477 31970 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:41.783419 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1861) took 1018 ms. Trace:
I20260709 12:18:41.783739 31970 rpcz_store.cc:276] 0709 12:18:40.764583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.653993 (+889410us) service_pool.cc:224] Handling call
0709 12:18:41.732034 (+ 78041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.783175 (+ 51141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.783365 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:41.785356 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3916234599998916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
I20260709 12:18:41.786896 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:41.752633  3523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.012s	sys 0.020s
I20260709 12:18:41.787560 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:41.709829  3436 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.706655  3377 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.781951 31867 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 8 to 9, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 9 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:41.792807 32047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1898) took 2189 ms (client timeout 1857 ms). Trace:
I20260709 12:18:41.664558  3417 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.792009 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1847) took 1281 ms. Trace:
I20260709 12:18:41.800087 31963 rpcz_store.cc:276] 0709 12:18:40.510664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.515482 (+  4818us) service_pool.cc:224] Handling call
0709 12:18:41.791836 (+1276354us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400312754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.791955 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677494016}
W20260709 12:18:41.793200 32047 rpcz_store.cc:269] 0709 12:18:39.603582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.605308 (+  1726us) service_pool.cc:224] Handling call
0709 12:18:41.792634 (+2187326us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540013ced4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:41.792727 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4769982208}
I20260709 12:18:41.793401 31867 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 8 to 13, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 13 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:41.795532 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.802630 32040 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:41.714700  3186 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.796011 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:41.796790 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.026s	sys 0.037s
I20260709 12:18:41.805907 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:41.806515 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.026s	sys 0.037s
W20260709 12:18:41.806835 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 796346us
I20260709 12:18:41.807168 32045 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:18:41.734009  3539 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.898s	user 0.008s	sys 0.002s
W20260709 12:18:41.803814 32085 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.006s	sys 0.023s
I20260709 12:18:41.728219  3303 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.771027  3528 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:41.824707 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.703290  3544 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.751300  3458 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:41.809990 32085 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.006s	sys 0.024s
I20260709 12:18:41.707490  3258 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:41.794550  3523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.013s	sys 0.020s
I20260709 12:18:41.807423 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:41.713179  3468 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:41.807961 31867 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.852550 31867 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 2 to 4, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 4 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.707321  3541 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.791394  3522 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.721457  3427 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.728789  3135 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:41.858949 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 44083.4us
I20260709 12:18:41.871251 32040 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:41.872112 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.873332 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:41.873751 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.876294 32038 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:41.881299 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.881930 32042 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:41.881979 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:41.857172 31867 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 6 to 7, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 7 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.883821 31867 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 10, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 10 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.722256  3507 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:41.829849 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 603531us
I20260709 12:18:41.889953  3399 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:41.893957  3253 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.894259 32652 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:41.895747 32659 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:41.895920 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:41.837883 31972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.013s	sys 0.021s
W20260709 12:18:41.896683 31972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.013s	sys 0.021s
I20260709 12:18:41.726529  3514 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.842267 31936 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:41.819051  3539 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.899530  3514 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:41.830686 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.010s	sys 0.023s
I20260709 12:18:41.899083 31867 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:41.900117 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.011s	sys 0.023s
I20260709 12:18:41.902444 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.902719 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.903050 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:41.904122  3468 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:41.904683  3468 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.906595  3458 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.907094  3458 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.907439 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.908651  3376 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.909162  3458 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:41.909133  3376 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.909428  3481 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:41.910094 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.910784 32112 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:41.911247  3376 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:41.911497 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 69292.6us
I20260709 12:18:41.912163 31936 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:41.913170 31936 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:41.913199  3066 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:41.910157  3481 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:41.914680 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:41.916198 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:41.916721 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:41.841220  3563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.000s	sys 0.039s
I20260709 12:18:41.918746 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.927106 32041 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:41.927809 32038 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:41.818599  3186 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.928733 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.929296 32115 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:41.931917 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:41.858566  3522 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:41.940675  3528 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.941201  3468 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:41.940688  3495 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:41.942265 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:41.940376  3458 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:18:41.943550  3528 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:41.943903  3458 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:41.944610  3528 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:41.930769  3453 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:18:41.945042  3528 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:41.945636  3453 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:41.946012  3453 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:41.880743  3523 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:18:41.855579  3541 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:41.947115  3541 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.947036 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:41.946910  3453 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.947901  3544 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:41.948153  3186 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:41.948297  3453 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.948326  3544 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:41.942073  3495 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.940797  3522 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.855274  3575 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:41.931375 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:41.950865 32047 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:41.932101 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:41.955242 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:41.966467 32651 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:41.967727 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:41.969151 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.969907 32651 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:41.845778  3258 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.929095  3501 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:41.929875  3563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.000s	sys 0.041s
I20260709 12:18:41.844214  3411 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 12:18:41.957779 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.983295 32659 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
W20260709 12:18:41.957333 32086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.022s	sys 0.023s
W20260709 12:18:41.984478 32086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.023s	sys 0.023s
I20260709 12:18:41.984791 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:41.855232  3210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.012s	sys 0.021s
I20260709 12:18:41.900583 31867 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:41.972615 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.954109 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:42.009588 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:42.012707 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58657us
I20260709 12:18:42.010246 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:42.013454 32626 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:41.948989  3544 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.974165  3428 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:41.932103  3417 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:41.933907  3521 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:41.978453 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:41.930135  3581 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.978289 32118 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:41.958504  3495 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:41.980731 31937 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.005s	sys 0.012s
I20260709 12:18:41.973017 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:41.930190  3399 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:18:41.721946  3489 rpcz_store.cc:276] 0709 12:18:38.415969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.416205 (+   236us) service_pool.cc:224] Handling call
0709 12:18:38.519341 (+103136us) ts_tablet_manager.cc:1151] Deleting tablet 826cffce8bb540c8beb56cc00a702782
0709 12:18:38.519450 (+   109us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:18:40.273931 (+1754481us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:41.596719 (+1322788us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:41.673787 (+ 77068us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":886918,"tablet-delete.queue_time_us":102110,"thread_start_us":59369,"threads_started":1}
I20260709 12:18:42.009083 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:42.040743 32657 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:18:42.020848 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:41.929908  3475 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:18:42.009555 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.002s	sys 0.018s
I20260709 12:18:41.945851  3528 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:41.955607 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:41.974109  3543 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:41.940117  2897 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 12:18:41.986061  3210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.013s	sys 0.022s
W20260709 12:18:42.018421  3526 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.002s
I20260709 12:18:41.975307  3458 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.977700  3575 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:42.008971  3386 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.000s	sys 0.017s
I20260709 12:18:42.020712 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:41.961649  3484 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.023742 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:42.030200 31937 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.012s
W20260709 12:18:42.030604 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.003s	sys 0.024s
I20260709 12:18:42.025846 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.016202 31867 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 2 to 7, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:42.038046 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.038254 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.029474 32626 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:42.039249 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:41.974102  3523 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:18:42.012926  3525 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:42.044324 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.003s	sys 0.019s
I20260709 12:18:42.048965 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:42.135653 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:42.135644 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 692382us
I20260709 12:18:42.008057  3558 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:42.037935  3521 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:42.009498  3307 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.053159  3494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.015s	sys 0.026s
I20260709 12:18:42.014010  2689 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:18:42.028118  3098 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:42.067057 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 94115.6us
I20260709 12:18:42.016353  3423 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.017427  3428 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:41.986199  3501 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.052116  3186 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:42.127559 32116 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:42.127903 32120 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:42.127995 32119 raft_consensus.cc:2410] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:42.127559 32117 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:42.130849 32118 raft_consensus.cc:2410] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:42.172328  3427 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:42.173646  3538 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:42.174194  3538 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:42.175215  3538 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.176517  3538 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:42.131089 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.003s	sys 0.024s
W20260709 12:18:42.178560 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 686166us
I20260709 12:18:42.180433 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:42.181180  3155 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:42.181589  3155 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.181931  3155 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.043772  3417 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:42.183121  3417 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.137054 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:42.138401 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.056172  3543 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.141773 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:42.143420 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 94501.8us
W20260709 12:18:42.146756 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 126116us
I20260709 12:18:42.147372 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:42.064586  3458 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.058863  3303 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:42.158252  3098 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.158411 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 826cffce8bb540c8beb56cc00a702782 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:18:42.159003  3573 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.159119 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:42.159852  3587 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:18:42.138645  3590 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:18:42.160218  3476 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.151599  2911 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:42.139096  3581 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.245070 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 92248.9us
I20260709 12:18:42.141085  3525 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.134604  3475 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:42.165509 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:42.150374  3494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.016s	sys 0.027s
I20260709 12:18:42.239840  2897 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.257113  2897 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.257457 32038 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:42.240170  3476 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.138875 31867 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 5 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4), now has a pending config: VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2), VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 8 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:42.258282  3476 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.144387  3528 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:42.259385  2897 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:42.260284  3258 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:42.261945  3396 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:42.262351  3396 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:42.258591 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.201987 32651 raft_consensus.cc:2410] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:18:42.182188 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:42.263185  3396 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:42.204686 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67747.2us
I20260709 12:18:42.263985 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:42.264393  3396 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.264714 32655 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:42.206084 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.259136 31867 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 1 to 4, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 4 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:42.266249 32116 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:42.266832  3535 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:42.142987  3411 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.267212  3535 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:42.267402  3411 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:42.267812  3411 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.268057  3535 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.162607  3423 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.216885 31963 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:42.217372 32045 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:42.043720  3481 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.168159  3428 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:42.231832 31857 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 98685.1us
I20260709 12:18:42.295079 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.187089  3186 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.219102  3098 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:42.146425  3307 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.142575  3558 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.208159  3468 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:42.207669  3458 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.208142  3543 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:42.070811  3526 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.269052 32659 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:42.216886  3578 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:42.267234 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:42.250643 31970 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:42.236397  3066 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:42.245395  3573 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.295704 32119 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:42.244699  2911 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.228574  3541 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.270036  3581 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.292901  3583 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:18:42.314610 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 132475us
W20260709 12:18:42.315104 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50793.2us
I20260709 12:18:42.315562  3494 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:42.314625 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.318667 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 163981us
W20260709 12:18:42.321302 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 291905us
I20260709 12:18:42.269393  3525 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:42.357808 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.324215 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.222868  3303 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.300251 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:42.292953  3423 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.334264 32045 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
W20260709 12:18:42.339840 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.335657  3436 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.006s
I20260709 12:18:42.341229  3535 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:42.357028 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:42.327265  3186 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:42.286976  3521 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.273102  3575 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:42.379582 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.366811 32113 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:42.350510 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2133) took 1566 ms. Trace:
I20260709 12:18:42.351460  3578 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:42.368036 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:42.353452  2911 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:42.347816  3468 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:42.394580  3589 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:18:42.395841 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2126) took 1785 ms. Trace:
I20260709 12:18:42.396196 31972 rpcz_store.cc:276] 0709 12:18:40.610647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.612863 (+  2216us) service_pool.cc:224] Handling call
0709 12:18:40.742260 (+129397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:40.742697 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:40.742736 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.743358 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:18:41.897080 (+1153722us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:41.897120 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.897183 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.395616 (+498433us) raft_consensus.cc:1685] finished
0709 12:18:42.395635 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.395781 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":25073,"thread_start_us":951411,"threads_started":1,"wal-append.queue_time_us":1246598}
W20260709 12:18:42.397341 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.398207 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 98039.1us
I20260709 12:18:42.399052 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:42.337450  3476 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.405364 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2134) took 1569 ms. Trace:
I20260709 12:18:42.405292 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:42.406024 32116 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.362587  3494 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.352358  3573 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.415843 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:42.401149 31964 rpcz_store.cc:276] 0709 12:18:40.784054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.662354 (+878300us) service_pool.cc:224] Handling call
0709 12:18:41.662702 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:41.663089 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.663119 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:41.664157 (+  1038us) log.cc:844] Serialized 52 byte log entry
0709 12:18:42.070462 (+406305us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:42.347691 (+277229us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.349241 (+  1550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.350235 (+   994us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.350278 (+    43us) raft_consensus.cc:1685] finished
0709 12:18:42.350289 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.350446 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322,"prepare.queue_time_us":87399,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":1875,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":19308,"thread_start_us":614415,"threads_started":2,"wal-append.queue_time_us":645707}
I20260709 12:18:42.387013  3386 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:42.422412 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:42.406909  3253 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:42.366254  3572 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:18:42.413007  3577 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:42.419458  3494 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.421995  3326 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.424849  3583 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.159s
I20260709 12:18:42.424880  3386 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:42.405846 31969 rpcz_store.cc:276] 0709 12:18:40.836286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.676928 (+840642us) service_pool.cc:224] Handling call
0709 12:18:41.677224 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.677423 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:42.209367 (+531944us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.404818 (+195451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:42.405108 (+   290us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:42.405157 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.405214 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.405314 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12590,"prepare.queue_time_us":480287,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":65096,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":13882,"thread_start_us":648771,"threads_started":2,"wal-append.queue_time_us":186118}
I20260709 12:18:42.585824 32116 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:42.425449  3317 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.162s
I20260709 12:18:42.649837 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.651172 32119 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:42.652578 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 213752us
I20260709 12:18:42.423030  3399 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.463838  3253 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:42.462693  3501 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:42.659555 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 291467us
W20260709 12:18:42.589975  3597 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:18:42.476243  3520 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.473073  3484 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:42.677330 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 320389us
I20260709 12:18:42.696280 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:42.677947 32116 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:42.678701 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:42.549870  3326 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:42.479879  3538 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:42.685067 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.705103 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.687137 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:42.595773  3603 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.652015  3386 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:42.702214 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:42.550585  3066 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.195s
I20260709 12:18:42.714922 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.421790  3578 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.5, Last appended by leader: 5, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.717273 32116 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:42.658650  3399 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:42.699882  3544 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.669s	user 0.004s	sys 0.007s
I20260709 12:18:42.704550  3326 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.683524  3520 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.696100  3501 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.735361 31964 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:42.771368 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:42.771968 31969 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:42.772701  3605 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:18:42.818305  3602 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.470s	user 0.000s	sys 0.001s
I20260709 12:18:42.719342 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:42.781577 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:42.784864  3544 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.803282 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.719241  3603 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.715308  3591 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:42.720096  3577 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.811630  3606 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 12:18:42.804071  3307 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:42.811736  3621 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:42.719420  3427 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.728928  3399 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.817487 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:42.819553 32653 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 5: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:42.808763  3558 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:42.828289  3501 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.829130 31972 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:42.829303  3610 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:18:42.658567  3468 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:42.866812  3577 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.778129  3526 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:42.813469  3520 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.820947 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.896689 32116 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:42.881091 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.882529 32652 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:42.870800  3427 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.869853  3572 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:42.869038  3303 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:42.907275 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 187952us
I20260709 12:18:42.883821  3476 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:42.909170 31964 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:42.905853 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.896220  3392 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:42.930756 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.930959 31969 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:42.931493 32116 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 11: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:42.932526 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:42.932936 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.916880 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 494565us
I20260709 12:18:42.896531  3581 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:42.919508 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.918954 31972 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
W20260709 12:18:42.938426 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.929003 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:42.915498 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.905032  3427 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.943001 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.836926  3253 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:42.908514  3387 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:42.940639 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.947786 31964 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:42.948341 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.948948 31968 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:42.946035 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.949813 31969 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 4.
I20260709 12:18:42.919700  3303 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.943806 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:42.921898  3572 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:42.966451 31965 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:42.936501 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:42.927124  3541 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:42.945173 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.981485 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.017092 31964 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:42.984386 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:42.946763 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:43.010468 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:43.018766 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 72071.5us
I20260709 12:18:43.019606 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:43.021184 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:43.019762 32115 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:43.021819  3523 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.944504  3579 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:18:42.946400  3155 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.764s	user 0.002s	sys 0.016s
I20260709 12:18:42.951560  3258 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.947315  3387 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:43.070130 32623 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.014s	sys 0.027s
W20260709 12:18:43.070501 32623 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.014s	sys 0.027s
W20260709 12:18:43.070778 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 523369us
I20260709 12:18:42.965993  3303 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:43.042958  3436 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:43.043191  3514 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:43.048501  3253 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.045130  3579 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:43.064671  3258 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.114996  3476 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.042634  3484 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:18:43.147030  3303 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.203406  3572 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:43.216058  3453 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.267s	user 0.011s	sys 0.004s
W20260709 12:18:43.221375  3411 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.953s	user 0.006s	sys 0.004s
I20260709 12:18:43.222749  3588 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:43.223233  3588 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:43.224015  3387 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.224197  3588 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.225279  3387 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:43.228570  3522 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.251s	user 0.010s	sys 0.000s
I20260709 12:18:43.229115  3522 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.231256 32121 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:43.232637  3541 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:43.233126  3541 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:43.233351  3396 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.969s	user 0.008s	sys 0.002s
I20260709 12:18:43.233525  3541 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.188922  3258 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:43.247203 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 310807us
W20260709 12:18:43.205027 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 526415us
I20260709 12:18:43.146268  3475 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.068785  3155 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:43.248203 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 227108us
I20260709 12:18:43.251622 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:43.254184  3475 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.255340  3625 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.255834  3625 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.256367  3475 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:18:43.257022  3475 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:43.263128  3603 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.264477  3577 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:43.258016  3625 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:43.268702  3507 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:43.271831  3495 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.221s	user 0.010s	sys 0.000s
I20260709 12:18:43.180052  3514 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:43.277410 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.183465  3436 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:43.287338  3558 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:43.287869 32653 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 12. Preceding OpId from leader: term: 11 index: 13. (index mismatch)
I20260709 12:18:43.287895  3558 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:43.291378  3377 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.479s	user 0.021s	sys 0.020s
I20260709 12:18:43.291419  3558 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.291951  3377 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:43.295885  3528 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.036s	user 0.007s	sys 0.014s
I20260709 12:18:43.281322  3514 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:43.297113 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.303668 32656 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:43.277614 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:43.221932  3411 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.279146  3625 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:43.245525  3476 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.302064  3541 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.280606  3495 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.325330  3409 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:43.313032  3544 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.373368  3541 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.326114  3579 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:43.375170  3525 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.007s	user 0.028s	sys 0.014s
I20260709 12:18:43.377403  3525 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:43.380033  3543 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.030s	user 0.016s	sys 0.000s
I20260709 12:18:43.380059  3525 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.380893  3543 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.381664  3579 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.383145  3520 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.386456  3520 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.388799  3495 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.327246  3514 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.365788  3501 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.441586 31971 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:43.443646 32120 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:43.447678 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 242145us
I20260709 12:18:43.317032  2897 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.451427 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:43.453401 32653 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:43.449767  3155 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.323155  3377 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.461851 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:43.470456 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:43.318478  3258 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.312722 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:43.465620  3155 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:18:43.483603  3258 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.485550  2897 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:43.486240  2897 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.486799  2897 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:43.486842 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 174083us
W20260709 12:18:43.463419  3386 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.726s	user 0.020s	sys 0.009s
I20260709 12:18:43.365571  3409 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:43.491171 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.492462 31966 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:43.492317  3409 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.477133  3411 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:43.495119  3386 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.494120  3409 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:43.499136 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:43.499528 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.500181 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.500509 31966 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:18:43.504475 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 457) took 1603 ms. Trace:
I20260709 12:18:43.504923 32113 rpcz_store.cc:276] 0709 12:18:41.901138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.902401 (+  1263us) service_pool.cc:224] Handling call
0709 12:18:41.902663 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.389198 (+486535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.503902 (+1114704us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54004344d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:43.504238 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066883456}
I20260709 12:18:43.483476 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:43.509722 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2193) took 1484 ms. Trace:
W20260709 12:18:43.510783  3326 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.692s	user 0.022s	sys 0.029s
I20260709 12:18:43.511390  3326 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.515893  2897 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:43.521658 32656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 755) took 1545 ms (client timeout 1781 ms). Trace:
I20260709 12:18:43.522377  3411 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.523583  3411 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.510216 32119 rpcz_store.cc:276] 0709 12:18:42.024764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.651848 (+627084us) service_pool.cc:224] Handling call
0709 12:18:42.652200 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.729626 (+ 77426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.729670 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.730201 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.062893 (+332692us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:43.062953 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.509047 (+446094us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b54001496d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:43.509130 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.509209 (+    79us) raft_consensus.cc:1685] finished
0709 12:18:43.509224 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.509543 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":175198,"spinlock_wait_cycles":188123904,"thread_start_us":250281,"threads_started":1,"wal-append.queue_time_us":375646}
W20260709 12:18:43.531410 32045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 425) took 1663 ms (client timeout 1749 ms). Trace:
W20260709 12:18:43.531963 32045 rpcz_store.cc:269] 0709 12:18:41.868277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.916481 (+ 48204us) service_pool.cc:224] Handling call
0709 12:18:43.530966 (+1614485us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54000d0f54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:43.531179 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667993856}
I20260709 12:18:43.521711 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.536226 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:43.538828 32120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2195) took 1512 ms (client timeout 1967 ms). Trace:
W20260709 12:18:43.539316 32120 rpcz_store.cc:269] 0709 12:18:42.026492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.323958 (+297466us) service_pool.cc:224] Handling call
0709 12:18:43.538502 (+1214544us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540043a3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:43.538692 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490891136}
I20260709 12:18:43.543813 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 650) took 1593 ms. Trace:
I20260709 12:18:43.544230 32652 rpcz_store.cc:276] 0709 12:18:41.950516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.950639 (+   123us) service_pool.cc:224] Handling call
0709 12:18:41.950917 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.898532 (+947615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.543265 (+644733us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b5400154d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:43.543596 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141967232}
I20260709 12:18:43.546387  3377 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:43.522157 32656 rpcz_store.cc:269] 0709 12:18:41.976396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.976532 (+   136us) service_pool.cc:224] Handling call
0709 12:18:43.521279 (+1544747us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54004c13d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:43.521467 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2950549376}
I20260709 12:18:43.547377  3377 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.549706 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:43.550199  3377 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.550777 32655 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:43.550486  3258 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.552085  3258 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.547849 32115 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:43.553630 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.549655  3210 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.554778 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:43.555684 32659 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:43.555704  3258 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.573252 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:43.574100 32112 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:43.574433 32010 connection.cc:256] RPC call timeout handler was delayed by -1.0601910989998942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
W20260709 12:18:43.575130  2911 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.163s	user 0.024s	sys 0.035s
I20260709 12:18:43.575685  2911 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:43.554225 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.571172 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1875) took 1431 ms. Trace:
I20260709 12:18:43.578282 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:43.579306 32113 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.581310 31967 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 9. Ops: []
I20260709 12:18:43.577205 31971 rpcz_store.cc:276] 0709 12:18:42.140010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.140154 (+   144us) service_pool.cc:224] Handling call
0709 12:18:42.140440 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.442951 (+1302511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.570140 (+127189us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400068454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:43.570977 (+   837us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2990562688}
I20260709 12:18:43.584408 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.577435 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.585486 32040 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:43.585588 32121 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:43.586735 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:43.587695 32652 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:43.588122 32045 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:43.590221  3210 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.590799 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.591627 32038 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:43.592496  2911 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.593415  3210 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.593608 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.595045 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:43.595850 32040 raft_consensus.cc:2376] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15. Re-sending same reply.
I20260709 12:18:43.596386 32120 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:43.596761 32120 pending_rounds.cc:85] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 4
I20260709 12:18:43.597077 32120 pending_rounds.cc:107] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:43.597483 32120 raft_consensus.cc:2889] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.599258 32115 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:43.600597 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.601455 32121 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:43.602699  3386 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:43.607311  3428 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.290s	user 0.011s	sys 0.007s
I20260709 12:18:43.607594  3522 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:43.608486  3428 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.369190  3436 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.611471  3468 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.612572  3468 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.616518  3475 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:43.618783 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.620249  3522 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:43.621238 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.621037  3522 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.621961 32119 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:43.635301 32112 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:43.688123 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:43.612584  3476 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.686605 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.689846 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.697297 32115 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:43.702741 31967 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:43.704252 32112 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.696903 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.715301 32038 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:43.720899  3427 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.001s
I20260709 12:18:43.722405 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:43.722766 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.723214 32041 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.723645 32042 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:43.727250 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.730031 32120 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 5.
I20260709 12:18:43.721942 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.732868 32112 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
W20260709 12:18:43.718007  3423 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.331s	user 0.017s	sys 0.011s
I20260709 12:18:43.702749  3543 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.734779 32121 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:43.738049 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:43.738548 32652 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:18:43.738963 32042 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
I20260709 12:18:43.739213  3514 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.742662 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.748530 32119 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:18:43.748896 32119 pending_rounds.cc:85] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 6
I20260709 12:18:43.754117 32119 pending_rounds.cc:107] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:43.754449 32119 raft_consensus.cc:2889] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.755820  3544 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:43.756291  2897 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.756027  3495 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:43.755910 32047 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:43.757107  3326 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:43.763185 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:43.762108  3186 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.355s	user 0.019s	sys 0.031s
I20260709 12:18:43.765513  3523 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.765730  3186 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.766279  3523 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.766750 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.767624 32039 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:43.740922  3633 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.768008  3186 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:43.770834 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.771644  3399 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.879s	user 0.011s	sys 0.010s
I20260709 12:18:43.750130  3386 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.773415 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 290211us
I20260709 12:18:43.775074  3640 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:43.775908 31963 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:43.776822  3495 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.778759 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:43.779754  3494 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.190s	user 0.017s	sys 0.019s
I20260709 12:18:43.780077 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:43.780561  3378 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.781536 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:43.747853  3423 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.780598 32038 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:43.784397  3468 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.785167  3428 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.785611 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.789746  3588 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.794407 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.795676 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.795986 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.798920 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.801419 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.802146  3596 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:43.802968 32045 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:43.803494 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.804207  3633 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:43.830956 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 254804us
I20260709 12:18:43.806596 32113 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
W20260709 12:18:43.772651  3458 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.425s	user 0.026s	sys 0.001s
I20260709 12:18:43.831854 32041 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:43.834784 32652 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.721545  3427 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.826494  3522 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:43.836637  3386 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.827235  3640 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:43.841447 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:43.838685  3494 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:43.860920 32010 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.871s (SENT)
I20260709 12:18:43.861236 31966 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.861456 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:43.851063 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:43.862924 32038 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 16 because replica has last-logged OpId of term: 13 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 4.
I20260709 12:18:43.842125  3458 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:43.862815  3494 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.865324  3458 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:43.854210  3378 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:43.850867 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.847102 32120 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:43.853616 31971 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:43.841921 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:43.846623 31965 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:43.859586 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:43.869614 31968 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.851140 31964 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:43.854084 31967 raft_consensus.cc:2376] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13. Re-sending same reply.
I20260709 12:18:43.858649 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:43.842557 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.833068  3596 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:43.853356  3427 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:43.904099 32117 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:43.904587  3378 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:43.905933  3573 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.481s	user 0.007s	sys 0.028s
I20260709 12:18:43.906338 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:43.868141  3640 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.910972 32119 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:43.911115 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:43.844756 32121 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:43.903174 32656 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:43.903303 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44754.9us
I20260709 12:18:43.853873 32043 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:43.880929 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.914273 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.853894  3098 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.518s	user 0.015s	sys 0.015s
I20260709 12:18:43.917042 31969 raft_consensus.cc:2410] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:43.918494 32652 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:43.910149 31966 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
W20260709 12:18:43.851215  3303 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.578s	user 0.014s	sys 0.001s
I20260709 12:18:43.932379 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:43.933240 32655 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:43.876816  3523 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:43.934046 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 484144us
W20260709 12:18:43.938669 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 85993.4us
I20260709 12:18:43.830899  3399 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:43.857273  3633 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.143983 31968 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:43.972483 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.163224 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 302315us
I20260709 12:18:43.928653  3458 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:43.974362 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.939155 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.091003 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:44.088980 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:44.110576 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.263055 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.242868 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:44.243912  3411 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:44.241654 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.959223 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:43.960945 32042 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.267144 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:44.162348  3501 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.247306 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2194) took 2221 ms. Trace:
I20260709 12:18:43.932438  3427 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:44.284701 32657 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:44.286091 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.286469  3399 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:44.286881 31965 raft_consensus.cc:2376] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10. Re-sending same reply.
I20260709 12:18:44.287227 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.287808 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.287976 31967 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:44.288977 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.289012 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.289794 31967 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:44.289674 31965 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:44.290668 31966 raft_consensus.cc:2368] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12.
I20260709 12:18:44.290746 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.291438 32119 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:44.291568 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.292404 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:44.292630 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.293097 32117 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.295825 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.298614  3528 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:44.293176 31966 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.302429 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:44.303090 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.303171 32120 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:18:44.316280  3535 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.929s	user 0.025s	sys 0.010s
I20260709 12:18:44.317255 32121 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:44.318128 32117 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
W20260709 12:18:44.318667 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.318457  3501 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:44.319085 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.306972 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.319172 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:44.319613  3468 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.320119  3468 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.305576 32652 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:44.305759  3423 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.306766 32651 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:44.307317 32660 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:44.240954  3596 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.307519 31963 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:44.323220  3596 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.323662 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.324299 32120 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:44.307790 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:44.308131  3575 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:44.324980  3575 consensus_queue.cc:1243] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:44.260944  3458 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.326591  3458 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:44.308861 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.320303 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.327649  3378 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:44.327904 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.328675 31964 raft_consensus.cc:2368] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13.
I20260709 12:18:44.329375 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.329998 31964 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:44.326197  3575 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:44.330878  3575 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:44.331653 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.324452 31963 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 7: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:44.307861 32114 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:44.332171  3575 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.305429 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:44.335294 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 395933us
I20260709 12:18:44.335417 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:44.335804 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2201) took 1629 ms. Trace:
I20260709 12:18:44.336266 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:44.337338  3423 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.338624 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.339437 31967 raft_consensus.cc:2376] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11. Re-sending same reply.
I20260709 12:18:44.339737  3640 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.340372 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.340912 32659 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:44.341454 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.343581 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:44.344412 31963 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 9: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:44.345659 31963 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:44.318404 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.347481 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:44.342089 31967 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12. Re-sending same reply.
I20260709 12:18:44.363121 31966 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:44.348875 32042 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:44.349208 31969 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:44.350405  3468 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.354071 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 757) took 2151 ms. Trace:
I20260709 12:18:44.330932 31965 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:44.330824  3622 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:18:44.355315 32120 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 4. Preceding OpId from leader: term: 13 index: 5. (index mismatch)
I20260709 12:18:44.336176 32114 rpcz_store.cc:276] 0709 12:18:42.706391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.947964 (+241573us) service_pool.cc:224] Handling call
0709 12:18:43.043710 (+ 95746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.332458 (+1288748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.335584 (+  3126us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400149bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:44.335753 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2814224256}
I20260709 12:18:44.337966 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:44.364622 31966 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:44.320343 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:44.368780 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 98605.4us
W20260709 12:18:44.369489 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.370402 32119 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 9: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:44.374203 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:44.370581  3399 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.368221 32657 rpcz_store.cc:276] 0709 12:18:42.203094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.204023 (+   929us) service_pool.cc:224] Handling call
0709 12:18:42.235310 (+ 31287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.285227 (+2049917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.353789 (+ 68562us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54003efb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:44.353997 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3773741440}
W20260709 12:18:44.391145 32113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.014s	sys 0.016s
I20260709 12:18:44.377583  3423 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.343576  3640 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.378737 31963 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:44.380324 32659 pending_rounds.cc:85] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 8
I20260709 12:18:44.395280 32659 pending_rounds.cc:107] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:18:44.395565 32659 raft_consensus.cc:2889] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:44.382541 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 111415us
I20260709 12:18:43.930218  3573 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.396821 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:44.397730 32042 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:44.388139 32121 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:44.399403 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:43.962445  3098 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.384433  3528 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:44.384999 32117 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:44.385222 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47435.3us
I20260709 12:18:43.945601  3303 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:44.391503 32113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.014s	sys 0.016s
W20260709 12:18:44.393101 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.394227 31968 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:44.392871 31971 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:44.400169 32656 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:44.430141 32039 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:44.271312 32118 rpcz_store.cc:276] 0709 12:18:42.026210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.680493 (+654283us) service_pool.cc:224] Handling call
0709 12:18:42.680957 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.683898 (+  2941us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:42.709299 (+ 25401us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.536679 (+827380us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b18004479a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:44.100622 (+563943us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.165736 (+ 65114us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:44.244878 (+ 79142us) log.cc:844] Serialized 26 byte log entry
0709 12:18:44.246351 (+  1473us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:44.246418 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.246563 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.246758 (+   195us) raft_consensus.cc:1685] finished
0709 12:18:44.246773 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.247050 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87935,"prepare.queue_time_us":25330,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":6450,"raft.queue_time_us":91859,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":19872,"spinlock_wait_cycles":12735872,"thread_start_us":475630,"threads_started":2,"wal-append.queue_time_us":973492}
W20260709 12:18:44.419194  3588 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 12:18:44.424858  3528 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:44.393721  3458 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.441408 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:44.447558 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:44.448698 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.448834 32654 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:44.449566 32042 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.423769  3399 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:44.457648 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.460943  3523 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.461776  3525 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:44.467236 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.471014 32042 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:44.439925  3411 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:44.447017  3528 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.446131  3535 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.454097  3386 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.479732  3522 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.490804 32114 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:18:44.491447 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 49028.6us
I20260709 12:18:44.485890  3378 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.497980 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:44.514815 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.515553 32117 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:44.516592 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.518115 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.520640 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.518894 32118 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
W20260709 12:18:44.526546 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48369.5us
I20260709 12:18:44.539144 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:44.539144 32120 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:18:44.541139 31970 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:44.542447  3522 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:44.544937  3522 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:44.545538  3522 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:44.547425 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:44.548369 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56678.2us
I20260709 12:18:44.496428  3378 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:44.550447 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:44.487490  3523 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:44.487778  3588 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:18:44.557976 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:44.518116  3386 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:44.562155  3386 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:44.564343  3386 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.492525  3411 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:44.557127 32121 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:44.549168  3423 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:44.577050 31964 raft_consensus.cc:2410] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 12:18:44.579247 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44939.6us
I20260709 12:18:44.592581 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:44.507964  3535 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:44.552790  3578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.016s	sys 0.022s
I20260709 12:18:44.507570  3525 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:18:44.615985 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:44.614156  3578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.016s	sys 0.023s
I20260709 12:18:44.622776  3303 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.625126  3640 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:44.625747  3640 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:44.627066  3411 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:44.626730  3640 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.627660  3411 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:44.628015  3411 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:44.628911  3411 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.629400  3507 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.629917  3507 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:44.630193 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:44.631649  3458 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.632010  3507 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.634642  3523 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:44.637640  3399 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:44.622800  3098 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.641470  3573 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:44.624859  3303 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:44.646637  3303 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:44.647043  3303 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:44.648000  3303 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.649340  3303 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:44.653151  3654 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
W20260709 12:18:44.657266 32654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 756) took 2679 ms (client timeout 2665 ms). Trace:
W20260709 12:18:44.657686 32654 rpcz_store.cc:269] 0709 12:18:41.978052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.978161 (+   109us) service_pool.cc:224] Handling call
0709 12:18:44.657037 (+2678876us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b5400603bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:44.657175 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5558005120}
I20260709 12:18:44.660876  3378 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.662062  3501 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.584582  3423 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.666687  3423 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.672416  3535 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:44.613657 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:44.673533 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60001us
I20260709 12:18:44.674672  3588 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:44.675380  3588 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:44.676466  3588 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.674170 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:44.821596  3423 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.822737  3528 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:44.825332  3423 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:44.826020  3528 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:44.930538 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.942507  3495 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:45.024641 32112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.010s	sys 0.045s
W20260709 12:18:45.025058 32112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.011s	sys 0.046s
I20260709 12:18:45.025906 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2089) took 1294 ms. Trace:
W20260709 12:18:45.029305 32659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.012s
W20260709 12:18:45.029696 32659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.000s	sys 0.013s
I20260709 12:18:45.031013  3507 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:45.031452  3507 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.032357  3507 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.026288 32112 rpcz_store.cc:276] 0709 12:18:43.734097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.734349 (+   252us) service_pool.cc:224] Handling call
0709 12:18:43.747658 (+ 13309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.801080 (+ 53422us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:44.097243 (+296163us) log.cc:844] Serialized 14 byte log entry
0709 12:18:45.025502 (+928259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:45.025564 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:45.025617 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.025694 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.025841 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":512332,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1556,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":1040201,"threads_started":2,"wal-append.queue_time_us":532180}
W20260709 12:18:45.035187 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 383556us
W20260709 12:18:45.050757 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 389) took 2276 ms (client timeout 2062 ms). Trace:
W20260709 12:18:45.051676 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.013s	sys 0.008s
W20260709 12:18:45.052177 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.013s	sys 0.008s
W20260709 12:18:45.052538 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 578014us
I20260709 12:18:45.062711  3468 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:45.063222  3468 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:45.064278  3468 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.051728 31970 rpcz_store.cc:269] 0709 12:18:42.774833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.775150 (+   317us) service_pool.cc:224] Handling call
0709 12:18:45.050448 (+2275298us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400312c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:45.050658 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3685310720}
I20260709 12:18:45.067791  3666 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:45.068449  3666 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.068987 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.070585  3666 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:45.073405 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.074047 32042 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:18:45.074110 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 399978us
I20260709 12:18:45.074891 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.075915 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.077190  3458 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:45.077327  3588 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:45.077611  3458 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.078032  3588 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.080037  3588 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:45.078183  3458 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:45.088701 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.089593  3528 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.090130  3528 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.090968 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.092175  3528 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:45.092417 32045 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:45.092713  3528 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:45.093143  3528 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.094837 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.095590 32047 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.095744  3528 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:45.098730 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62828.9us
I20260709 12:18:45.102934 32045 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:45.103219  3468 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.103935 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.104492 32038 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.104696 32042 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:45.105839 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.107682 32044 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 12. Preceding OpId from leader: term: 11 index: 13. (index mismatch)
I20260709 12:18:45.109438 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.110157 32040 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:18:45.112062 32660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.018s	sys 0.072s
W20260709 12:18:45.112413 32660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.018s	sys 0.072s
I20260709 12:18:45.112588 32038 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:45.077301  3640 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:45.116367 32653 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.046s	sys 0.044s
I20260709 12:18:45.116459  3640 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:45.116771 32653 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.046s	sys 0.044s
I20260709 12:18:45.117054  3640 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.120342 31972 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:45.123317 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2146) took 1330 ms. Trace:
I20260709 12:18:45.124985 31972 rpcz_store.cc:276] 0709 12:18:43.793271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.794427 (+  1156us) service_pool.cc:224] Handling call
0709 12:18:45.120006 (+1325579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:45.120759 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.123075 (+  2316us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400312c50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:45.123236 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047604608}
I20260709 12:18:45.128123 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.128950 32112 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:45.128749 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.129941 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.130671 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.132145 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.132325 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.132803 32119 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:45.130165 32116 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:45.134035 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.135542 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:45.138994 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.141119 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:45.141466 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.143082 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.143453 32112 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.143666 32121 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:45.144820 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.148780 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:45.150812 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.153081 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.153304 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:45.153824 32112 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:45.155416 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.158565 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.162930 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.150079 32047 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:45.168514 32114 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:45.165169 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:45.175318  3525 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:45.176014  3525 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.176173 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.176442  3525 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.182819 32119 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.189292 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.188738  3525 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.189640  3186 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:45.216328  3186 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:45.216722  3186 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.217384  3525 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:45.217908 32115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.006s	sys 0.006s
W20260709 12:18:45.218258 32115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.007s	sys 0.006s
I20260709 12:18:45.219461 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.220186 32115 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:45.220924 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.015s	sys 0.006s
W20260709 12:18:45.221235 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.015s	sys 0.006s
I20260709 12:18:45.221648 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.222262 32115 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:18:45.222590 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.222664 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.223207 32119 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.223505 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.225052 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.225633 32115 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:45.226679 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1894) took 1374 ms. Trace:
I20260709 12:18:45.227056 31964 rpcz_store.cc:276] 0709 12:18:43.852070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.330546 (+478476us) service_pool.cc:224] Handling call
0709 12:18:45.226489 (+895943us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54004560d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:45.226627 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485414656}
I20260709 12:18:45.175406 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:45.191653  3681 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:45.235175  3663 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:45.198154 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:45.201498 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.000s	sys 0.013s
I20260709 12:18:45.241371 32117 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:18:45.241458 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.000s	sys 0.015s
W20260709 12:18:45.242026 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 671976us
W20260709 12:18:45.203460 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.000s	sys 0.021s
I20260709 12:18:45.205158 32116 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:45.235299  3681 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:45.244434  3681 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.250660  3303 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:45.251027 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 75698.4us
I20260709 12:18:45.020169 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:45.243027 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.000s	sys 0.021s
W20260709 12:18:45.252578 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 753468us
I20260709 12:18:45.195029  3501 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.253506 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.235898  3663 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.254711 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.255182 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.243786 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.251287  3303 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.256714  3663 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.257033  3303 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.257570  3672 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:18:45.260221 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.262732 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.264676 32660 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:45.244292  3664 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.267076 32084 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
I20260709 12:18:45.267601 32084 leader_election.cc:403] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:18:45.267307  3664 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.268952 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:45.270182  3671 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:45.270654  3669 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.270776  3671 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.271469  3671 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.265969 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:45.274778 32659 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:45.275998 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.758s (SENT) [suppressed 33 similar messages]
I20260709 12:18:45.275532 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.280668 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.281440 32047 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:45.283257 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.284416 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.284560 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.284983 32041 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.285565 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.285810 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.217595  3186 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.253898 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.290832 32039 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:45.290938  3663 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:45.291411 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.300724 32043 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:45.291734 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:45.301618 31965 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.303123  3663 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:45.303951  3663 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:45.304275  3663 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.304601  3663 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:45.292368 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:45.306614 31972 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.311882  3663 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.313843 31972 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:45.314081  3663 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:45.316689 31965 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:45.299088 32041 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:45.291414 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:45.296171 32117 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:45.318696 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.319424 32117 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.329675 32117 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:45.333364 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.242087  3525 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:45.335986 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.293756  3186 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.457547  3681 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.463912  3528 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:45.391050 32119 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:45.401791 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.402647 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.402944 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.442929 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:45.489135 32659 raft_consensus.cc:2410] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:45.489352 31965 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:45.443078 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.490679 32038 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:45.443598 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:45.443714 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:45.445040 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.349301 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:45.389916 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:45.492296 31963 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.466883 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.402657 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.337873  3664 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.492125 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:45.497743 32660 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:45.501968 32040 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:45.501009 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:45.507191 32041 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:45.457953  3588 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.392549  3458 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.522799 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 238420us
I20260709 12:18:45.530325 32654 raft_consensus.cc:2368] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:45.542938 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.558023 31963 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:45.562477  3692 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.565146 32660 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.567291  3672 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:45.568333  3672 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:45.568749  3672 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.545626  3528 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:45.554147  3669 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
W20260709 12:18:45.578711 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 324907us
I20260709 12:18:45.587862 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.589730 32047 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:45.570672  3690 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:45.602833 32114 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:45.604230 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:45.604578 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61712.1us
I20260709 12:18:45.569756  3672 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.608938 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55005.8us
I20260709 12:18:45.617022 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.604214  3664 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:45.621130 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.622372 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.617285 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.604084  3669 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.606256  3690 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:45.642258 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.623415  3688 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:45.626407  3664 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.630283  3696 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:45.645859 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.651333 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.647687  3303 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:45.664812 32115 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:45.676002 32659 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:45.667208 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60051.2us
I20260709 12:18:45.643297  3691 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.675428 32660 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:45.649794  3640 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.532s	user 0.016s	sys 0.000s
I20260709 12:18:45.627490  3681 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:45.667546  3685 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:18:45.724061 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 109150us
I20260709 12:18:45.663276  3458 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:45.663405  3664 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.656505  3694 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:45.675710 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.682519  3186 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:45.678844  3671 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.736228  3436 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:45.737069  3436 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.737524  3436 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.671861  3501 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.664444  3688 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.664269  3528 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.732288  3686 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:18:45.734484  3681 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.735416  3672 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.746829 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:45.725286  3691 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.736110 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.733685  3694 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.738801 32112 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.737095 32113 raft_consensus.cc:2368] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12.
I20260709 12:18:45.738431  3692 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:45.742288  3501 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.676047  3699 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.753890  3692 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.754940 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.23531e+06us
I20260709 12:18:45.755183  3501 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:45.755944 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:45.754820  3699 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.758432  3699 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:45.767412 32118 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:45.767884 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2110) took 1213 ms. Trace:
I20260709 12:18:45.764199 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.738488  3535 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.768707  3535 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.648801  3669 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.667233  3303 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:45.761644 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 652753us
W20260709 12:18:45.760737 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.743091  3681 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.762797  3484 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:45.752082  3525 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:45.764127 32659 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:45.764437  3700 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:45.765614  3436 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.767827  3697 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.739240 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:45.741078  3671 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.770291 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.770388 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.772917 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.789674 32112 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:45.790611 32653 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:45.664570  3696 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.780737 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.780844 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:45.781735 32119 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:45.782007 32660 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.793619 32657 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:45.768347 32118 rpcz_store.cc:276] 0709 12:18:44.554256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.559257 (+  5001us) service_pool.cc:224] Handling call
0709 12:18:45.767827 (+1208570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:45.794729 32659 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:45.773506  3484 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.789269  3694 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.771243  3303 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.796528  3484 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.797851  3484 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.773619  3525 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.799237 32084 leader_election.cc:400] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12.
I20260709 12:18:45.799664 32084 leader_election.cc:403] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:45.799163  3525 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.800271 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61538.1us
I20260709 12:18:45.800614  3525 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.800802 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.800993 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.801756  3501 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:45.774812  3681 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:45.775050  3686 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:18:45.803663  3686 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:45.804005  3686 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.804809  3686 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.775034  3700 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:45.775113  3669 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.806520  3700 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:45.779198  3693 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:18:45.799698 32656 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:45.811298  3665 raft_consensus.cc:2764] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:45.774206  3399 raft_consensus.cc:2764] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:45.774516  3690 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.812307  3669 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:45.812389  3697 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12.
I20260709 12:18:45.806871  3688 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.771531  3535 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.811653  3436 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:45.811395 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.724270  3640 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:45.811414  3468 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.004s
I20260709 12:18:45.815837 32660 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:45.812980  3669 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:45.789412  3691 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:45.811652  3686 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:45.814538  3186 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:45.823850 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.846975 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:45.851781 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.823186  3535 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:45.856348 32112 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:45.857342 32118 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:45.853340  3468 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.861635  3468 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.863407 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.863914  3468 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:45.864869 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.866602  3501 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.866550  3303 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:45.866549 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55267.7us
I20260709 12:18:45.866158 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.868191 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.870893 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.873211 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.873905  3691 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.876806 32010 leader_election.cc:400] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13.
I20260709 12:18:45.878381 32010 leader_election.cc:403] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:18:45.880183 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.873463  3694 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.883965 32120 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:45.885013  3694 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:45.867280  3303 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:45.885593  3303 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:45.886118  3672 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:45.863448  3696 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:45.874423  3703 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:45.888526 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.889947 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:45.867400  3501 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.881939  3686 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13.
I20260709 12:18:45.890450  3703 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:45.891408  3696 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:45.891912  3696 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:45.892225  3522 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:45.892864  3696 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.877485  3593 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:45.904407 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:45.905377 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.906142 32043 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:45.906934 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.907470 32043 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:18:45.908172 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.909094 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.916688  3468 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.916880  3528 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:45.916996  3694 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:45.917236  3468 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.917670 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.917773  3694 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:45.918880 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.919459 32040 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:45.854238  3436 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.874725  3697 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:45.932271  3423 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:45.919092  3694 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.934600 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.935143 32116 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.935832 32660 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:45.937218 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.937281  3703 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:45.937868 32114 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:45.881906  3578 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.882236  3699 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:45.938561 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.939137 32112 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 5.
I20260709 12:18:45.942502  3522 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:45.943634 32047 raft_consensus.cc:1217] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[9.8-13.10]   Dedup: 9.8->[11.9-13.10]
I20260709 12:18:45.944247 32047 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: [9.8-13.10]
I20260709 12:18:45.919575  3468 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:45.944847 32114 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:45.945317 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.945791  3507 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.945933 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.946026  3700 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.947403 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.897223 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.898582 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.900220 32045 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:45.901775 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.948864 32038 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:45.949626 32046 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:18:45.920958 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.950062 32042 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.927256  3689 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:45.929785 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.951334 32041 raft_consensus.cc:2410] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:18:45.929876 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.930462  3669 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.931402 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:45.931740  3436 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.932420 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.897914 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.874656  3664 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.958163 32039 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:45.951545 32116 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:45.952574 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:45.950250 32118 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:45.943682 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:45.953929 32040 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:45.962631 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:45.963488 32038 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:45.955066 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:45.954983  3303 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:45.955924  3700 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:45.955868 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.970194 32117 raft_consensus.cc:2368] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:45.964851 32115 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:45.966677 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.967368 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.972043 32046 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:18:45.968609  3423 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:45.972864  3423 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:45.973222  3423 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:45.965302  3468 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:45.979193  3468 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.979611  3468 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:45.978763  3186 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:45.981016  3699 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:45.981415  3699 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:45.978801 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.983366 32039 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:45.985162  3689 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:45.959385  3703 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:45.987008  3703 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.988147  3501 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.988885  3501 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:45.968891 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:45.990901 32041 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:45.969143 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:45.994119 32013 leader_election.cc:400] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12.
I20260709 12:18:45.996004 32013 leader_election.cc:403] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:18:45.998013  3686 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.001489 32044 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:46.002436  3699 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.003545 32045 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:45.962041 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.004252  3640 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.004849  3640 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:46.005330 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.006184 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.006444 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:46.006990 32117 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.007156  3681 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:46.007396  3528 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:46.007560  3681 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.008023  3528 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:46.005302  3640 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:46.008821 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:46.009017  3528 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.009112  3640 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.003731  3699 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.012389  3378 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.987898  3703 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.013145  3378 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:46.014940 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83690us
I20260709 12:18:46.015600 32013 leader_election.cc:400] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:46.015926 32013 leader_election.cc:403] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:46.008039  3681 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.018821 32115 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:18:46.023675 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:46.021100 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.027529 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:46.028714 32043 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:46.033792 32117 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:46.033447 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.041113 32041 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:46.040751 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.068185 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:46.040434 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:46.071781 32046 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:18:46.156002 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.173559 32042 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:46.020781  3522 raft_consensus.cc:2764] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:46.124053  3507 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:46.181787 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 146533us
W20260709 12:18:46.188356 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 120247us
I20260709 12:18:45.990061  3501 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.071128  3689 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.067574  3458 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:46.181957 32681 debug-util.cc:398] Leaking SignalData structure 0x7b080025c5a0 after lost signal to thread 30823
W20260709 12:18:46.208068 32681 debug-util.cc:398] Leaking SignalData structure 0x7b080024f280 after lost signal to thread 31904
W20260709 12:18:46.209620 32681 debug-util.cc:398] Leaking SignalData structure 0x7b080025c140 after lost signal to thread 31996
W20260709 12:18:46.209986 32681 debug-util.cc:398] Leaking SignalData structure 0x7b080025c1e0 after lost signal to thread 31997
I20260709 12:18:46.211015  3458 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.192016  3484 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:46.195416  3694 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.139279  3468 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:46.229056 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08002e2900 after lost signal to thread 32071
W20260709 12:18:46.251569 32681 debug-util.cc:398] Leaking SignalData structure 0x7b080039db00 after lost signal to thread 32145
W20260709 12:18:46.251961 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08001421e0 after lost signal to thread 32146
I20260709 12:18:46.212162  3689 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:46.219473 32047 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:46.220129  3458 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:46.111472 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78120.5us
W20260709 12:18:46.269068 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 253417us
W20260709 12:18:46.270862 32681 debug-util.cc:398] Leaking SignalData structure 0x7b0800345240 after lost signal to thread 32684
I20260709 12:18:46.270530 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:46.247728  3694 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:46.274853 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08003324a0 after lost signal to thread 32685
I20260709 12:18:46.260386  3689 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.215031  3528 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:46.285398 32039 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:46.034305  3664 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.273458  3697 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:46.287165  3672 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:46.272487  3378 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:46.283252  3694 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.287036  3666 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9.
I20260709 12:18:46.312135 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:46.330554 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.335209 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:46.315850  3707 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:18:46.338873 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:46.339661 32117 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.275177  3688 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:18:46.320313  3303 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:46.352882 32117 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:46.356128  3528 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:46.359357  3528 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:46.364634 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.336591  3672 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:46.375445 31858 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.493s	user 0.001s	sys 0.000s
W20260709 12:18:46.399619 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 123787us
I20260709 12:18:46.365635  3697 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:46.400295 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:46.400839 32117 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:46.402933 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:46.392912  3525 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.352520  3423 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.413026 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:46.416055  3468 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:46.417783  3303 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:46.418256  3714 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:18:46.418385  3303 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:46.420181  3710 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:18:46.422966 32118 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:46.423380 32117 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:46.426796 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:46.434054 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 121884us
I20260709 12:18:46.436314 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:46.441824 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:46.442472 32117 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.434849 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.440405 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:46.430016  3700 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:46.419464  3303 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.413002  3525 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.436879  3690 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:46.444260  3697 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:46.443723  3588 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.469383 32117 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:18:46.434373  3664 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 12:18:46.476275 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.450639  3507 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:18:46.486768  3720 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:46.490252  3525 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:46.491739  3633 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:46.492552  3686 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12.
W20260709 12:18:46.493093 32044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2075) took 1357 ms (client timeout 1633 ms). Trace:
W20260709 12:18:46.493502 32044 rpcz_store.cc:269] 0709 12:18:45.135203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.135313 (+   110us) service_pool.cc:224] Handling call
0709 12:18:46.492899 (+1357586us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b5400058cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:46.493037 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1989439232}
I20260709 12:18:46.453488  3696 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:46.470259  3697 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:46.472941  3711 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:18:46.458572  3672 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.5, Last appended by leader: 5, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.461551  3718 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:18:46.470165  3700 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:46.498756  3697 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.500574  3718 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:46.506996  3640 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:46.548650  3718 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:46.549674  3718 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.525621  3411 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.502599  3700 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.550817  3700 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.550952  3718 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:46.503456  3695 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.835s	user 0.005s	sys 0.007s
I20260709 12:18:46.505864  3681 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.556442  3681 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.558799 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:46.538571 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08003c3e60 after lost signal to thread 3573
W20260709 12:18:46.559962 32681 debug-util.cc:398] Leaking SignalData structure 0x7b0800275dc0 after lost signal to thread 3575
I20260709 12:18:46.538837  3694 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:46.560333 32681 debug-util.cc:398] Leaking SignalData structure 0x7b080034e180 after lost signal to thread 3622
I20260709 12:18:46.546602  3423 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.532205  3507 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:46.549721  3689 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.575729 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.523253 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:46.473728 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:46.598595 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 124995us
I20260709 12:18:46.599980  3696 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:46.601751 32115 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:46.603113 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 124162us
I20260709 12:18:46.583158  3495 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:46.610615 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:46.611199 32113 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.612557  3458 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.614584 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:46.615213 32121 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:46.609277  3696 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:46.582288  3699 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:46.628688  3699 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.629199  3699 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.576412  3507 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:46.558859  3697 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.603374  3423 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:46.658198  3694 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:46.595460  3720 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.709576 32656 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:46.713691 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 106215us
W20260709 12:18:46.714401 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 274062us
I20260709 12:18:46.722640 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:46.721953 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.728399 32116 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:46.743733 32118 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:46.744520  3423 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:46.748395 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:46.749157 32047 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.730180  3690 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:46.723207  3588 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:18:46.788823 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:46.760298  3528 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:46.744935  3507 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.755559  3423 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.823447  3697 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:46.773931  3378 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.786126  3694 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:46.730451  3720 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.837941  3686 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.825963  3700 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.845063 32113 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
W20260709 12:18:46.845651 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 1.0726e+06us
I20260709 12:18:46.852137 32047 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:18:46.852170 32121 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
W20260709 12:18:46.870081 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.870368 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 147713us
I20260709 12:18:46.847856  3718 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.877485 32114 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:18:46.877967 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 147901us
I20260709 12:18:46.767359  3729 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:18:46.858595  3692 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.882761 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:46.864749  3411 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.885373 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:46.867961  3694 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:46.883705 32046 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:46.861227  3303 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:18:46.888447 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:46.889655 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:46.883193 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:46.896474 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:46.897181 31972 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:46.898811 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:46.899463 32038 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:46.898821 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:46.900635 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.846961  3378 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.902281  3694 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.904183 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:46.904739 31969 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.905826 31967 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:46.906291 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:46.805338  3696 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.862404  3686 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.911398 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:46.914649 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:46.915177 32044 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:46.915994 31971 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:46.921002  3720 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.921938 31968 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:46.910292 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:46.934109 31965 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.914703 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:46.932843 31971 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:46.892371  3458 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:46.955448 32046 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:46.956162 32654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 666) took 1696 ms (client timeout 1720 ms). Trace:
W20260709 12:18:46.960520 32654 rpcz_store.cc:269] 0709 12:18:45.259716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.259908 (+   192us) service_pool.cc:224] Handling call
0709 12:18:46.955885 (+1695977us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54004a2c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:46.956073 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616604544}
I20260709 12:18:46.956179 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:46.961862 31963 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:46.970920 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.20756e+06us
I20260709 12:18:46.880491  3718 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.889052  3697 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:18:46.978641 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:46.981771 32047 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:46.981930 31964 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.891355  3672 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:18:46.893764  3692 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.983516 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:46.985170 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:46.975881 31971 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:46.895234  3411 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:46.991104 31965 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:46.909464  3699 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:46.991258 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90382.9us
I20260709 12:18:46.994158 31963 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:46.998404  3694 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:46.884310  3690 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:47.002759 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 143426us
I20260709 12:18:46.981748  3528 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:46.985664  3588 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:46.987139  3622 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:47.003091 31971 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.987744  3468 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:46.879889 31858 maintenance_manager.cc:643] P a0887d9cdc8044caa86af7acdb454104: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 7.921s	user 0.191s	sys 0.041s Metrics: {"bytes_written":30023,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":100819,"dirs.run_cpu_time_us":443,"dirs.run_wall_time_us":56582,"drs_written":1,"lbm_read_time_us":395,"lbm_reads_lt_1ms":4,"lbm_write_time_us":24752,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":33,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":108940,"threads_started":1,"update_count":20}
I20260709 12:18:47.020496 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:47.013121  3718 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:46.984932  3495 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:18:47.014586  3528 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.023556  3686 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:47.010046  3622 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:47.024430  3700 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:46.988672  3737 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:47.026684  3737 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.029575  3737 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.033666 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:47.042945 31969 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:47.063467 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:47.025424  3622 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:47.080926 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 313825us
I20260709 12:18:47.087707  3578 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:47.089273 31859 maintenance_manager.cc:419] P a0887d9cdc8044caa86af7acdb454104: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9373 bytes on disk
I20260709 12:18:47.089880  3699 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.091131 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:47.091734 32654 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.092293  3411 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:47.095685 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 90521.4us
I20260709 12:18:47.096912 32040 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:47.099723 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.081540  3697 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:47.006996  3694 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.106189 31858 maintenance_manager.cc:643] P a0887d9cdc8044caa86af7acdb454104: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.012s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":344,"lbm_reads_lt_1ms":4}
I20260709 12:18:47.074265  3588 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:47.100652 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:47.101197 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.074043  3468 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:47.107376 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.105911 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:47.075757  3573 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:18:47.126690 31964 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:47.106364  3622 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.135673 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.137125 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:47.123819  3411 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:47.122905  3378 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:47.141640 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:47.102703  3748 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:47.100627  3495 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 12:18:47.147249 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 167256us
I20260709 12:18:47.148871 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:47.152107  3378 raft_consensus.cc:2764] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:18:47.152566  3741 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:18:47.153563  3495 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:47.154119  3495 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:47.153909  3588 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.155236  3495 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.158710 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:47.160651  3696 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:47.161155  3696 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:47.161621  3696 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.166859 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:47.170787  3411 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:47.171331  3411 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.171721  3411 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.126564  3626 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:18:47.159487  3690 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:47.156829  3495 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.170693  3468 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.158917 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:47.176514 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:47.141086  3697 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:47.177175 31964 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.182611  3746 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:47.183285  3746 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:47.184197  3746 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.181649  3697 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:47.186729  3697 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.178960 31971 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:47.182551 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:47.189314 31965 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.178290 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:47.190572 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:47.192277 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:47.193307 32117 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:47.150758  3699 raft_consensus.cc:2764] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:47.196466 32114 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:47.200229  3742 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:47.174521  3690 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.196703 32654 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:47.221843 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:47.223346 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:47.223501 31965 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:47.248172  3689 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.246558  3749 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:47.223800  3690 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:47.254406 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 95590.9us
I20260709 12:18:47.250025  3640 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:47.238504  3756 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:18:47.227244 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:47.275309 31969 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 4.
I20260709 12:18:47.276157 32660 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:47.198951  3713 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.922s	user 0.005s	sys 0.008s
I20260709 12:18:47.248895  3696 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:47.256274 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56726.6us
I20260709 12:18:47.177711  3748 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:18:47.213052  3700 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:47.256616  3747 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:47.228387  3742 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:18:47.273569 32657 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:47.281404 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2228) took 2119 ms. Trace:
I20260709 12:18:47.285033  3746 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.310602 32039 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:47.287425  3716 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.764s	user 0.012s	sys 0.000s
I20260709 12:18:47.292868  3694 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.304423 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.309221 32119 rpcz_store.cc:276] 0709 12:18:45.161921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.605652 (+443731us) service_pool.cc:224] Handling call
0709 12:18:45.771103 (+165451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.793048 (+ 21945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.885591 (+1092543us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54000807d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:47.281200 (+395609us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54000807d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:47.281343 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387092096}
I20260709 12:18:47.312726 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 449) took 1404 ms. Trace:
W20260709 12:18:47.313539 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.316941 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.315888 32041 rpcz_store.cc:276] 0709 12:18:45.904075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.991646 (+ 87571us) service_pool.cc:224] Handling call
0709 12:18:45.992048 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.071048 (+ 79000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:47.308131 (+1237083us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54001a5ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:47.308336 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112252160}
I20260709 12:18:47.303157  3700 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:47.322554 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.313342  3747 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:47.329367  3749 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:47.327081  3622 raft_consensus.cc:2764] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:47.332547  3696 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.332151  3747 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.334991 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.335234 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.276384  3640 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:47.390689 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54750.5us
W20260709 12:18:47.391683 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56524.5us
I20260709 12:18:47.325243  3718 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.322645  3700 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.314720  3746 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.399236  3718 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.398685  3575 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:47.417049 32656 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:47.397539  3622 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:47.428845  3303 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.425050 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.434788 32113 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:47.438714  3765 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:47.439630 32121 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:47.441437 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 486) took 1435 ms. Trace:
I20260709 12:18:47.441792 32112 rpcz_store.cc:276] 0709 12:18:46.005650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.012945 (+  7295us) service_pool.cc:224] Handling call
0709 12:18:46.013318 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.026641 (+ 13323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.026672 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.073805 (+ 47133us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.376886 (+303081us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:46.376927 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.377009 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:47.286734 (+909725us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:47.432501 (+145767us) raft_consensus.cc:1685] finished
0709 12:18:47.432526 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.441374 (+  8848us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2789,"raft.queue_time_us":14298,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":34311,"spinlock_wait_cycles":19803392,"thread_start_us":164027,"threads_started":1,"wal-append.queue_time_us":260936}
I20260709 12:18:47.443673 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:47.405191  3690 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.452246 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:47.454736 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48248us
I20260709 12:18:47.405958  3573 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:18:47.455868 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49115.9us
I20260709 12:18:47.426064  3666 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.428745  3525 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.429682  3622 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.457623 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.405701  3749 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:18:47.461653 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:47.468333 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:47.474229 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.469720  3666 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.459254  3762 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:47.480933  3762 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:47.481751  3666 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.457151  3765 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.481882  3762 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.461687  3748 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:18:47.471030  3525 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.488729  3525 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.437894  3697 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.457037  3694 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.451327  3664 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 12:18:47.501943 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81959.3us
I20260709 12:18:47.470448  3622 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.511600 32112 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:47.478116  3696 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:47.513768 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45516.9us
W20260709 12:18:47.515125 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53531.5us
I20260709 12:18:47.437526  3303 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.517980 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:47.515987 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.518621 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:47.519388 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.520879 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:47.523373 32121 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:47.500536  3764 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:47.519430 32113 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:18:47.503240  3748 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.510673  3664 consensus_queue.cc:1243] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:47.511214  3697 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.512782  3525 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:18:47.541198  3718 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:47.541314  3522 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:47.542021  3522 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:47.536800  3775 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:47.543164  3522 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.544042  3775 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:47.551615 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.553396  3690 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.538518  3672 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:47.517203  3495 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:18:47.578966 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59650.8us
I20260709 12:18:47.535058  3578 raft_consensus.cc:2764] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:47.582955 32654 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:47.490513  3761 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:18:47.583112 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.513170  3666 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:18:47.590400 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.530818  3575 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:18:47.530730  3748 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.577637  3378 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:18:47.598975  3754 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:47.599417  3754 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:47.599788  3754 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:47.582856  3672 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:47.601673  3767 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:18:47.592140  3749 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:47.588714  3640 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.613214  3495 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.621544  3495 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.596771  3773 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:47.596333  3746 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.614961  3749 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:47.594139  3690 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:18:47.602092  3672 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.595927  3696 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:47.629112  3697 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:47.610332  3578 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:18:47.627241  3777 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:18:47.644375 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:47.629882  3378 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
W20260709 12:18:47.646147  3778 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:18:47.646783  3501 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:47.650863  3640 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:47.652875  3696 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.659327  3507 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:47.659525  3748 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.660950  3495 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.637780  3697 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:47.661972 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:47.728317 32654 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:47.634981  3773 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:47.713793 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.645197  3761 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.636998  3775 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:47.736281 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.641386  3411 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.715339  3501 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:47.759090 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 165927us
I20260709 12:18:47.759438  3782 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:47.759946  3782 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.761003 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.760336  3782 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.765395  3794 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:47.730768  3697 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:47.752393  3703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.198s	user 0.021s	sys 0.017s
W20260709 12:18:47.756378 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 115964us
I20260709 12:18:47.719141  3764 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:47.661684  3749 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.783977  3622 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.724428  3640 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.775322  3666 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:47.787472  3593 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:47.789696 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.782328  3501 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:47.781416  3703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.227s	user 0.021s	sys 0.017s
I20260709 12:18:47.786065  3411 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.756690  3761 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.792909 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:47.856091  3782 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:47.846593 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53846.3us
I20260709 12:18:47.858906 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:18:47.859800 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.862957 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 81933.4us
I20260709 12:18:47.801971  3779 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.835449  3303 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.836412  3622 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:47.803663  3593 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:47.842721  3794 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:47.833889  3666 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.875061  3666 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:47.893231  3793 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:47.893810  3793 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:47.894743  3793 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.897851  3622 raft_consensus.cc:2764] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:47.898561  3622 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:47.902411  3593 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.904088  3593 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.911293  3640 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:47.943466 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71492us
I20260709 12:18:47.945364 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:47.948678 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 89881.8us
I20260709 12:18:47.837255  3764 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.891404  3578 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:18:47.870332  3779 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:47.958302  3792 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:18:47.959208  3784 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:18:47.865926  3773 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:47.957692  3802 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:47.954094  3782 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:47.952570  3633 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.460s	user 0.016s	sys 0.031s
I20260709 12:18:47.964448 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:47.971272  3802 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:47.978925  3633 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.487s	user 0.016s	sys 0.031s
I20260709 12:18:47.980481  3802 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.980551  3633 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:47.981603  3633 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.985895  3761 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.989530  3697 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.992190  3779 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:47.994555  3754 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:47.996654  3754 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.996869  3692 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:47.998289 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:47.998943 31968 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:47.999696 31966 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:47.999980 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:48.000800 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:48.003371 31969 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:48.005082 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:47.969544  3578 consensus_queue.cc:1243] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:48.005625 31970 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:48.006636 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:48.007732 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:48.009191 31966 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:18:48.011256  3726 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.463s	user 0.003s	sys 0.009s
I20260709 12:18:48.015120 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:48.016304 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:48.017571  3378 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.017894 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:48.035068 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:48.035391 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71027.1us
I20260709 12:18:48.036387 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:48.039134  3578 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:48.039557  3578 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:48.039970  3578 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.018178  3378 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:48.042987 32119 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:48.040758  3782 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.040493  3801 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:48.044581  3697 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:48.045315 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:48.045873 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:48.037808  3694 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.000s
I20260709 12:18:48.049649 31970 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:48.054543  3694 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:48.057034  3802 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:48.051734  3765 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
I20260709 12:18:48.058610  3765 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.047878  3801 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:48.059640 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.059970 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.061151 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.059669  3801 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.061777 32651 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:48.062124  3801 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.062701 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.063282  3507 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.063997  3762 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:48.063848  3507 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.064383  3762 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:48.065223 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:48.065186  3762 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.065838  3507 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:48.066438 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:48.066641 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.066572 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.067471 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:48.060876 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.063452 32652 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:48.069656 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.070195 32041 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.076081 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:48.084041 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47755.9us
I20260709 12:18:48.085095 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:48.098680 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54764us
I20260709 12:18:48.102720 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.104171 32038 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:48.106415 32041 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:48.108513 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.018603  3793 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:18:48.074153  3762 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.074823  3688 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.094669  3484 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:48.095939  3794 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:48.097144  3666 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.102224  3378 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:48.108039  3303 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:48.114715 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:48.133946 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48969.8us
I20260709 12:18:48.139362 32038 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:48.141325 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:48.147177 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.131309 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.148974 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.154510 32041 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.159324 32040 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:48.159174 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:48.112440  3722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.596s	user 0.032s	sys 0.034s
I20260709 12:18:48.170254 32041 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:48.170444 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.171128 32047 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:48.168057 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.171211 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.171870 32041 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.172468 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.173569 32047 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.6-15.6]
I20260709 12:18:48.176486 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.177778 32042 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.158470  3794 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.167379  3782 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.180583  3782 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.182541  3782 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:48.184072  3495 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:48.184573  3495 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:48.185529  3495 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.176492 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.188868 32038 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:48.192950 32041 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:48.195564 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2239) took 1753 ms. Trace:
I20260709 12:18:48.196098 32118 rpcz_store.cc:276] 0709 12:18:46.442193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.442354 (+   161us) service_pool.cc:224] Handling call
0709 12:18:46.442630 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.744389 (+301759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.195266 (+1450877us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540010efd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:48.195489 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691816192}
I20260709 12:18:48.207371  3378 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:48.212319 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.251111  3501 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.136042  3303 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.268334  3303 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.147428  3802 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:48.263006  3578 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.127383  3688 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.137315  3807 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:48.120426  3754 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 12:18:48.266358  3795 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:48.302860 32044 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:48.302958  3795 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.305320  3795 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:48.169904  3722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.664s	user 0.033s	sys 0.034s
I20260709 12:18:48.283035 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.307559 32042 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:48.284471 32038 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:48.285207 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.309011 32040 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.286132 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:48.287356 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.287217 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.293046  3436 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.314590 32046 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:48.297586  3791 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:48.302412 32043 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:48.324388 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.317461 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:48.294488 32045 pending_rounds.cc:85] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 9
I20260709 12:18:48.326802 32045 pending_rounds.cc:107] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:18:48.327147 32045 raft_consensus.cc:2889] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:48.279870  3633 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.330919 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.330716  3436 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.343544 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.344417 32042 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:48.345775  3436 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:48.346458  3633 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.324466  3791 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:48.350296  3791 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.351842  3791 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:48.364744 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.370780 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.375244  3777 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.375934  3777 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.382701  3528 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:48.383888  3578 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:48.385051  3528 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:48.385596  3528 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:48.390573  3694 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.391130  3694 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:48.392419 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.393177  3694 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:48.332176  3795 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:18:48.395933  3795 consensus_queue.cc:1243] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:48.396517  3795 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:48.396898  3795 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:48.403072 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 249532us
I20260709 12:18:48.404536 32040 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
W20260709 12:18:48.406381 32623 leader_election.cc:336] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.062s (SENT)
I20260709 12:18:48.406898 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:48.386081  3633 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:48.369513  3688 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:48.413199 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:48.413488 32040 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:48.431926 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:48.397882  3795 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:48.436450 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 144931us
I20260709 12:18:48.436652  3742 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.439821 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:48.438707  3688 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.444082  3688 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.446471  3764 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:48.447081  3764 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.439029  3765 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:48.438865  3690 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:18:48.437804  3737 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:48.439214  3766 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:18:48.454026 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:48.455534 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49206.4us
I20260709 12:18:48.434592  3777 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:48.431284  3783 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:18:48.453153  3764 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.455379  3795 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.453794  3725 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:48.471849 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:48.476612 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.477564 32040 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.412230  3528 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.438170  3773 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:18:48.473412 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:48.461712  3697 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.464468  3765 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:48.469453  3737 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:48.474623  3688 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:48.522377 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50653.7us
I20260709 12:18:48.533380 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:48.534936 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:48.474526  3725 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:48.493541  3800 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:18:48.497135 32010 leader_election.cc:336] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.809s (SENT)
I20260709 12:18:48.542752 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:48.554850 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:48.557732 32118 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:48.560150 32112 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:18:48.562815 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:48.540539  3747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.017s	sys 0.016s
I20260709 12:18:48.531960  3777 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:48.523690  3722 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:18:48.537688  3783 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:48.490352  3779 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:18:48.572721 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:48.573653  3779 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:48.574119  3779 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.574550  3779 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:48.575802  3785 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.858s	user 0.004s	sys 0.000s
I20260709 12:18:48.581739 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:48.582721 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:48.584369 32654 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:48.582741 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:48.493897  3694 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 12:18:48.589490 32655 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:48.592170 32120 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:48.556428  3775 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.569877 32112 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:48.569736 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:48.564236  3688 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.565425  3800 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:48.567185  3747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.017s	sys 0.017s
I20260709 12:18:48.547055  3766 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.564208  3699 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.598142 32121 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:48.568356 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:48.571456 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:48.544642 32040 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:48.548138 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:48.549139  3761 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:18:48.566913  3777 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:48.610864  3699 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:48.661887  3823 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:18:48.663002  3699 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:48.668908 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 171813us
W20260709 12:18:48.676532 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.704283 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.638243  3813 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:18:48.599795  3303 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.638378  3777 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:48.639200  3436 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:48.607815  3688 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.601984  3775 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:48.733656 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 118701us
I20260709 12:18:48.632625  3696 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:18:48.875731  3699 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.876318  3699 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.879272  3699 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:48.792034 32010 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.873s (SENT)
I20260709 12:18:48.795964  3764 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:48.888679 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 96676us
I20260709 12:18:48.889688  3764 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:48.798038 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:48.890043 32010 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.947s (SENT)
I20260709 12:18:48.890573 32045 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
W20260709 12:18:48.891086 32010 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.013s (SENT)
I20260709 12:18:48.891518 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:48.891552 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.892944 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:48.893534 32045 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.895530  3819 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.760452  3800 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.896085  3819 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.898475  3819 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:48.865458 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:48.907008 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.908298 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.908617 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:48.910111 32039 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:48.910503  3775 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:48.605546  3737 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.595432  3725 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.873526 32117 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:48.885052 32623 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.506s (SENT)
I20260709 12:18:48.887516  3782 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.915552  3782 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:48.918700  3782 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:48.914508 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:48.915005  3501 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.920075  3692 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:48.920331  3423 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.920504  3692 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:48.921304  3692 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:48.922603 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 280166us
W20260709 12:18:48.924983 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.925505 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.925949 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.927076 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.927524 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.929824  3764 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:48.930934 32113 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:48.932976 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:48.933743 31969 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.938079 31968 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:48.936707 32117 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:48.846916  3765 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:48.955245 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.960325 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:48.961843 31972 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:48.970301 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:48.971051 32655 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.936225  3692 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.939482  3782 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:48.980083 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.980005 31968 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:48.980410  3795 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.005s
I20260709 12:18:48.981480  3779 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.982196  3777 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.983906  3697 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:48.990077 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99005.6us
I20260709 12:18:49.003258  3782 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:49.008371  3688 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:49.009410  3782 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:48.992193  3436 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.000751 31969 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:48.945214  3303 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:49.011781  3779 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:49.014622 32121 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:49.015785 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:49.016893  3761 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:49.016961  3423 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:49.017195  3773 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:49.003072 32113 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:49.021198 32045 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:49.015977 32119 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:49.016989 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:49.022430 31969 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:49.022300 32655 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:49.023787 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:48.942521  3764 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:48.990931  3725 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.037509 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:49.039700 32045 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:49.021669  3773 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.010131  3436 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.038614  3699 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:49.031340  3779 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:18:49.036597  3303 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:49.053725 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.056361 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:49.056952 31969 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.041164  3723 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:18:49.039012  3742 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:18:49.051613  3773 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.076655  3501 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.077345  3501 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:49.080009  3501 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:49.057385  3827 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:18:49.058207  3699 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:49.084139 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:49.084810 32047 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:18:49.084460  3699 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:49.087488  3764 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:49.088616  3777 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:49.047664  3722 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 12:18:49.098852 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.100526 31969 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:49.061203  3303 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:49.118124 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:49.123750 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:49.124593 31969 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.126097  3737 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:49.145948 31867 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 150) took 13308 ms (client timeout 15000 ms). Trace:
I20260709 12:18:49.147514 31969 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:49.149055 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:49.168664 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:49.169955 31969 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:49.135725  3303 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:49.167057  3501 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.168493  3825 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:49.175973 32118 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:49.168478  3737 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:49.146497 31867 rpcz_store.cc:269] 0709 12:18:35.870596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.878700 (+  8104us) service_pool.cc:224] Handling call
0709 12:18:49.116865 (+13238165us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:42.812354 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:42.816419 (+  4065us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:42.816454 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:18:42.816471 (+    17us) tablet.cc:662] Decoding operations
0709 12:18:42.817761 (+  1290us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:42.817821 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:42.817849 (+    28us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:18:43.207701 (+389852us) tablet.cc:701] Row locks acquired
0709 12:18:43.240829 (+ 33128us) write_op.cc:260] PREPARE: finished
0709 12:18:43.240985 (+   156us) write_op.cc:270] Start()
0709 12:18:43.241135 (+   150us) write_op.cc:276] Timestamp: P: 1783599523240939 usec, L: 0
0709 12:18:43.241157 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:18:43.241793 (+   636us) log.cc:844] Serialized 6490 byte log entry
0709 12:18:43.661642 (+419849us) op_driver.cc:464] REPLICATION: finished
0709 12:18:43.814272 (+152630us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f22890c50ac 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228fc05ef0
0709 12:18:44.309746 (+495474us) write_op.cc:301] APPLY: starting
0709 12:18:44.309895 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:44.325944 (+ 16049us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:44.325993 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:44.356547 (+ 30554us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:44.356973 (+   426us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=32
0709 12:18:44.357029 (+    56us) write_op.cc:312] APPLY: finished
0709 12:18:44.357555 (+   526us) log.cc:844] Serialized 113 byte log entry
0709 12:18:44.379051 (+ 21496us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:44.379345 (+   294us) write_op.cc:454] Released schema lock
0709 12:18:44.379422 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:44.379974 (+   552us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:44.380005 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":779031,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":647716,"apply.run_cpu_time_us":5429,"apply.run_wall_time_us":70209,"num_ops":16,"prepare.queue_time_us":496523,"prepare.run_cpu_time_us":31929,"prepare.run_wall_time_us":558584,"raft.queue_time_us":255964,"raft.run_cpu_time_us":64821,"raft.run_wall_time_us":589282,"replication_time_us":420072,"spinlock_wait_cycles":55970432,"thread_start_us":991415,"threads_started":4,"wal-append.queue_time_us":155388,"wal-append.run_cpu_time_us":31796,"wal-append.run_wall_time_us":3996800}]]}
I20260709 12:18:49.203123 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:49.204164 31969 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.207756 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:49.216380  3835 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.237s	user 0.000s	sys 0.002s
I20260709 12:18:49.183213  3436 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:49.184574  3501 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.229084 31969 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:18:49.256302 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:49.256935 31969 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:49.255962  3723 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:18:49.259518  3501 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:49.260396 32038 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:49.253343  3458 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:49.263026  3458 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:49.274834  3825 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:49.276361  3458 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:49.277407  3779 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.279080  3779 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.281277 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:49.288265  3779 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:49.288542  3761 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:49.289321  3761 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.292235  3764 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.294991  3764 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.297230  3764 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:49.256070  3737 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:49.299683  3841 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:49.300606 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:49.301224 32045 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.307650 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:49.291522  3699 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.307489 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:49.308943 32120 pending_rounds.cc:85] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 5
I20260709 12:18:49.318141 32120 pending_rounds.cc:107] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:49.316710 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:49.318406 32120 raft_consensus.cc:2889] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:49.318742 32120 pending_rounds.cc:107] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:49.320401 32120 raft_consensus.cc:2889] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:49.321527 32045 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:49.306990  3765 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.277030  3825 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.328835 32045 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:49.344558 32045 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:49.346571 32045 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:18:49.360656 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53112.9us
W20260709 12:18:49.362092  3813 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:49.345104  3764 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:49.385984 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.337095  3699 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:49.414760 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72251us
W20260709 12:18:49.451427 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.410718  3692 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:49.344681  3765 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.385102  3764 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.458673  3761 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:49.521286 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 149745us
W20260709 12:18:49.622430  3848 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.297s	user 0.000s	sys 0.002s
W20260709 12:18:49.626780  3855 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:18:49.628381 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 465) took 1161 ms. Trace:
W20260709 12:18:49.550658  3847 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 12:18:49.462914  3692 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:49.628911 32039 rpcz_store.cc:276] 0709 12:18:48.467281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.239382 (+772101us) service_pool.cc:224] Handling call
0709 12:18:49.239704 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:49.240044 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.240078 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:49.240557 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:18:49.325434 (+ 84877us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:49.325743 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.325948 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.326019 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:49.628088 (+302069us) raft_consensus.cc:1685] finished
0709 12:18:49.628116 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.628309 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":194,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":42,"raft.queue_time_us":1260,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":2875,"spinlock_wait_cycles":1461504,"thread_start_us":84517,"threads_started":1,"wal-append.queue_time_us":84858}
W20260709 12:18:49.759020 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.760478 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.762064  3725 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.762677  3725 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:49.765084  3725 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:49.653154  3777 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.772022 31965 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:49.793510 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.769522  3303 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.779362  3777 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:50.053736  3857 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.499s	user 0.000s	sys 0.001s
I20260709 12:18:49.918217  3501 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.838056  3303 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.361294 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1988) took 1772 ms. Trace:
I20260709 12:18:50.375927 32045 rpcz_store.cc:276] 0709 12:18:48.588613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.361990 (+773377us) service_pool.cc:224] Handling call
0709 12:18:49.362385 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:49.367836 (+  5451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.367875 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:49.448990 (+ 81115us) log.cc:844] Serialized 26 byte log entry
0709 12:18:49.772282 (+323292us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:49.772347 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.772434 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.361072 (+588638us) raft_consensus.cc:1685] finished
0709 12:18:50.361098 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.361230 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5392,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9051,"thread_start_us":322915,"threads_started":1,"wal-append.queue_time_us":910038}
I20260709 12:18:50.378831 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1993) took 1427 ms. Trace:
I20260709 12:18:50.231065 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1961) took 1158 ms. Trace:
W20260709 12:18:50.341671 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 5.018s (SENT) [suppressed 48 similar messages]
I20260709 12:18:50.380743 31971 rpcz_store.cc:276] 0709 12:18:49.072221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.301080 (+228859us) service_pool.cc:224] Handling call
0709 12:18:49.301347 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.301534 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:49.301792 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.230273 (+928481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.230625 (+   352us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.230698 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.230811 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.230986 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":928569,"prepare.queue_time_us":2399,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":35,"raft.queue_time_us":934981,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":30181}
W20260709 12:18:50.254813 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 343723us
I20260709 12:18:50.379393 32038 rpcz_store.cc:276] 0709 12:18:48.951741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.207417 (+255676us) service_pool.cc:224] Handling call
0709 12:18:50.347875 (+1140458us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b54001d0954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:50.378757 (+ 30882us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118609152}
I20260709 12:18:50.394148 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:50.409993 32038 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:50.410480 32038 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:50.411450 32038 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.415700 32038 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:50.305150  3692 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:50.230006  3818 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.186038  3861 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:50.185704  3699 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:50.424258 32039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.043s
W20260709 12:18:50.416455  3773 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.343s	user 0.011s	sys 0.003s
I20260709 12:18:50.420596 31965 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 2 to 7, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:18:50.431125 32039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.007s	sys 0.043s
W20260709 12:18:50.424424  3742 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.010s	sys 0.015s
I20260709 12:18:50.438344 32038 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:50.439523 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1992) took 1502 ms. Trace:
I20260709 12:18:50.440363 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1949) took 1477 ms. Trace:
I20260709 12:18:50.412530  3725 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:18:50.205538  3501 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.432556  3819 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.441090 32038 rpcz_store.cc:276] 0709 12:18:48.936663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.409589 (+1472926us) service_pool.cc:224] Handling call
0709 12:18:50.409879 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.439178 (+ 29299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.439446 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:50.436318  3742 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.010s	sys 0.016s
I20260709 12:18:50.442408 31972 rpcz_store.cc:276] 0709 12:18:48.963463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.963647 (+   184us) service_pool.cc:224] Handling call
0709 12:18:49.005233 (+ 41586us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:49.005661 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.005692 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:49.021732 (+ 16040us) log.cc:844] Serialized 52 byte log entry
0709 12:18:49.145715 (+123983us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:49.317733 (+172018us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.439953 (+1122220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.440057 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.440126 (+    69us) raft_consensus.cc:1685] finished
0709 12:18:50.440144 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.440308 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":912525,"prepare.queue_time_us":94483,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":24,"raft.queue_time_us":919223,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":28533,"thread_start_us":458096,"threads_started":3,"wal-append.queue_time_us":171478,"wal-append.run_cpu_time_us":3068,"wal-append.run_wall_time_us":1201622}
I20260709 12:18:50.452538 32120 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 4. Preceding OpId from leader: term: 16 index: 6. (index mismatch)
I20260709 12:18:50.453121 32115 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:50.436275  3856 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.008s	sys 0.012s
I20260709 12:18:50.455533 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.456218 32115 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:18:50.436663  3755 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:50.452056 32114 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:50.439293  3777 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:50.429013  3818 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.461673 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.464089 32114 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:50.462898 32045 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:18:50.468721  3854 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:50.470793  3303 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:50.471102 32117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2121) took 1898 ms (client timeout 2038 ms). Trace:
I20260709 12:18:50.471832  3764 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:50.445250  3765 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:50.453750  3870 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:50.473676  3870 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:50.435760  3773 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.474630  3501 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:50.475984  3773 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:50.454739  3819 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.478610  3819 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:50.433605  3861 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.461911  3755 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.481237  3861 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:50.471504 32117 rpcz_store.cc:269] 0709 12:18:48.572349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.572492 (+   143us) service_pool.cc:224] Handling call
0709 12:18:50.470933 (+1898441us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b54004394d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:50.471042 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223622528}
I20260709 12:18:50.465075 32115 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:50.458720  3856 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.009s	sys 0.013s
W20260709 12:18:50.450915  3860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.022s
I20260709 12:18:50.492869 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.474076  3870 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:50.494468 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.500365 32660 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:50.501459 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.502112 32655 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:50.497423 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.502981 32656 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
W20260709 12:18:50.504386 31936 leader_election.cc:336] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.515s (SENT)
W20260709 12:18:50.499117  3860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.001s	sys 0.022s
I20260709 12:18:50.519465 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:50.522215 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.521633  3755 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:50.521677 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:50.526098 32655 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:50.527027 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.527658 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.528101 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.528988 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.523252 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1956) took 1509 ms. Trace:
I20260709 12:18:50.523945 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.532187 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:50.531587 31963 rpcz_store.cc:276] 0709 12:18:49.014131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.521970 (+1507839us) service_pool.cc:224] Handling call
0709 12:18:50.523182 (+  1212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:50.539716 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.540416 32117 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:50.510573 32656 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 2 to 7, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:50.531245  3819 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:50.547964 32655 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:50.555197 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1963) took 1435 ms. Trace:
I20260709 12:18:50.560835 31971 rpcz_store.cc:276] 0709 12:18:49.119366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.408996 (+1289630us) service_pool.cc:224] Handling call
0709 12:18:50.409290 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.409655 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.409684 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:50.415581 (+  5897us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.553035 (+137454us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:50.553090 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.553183 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.554936 (+  1753us) raft_consensus.cc:1685] finished
0709 12:18:50.554958 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.555123 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4112,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":1290,"thread_start_us":107987,"threads_started":1,"wal-append.queue_time_us":121565}
W20260709 12:18:50.548717 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 207061us
I20260709 12:18:50.568768 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.570192 32115 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:50.544179  3755 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:50.573546 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.541169  3501 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:50.577972  3892 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.578466 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.567777  3819 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.599972 32115 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:50.576385  3755 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.615777 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1947) took 1675 ms. Trace:
I20260709 12:18:50.614809 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:50.601042 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.567s (SENT)
W20260709 12:18:50.627954 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 218285us
I20260709 12:18:50.598810  3501 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.616532 31966 rpcz_store.cc:276] 0709 12:18:48.940430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.298461 (+358031us) service_pool.cc:224] Handling call
0709 12:18:49.298779 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.230134 (+931355us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:50.454230 (+224096us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.615228 (+160998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.615331 (+   103us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:50.615407 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.615509 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.615693 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":932262,"prepare.queue_time_us":319145,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":937824,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":22600,"spinlock_wait_cycles":962432,"thread_start_us":363128,"threads_started":2}
I20260709 12:18:50.628734 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.679338 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.710606 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:50.604223  3819 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.677896 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.614715  3755 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:50.666944  3879 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:18:50.730827 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 102151us
I20260709 12:18:50.734560 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:50.604591  3892 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.688038  3825 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.741123 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:50.740257 32623 leader_election.cc:336] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.552s (SENT)
I20260709 12:18:50.724496  3880 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.740648  3818 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:50.753222  3501 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:50.754395  3501 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:50.755946  3501 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:50.756561  3501 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:50.757030  3501 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:50.757645 32623 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.738s (SENT)
W20260709 12:18:50.758781  3877 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:18:50.760058 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1958) took 1745 ms. Trace:
I20260709 12:18:50.760409 31969 rpcz_store.cc:276] 0709 12:18:49.014725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.257469 (+242744us) service_pool.cc:224] Handling call
0709 12:18:49.257841 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.205534 (+947693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.205585 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:50.230220 (+ 24635us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.435061 (+204841us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.435108 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.435190 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.759852 (+324662us) raft_consensus.cc:1685] finished
0709 12:18:50.759870 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.760002 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":947234,"raft.queue_time_us":978653,"spinlock_wait_cycles":49158016,"thread_start_us":146826,"threads_started":1,"wal-append.queue_time_us":301516}
I20260709 12:18:50.762760  3888 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:50.763170  3888 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:50.763949  3888 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.765105  3888 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:50.775802 32623 leader_election.cc:336] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.913s (SENT)
W20260709 12:18:50.777107 32623 leader_election.cc:336] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.914s (SENT)
W20260709 12:18:50.778066 32623 leader_election.cc:336] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.925s (SENT)
I20260709 12:18:50.778529 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:50.780682 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46221.8us
I20260709 12:18:50.784680 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1952) took 1793 ms. Trace:
W20260709 12:18:50.784865  3872 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:18:50.785758  3892 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:50.786257 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1989) took 2194 ms. Trace:
I20260709 12:18:50.787070  3501 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:50.786734 32039 rpcz_store.cc:276] 0709 12:18:48.591756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.752812 (+1161056us) service_pool.cc:224] Handling call
0709 12:18:49.753209 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:49.753581 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.753614 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:49.759313 (+  5699us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.434406 (+675093us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.434450 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.434519 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.786063 (+351544us) raft_consensus.cc:1685] finished
0709 12:18:50.786082 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.786199 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":689,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11491,"thread_start_us":615318,"threads_started":1,"wal-append.queue_time_us":785640}
I20260709 12:18:50.783712 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:50.788654 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:50.788830 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.789424 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.613s	user 0.023s	sys 0.043s
W20260709 12:18:50.789757 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.614s	user 0.023s	sys 0.043s
I20260709 12:18:50.790671 32656 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:50.791038 32656 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:50.791751 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.929s (SENT)
W20260709 12:18:50.792698 32623 leader_election.cc:336] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.050s (SENT)
I20260709 12:18:50.794166 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.796072 32655 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:50.745134  3825 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.791836 31968 rpcz_store.cc:276] 0709 12:18:48.991332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.991479 (+   147us) service_pool.cc:224] Handling call
0709 12:18:48.993203 (+  1724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.993442 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:49.170168 (+176726us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.178917 (+  8749us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.135110 (+956193us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b18001e7ae0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:18:50.430536 (+295426us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.781711 (+351175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.781758 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.781797 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.781861 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.784621 (+  2760us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":997138,"prepare.queue_time_us":703661,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":4663,"raft.queue_time_us":1018469,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":3096,"spinlock_wait_cycles":11189120,"thread_start_us":797688,"threads_started":3}
I20260709 12:18:50.800352 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:50.801278 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.795586  3501 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:50.810048 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.798179 32656 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.813235 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1994) took 1836 ms. Trace:
I20260709 12:18:50.813637 32656 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:50.813671 32038 rpcz_store.cc:276] 0709 12:18:48.977077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.450436 (+1473359us) service_pool.cc:224] Handling call
0709 12:18:50.451075 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.684225 (+233150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.684272 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:50.684886 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.742878 (+ 57992us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:50.742927 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.743019 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.813038 (+ 70019us) raft_consensus.cc:1685] finished
0709 12:18:50.813056 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.813178 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20914,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":289845,"threads_started":2,"wal-append.queue_time_us":96551}
I20260709 12:18:50.815452 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:50.816023 32038 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:50.816756 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:50.817270 32038 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:50.747680  3880 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.819449 32039 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:50.795562  3860 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:50.794819  3765 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:50.795791  3694 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.822649  3898 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.822899  3694 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.823176  3898 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.824353  3749 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:50.824971  3694 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:50.825194  3898 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:50.820852  3692 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:18:50.826014 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.826593 31970 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
W20260709 12:18:50.827042 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1948) took 1871 ms (client timeout 2231 ms). Trace:
W20260709 12:18:50.827361 31970 rpcz_store.cc:269] 0709 12:18:48.956010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.825763 (+1869753us) service_pool.cc:224] Handling call
0709 12:18:50.826969 (+  1206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:50.828464 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 472) took 1786 ms. Trace:
I20260709 12:18:50.828810 32039 rpcz_store.cc:276] 0709 12:18:49.034469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.818974 (+1784505us) service_pool.cc:224] Handling call
0709 12:18:50.819270 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.820408 (+  1138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.820589 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:50.829936 32041 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:50.830197  3681 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:50.830636  3681 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:50.831512  3681 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.832567  3681 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:50.833148 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1991) took 1904 ms. Trace:
I20260709 12:18:50.836429 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:50.833510 32045 rpcz_store.cc:276] 0709 12:18:48.928396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.411086 (+1482690us) service_pool.cc:224] Handling call
0709 12:18:50.411426 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.465063 (+ 53637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.832939 (+367876us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b5400216e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:50.833091 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114303872}
I20260709 12:18:50.839037 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 473) took 1712 ms. Trace:
I20260709 12:18:50.839525 32041 rpcz_store.cc:276] 0709 12:18:49.126274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.829394 (+1703120us) service_pool.cc:224] Handling call
0709 12:18:50.829645 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.830264 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.838214 (+  7950us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:50.840843 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2255) took 1535 ms. Trace:
I20260709 12:18:50.840765 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.841749 31970 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:50.821781 32038 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:50.843364 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 1996) took 1832 ms. Trace:
I20260709 12:18:50.843997 32038 rpcz_store.cc:276] 0709 12:18:49.011369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.819309 (+1807940us) service_pool.cc:224] Handling call
0709 12:18:50.819571 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.843131 (+ 23560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.843313 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:50.844754 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.845582 31969 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:50.850981 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.841377 32119 rpcz_store.cc:276] 0709 12:18:49.305559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.305701 (+   142us) service_pool.cc:224] Handling call
0709 12:18:49.326926 (+ 21225us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:49.456512 (+129586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.456559 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:49.457331 (+   772us) log.cc:844] Serialized 52 byte log entry
0709 12:18:49.771094 (+313763us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:49.773813 (+  2719us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.341426 (+567613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.840500 (+499074us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540013a954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:50.840545 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.840587 (+    42us) raft_consensus.cc:1685] finished
0709 12:18:50.840661 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.840788 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3151,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":5359,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":24970,"spinlock_wait_cycles":296850176,"thread_start_us":258665,"threads_started":1,"wal-append.queue_time_us":990085}
I20260709 12:18:50.851747 31968 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:18:50.852468 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.818612 32657 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:50.833897  3894 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:18:50.858549 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.859215 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:50.860236 32656 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:50.862490 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:50.863098 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:18:50.863989 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 62217.6us
I20260709 12:18:50.864636 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:50.867558 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.868400 31968 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.869567 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.852934  3694 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.822672  3765 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.877507  3870 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.877828 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.858640  3692 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:50.880609 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.888773 31971 raft_consensus.cc:2368] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7.
I20260709 12:18:50.880285 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:18:50.881968 32623 leader_election.cc:336] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.662s (SENT)
I20260709 12:18:50.886416 31966 raft_consensus.cc:2368] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:18:50.890060 32657 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:50.890105 32623 leader_election.cc:336] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.951s (SENT)
I20260709 12:18:50.858556  3764 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:50.891510 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.870477  3773 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:50.887354  3725 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:50.908078 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:50.912742 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:50.894209  3692 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.917953 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.918919 31963 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:50.919862  3692 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:50.921185  3891 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:18:50.921566 32623 leader_election.cc:336] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.333s (SENT)
I20260709 12:18:50.922622 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1962) took 1850 ms. Trace:
W20260709 12:18:50.923054  3893 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:18:50.923003 31965 rpcz_store.cc:276] 0709 12:18:49.072496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.784199 (+711703us) service_pool.cc:224] Handling call
0709 12:18:50.409268 (+625069us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.414179 (+  4911us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:50.456175 (+ 41996us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.602565 (+146390us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:50.605689 (+  3124us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.605894 (+   205us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.605951 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.606055 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.922427 (+316372us) raft_consensus.cc:1685] finished
0709 12:18:50.922445 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.922570 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4459,"raft.queue_time_us":4743,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":3408,"thread_start_us":106826,"threads_started":1,"wal-append.queue_time_us":318655}
I20260709 12:18:50.925333 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 1964) took 1802 ms. Trace:
I20260709 12:18:50.925663 31972 rpcz_store.cc:276] 0709 12:18:49.122854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.459081 (+1336227us) service_pool.cc:224] Handling call
0709 12:18:50.496041 (+ 36960us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:18:50.505308 (+  9267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.505340 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:18:50.512623 (+  7283us) log.cc:844] Serialized 104 byte log entry
0709 12:18:50.711029 (+198406us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.739871 (+ 28842us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.740768 (+   897us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.745343 (+  4575us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.745755 (+   412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.746508 (+   753us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.782475 (+ 35967us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b18001e7cc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:50.925118 (+142643us) raft_consensus.cc:1685] finished
0709 12:18:50.925140 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.925276 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28004,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":5833,"raft.queue_time_us":17295,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":17655,"spinlock_wait_cycles":12079488,"thread_start_us":161470,"threads_started":1,"wal-append.queue_time_us":213660}
I20260709 12:18:50.928905 31968 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:50.894454  3773 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.873687  3737 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:18:50.930733 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:50.880633  3694 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.935191 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:50.886273  3900 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:18:50.964816 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:50.925706  3725 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:50.966435 31968 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.931558  3303 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:50.910513  3892 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:50.905351  3764 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:50.934257  3773 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:50.931207 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61424.8us
I20260709 12:18:50.934083  3898 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.930423  3894 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:50.981307 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.960819  3882 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:50.969313  3725 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:50.988727  3898 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:50.991375 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:50.991847  3303 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:50.988034  3894 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.994508  3825 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:50.971560  3892 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.997610 31972 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
W20260709 12:18:51.008476 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.009050  3894 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:51.009050  3892 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.014463  3880 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:51.014891 32623 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.789s (SENT)
I20260709 12:18:51.015602  3870 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.015476 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:50.989290  3882 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.020067  3882 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:50.929987  3899 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:18:51.022874 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.023865 31972 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:51.050074  3903 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:51.056478  3818 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.060436  3818 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:51.060911  3694 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:51.062129  3764 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.064755  3819 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:51.066025 31969 raft_consensus.cc:2368] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:18:51.068042  3765 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.069705 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.070458 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.068971 31968 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:51.071035 31965 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:51.071445 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.901s	user 0.005s	sys 0.050s
W20260709 12:18:51.071769 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.901s	user 0.005s	sys 0.050s
I20260709 12:18:51.074250 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.075981 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.075915 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.076737 31966 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:18:51.078377  3819 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.091861 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.092344 31965 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:51.093287 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.093289 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.094285 31963 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:51.094838 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.095785 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.096360 31971 raft_consensus.cc:2376] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:18:51.075067 31970 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:51.096647 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.097450 31968 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:51.098955 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.099881 31971 raft_consensus.cc:2376] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:18:51.101100 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.101897 31971 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:51.102720 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.103714 31971 raft_consensus.cc:2368] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12.
I20260709 12:18:51.104519 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.105209 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.105716 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.106810 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:51.107352 31966 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:51.108101 31970 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:51.108628 31966 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:51.109298 31970 raft_consensus.cc:1217] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[9.5-13.7]   Dedup: 13.7->[]
I20260709 12:18:51.109782 31970 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: [9.5-13.7]
I20260709 12:18:51.102864 31968 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:51.106217 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:51.109786 31966 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.378281 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.533046 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.533919 31966 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:51.537205  3892 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.006s
I20260709 12:18:51.537699  3892 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:51.776427  3892 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:51.120862  3898 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:51.097751  3818 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.110348 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:51.857046  3894 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.846s	user 0.004s	sys 0.007s
I20260709 12:18:51.543470 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:51.116575 31963 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:51.548622  3882 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.000s
I20260709 12:18:51.956048 31971 raft_consensus.cc:2368] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:18:51.752111 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:51.542723 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.957178 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.957660 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.956029  3894 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:51.959694  3894 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:51.952211 32040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1995) took 2947 ms (client timeout 2495 ms). Trace:
W20260709 12:18:51.963120 32040 rpcz_store.cc:269] 0709 12:18:49.004831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.951979 (+2947148us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:51.952113 (+   134us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:18:51.952466 31935 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:58058 dropped due to backpressure: service queue is full with 50 items
I20260709 12:18:51.806087 31966 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:18:51.953672 31938 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:58058 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 442)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
W20260709 12:18:51.954341 32044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1970) took 3890 ms (client timeout 2160 ms). Trace:
W20260709 12:18:51.970745 32044 rpcz_store.cc:269] 0709 12:18:48.064235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.093634 (+ 29399us) service_pool.cc:224] Handling call
0709 12:18:51.954079 (+3860445us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001a63d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:51.954221 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351329536}
I20260709 12:18:51.956245  3882 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:51.958009 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.976637  3882 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:51.976801 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.979110  3818 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:51.979696  3795 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:51.980427  3795 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:51.979713  3818 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:51.983384  3795 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.983958  3818 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:51.915714 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:51.984930 32085 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.005s	sys 0.013s
W20260709 12:18:51.986677 32085 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.005s	sys 0.013s
W20260709 12:18:51.987147 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 988858us
I20260709 12:18:51.985049 31968 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:51.965565 31935 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:58058 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
I20260709 12:18:51.986445 31937 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56072 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
W20260709 12:18:51.990247 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.993678 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.995913 32659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 774) took 4896 ms (client timeout 1515 ms). Trace:
W20260709 12:18:51.996361 32659 rpcz_store.cc:269] 0709 12:18:47.099118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.099609 (+   491us) service_pool.cc:224] Handling call
0709 12:18:51.995675 (+4896066us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001ebc54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:51.995814 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369945472}
W20260709 12:18:51.998667 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1942) took 3970 ms (client timeout 2837 ms). Trace:
I20260709 12:18:51.995136 31937 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56072 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
W20260709 12:18:52.006059 31967 rpcz_store.cc:269] 0709 12:18:48.028270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.028370 (+   100us) service_pool.cc:224] Handling call
0709 12:18:51.998469 (+3970099us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000b5c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:51.998602 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010725248}
I20260709 12:18:51.986491 31938 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:58058 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
I20260709 12:18:52.007550 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.008445 31967 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:18:52.015455 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 1.00051e+06us
W20260709 12:18:52.016129 32623 connection.cc:256] RPC call timeout handler was delayed by -1.0000647090000712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 12:18:52.017082 32623 leader_election.cc:336] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.855s (SENT)
W20260709 12:18:52.017860 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.008601 31937 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56072 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 451)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
W20260709 12:18:52.018289 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.907s (SENT)
W20260709 12:18:52.021539 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.907s (SENT)
I20260709 12:18:52.021894 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:52.024130 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.061s	user 0.013s	sys 0.005s
W20260709 12:18:52.024442 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.013s	sys 0.005s
W20260709 12:18:52.024945 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.08976e+06us
I20260709 12:18:52.025550 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:52.020809 31937 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56072 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 451)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
I20260709 12:18:52.031718  3882 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:52.032176  3882 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.032506  3882 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.029407 31937 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56072 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 451)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
I20260709 12:18:52.045006 32114 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:52.052313 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 506) took 3343 ms. Trace:
I20260709 12:18:52.052737 32118 rpcz_store.cc:276] 0709 12:18:48.708764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.862179 (+153415us) service_pool.cc:224] Handling call
0709 12:18:48.862628 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.182641 (+320013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.052073 (+2869432us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400439ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:52.052226 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720162048}
I20260709 12:18:52.054528 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.055960 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.056512 32114 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:52.064842 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.065685 32113 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:52.065410  3882 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.067195 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.072108 32114 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:52.073515  3892 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:52.080909  3892 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:52.081272  3892 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:52.081521  3742 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:52.082095  3892 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.083271  3892 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.083468 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.084069 32118 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:18:52.086644 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.087088 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.076309 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.088971 31971 raft_consensus.cc:2376] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12. Re-sending same reply.
I20260709 12:18:52.090042 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.090773 31971 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:18:52.091547 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.092614 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.093591 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:52.094120  3755 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.355s	user 0.015s	sys 0.010s
I20260709 12:18:52.094619  3755 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.096782  3755 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:52.097761 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 76239.8us
W20260709 12:18:52.098393 32623 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.959s (SENT)
I20260709 12:18:52.098701 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.099831 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.100412  3911 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.100656 32112 raft_consensus.cc:2410] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:52.100937  3911 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.101444 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.101996 32112 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.102098 32118 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:52.104779  3911 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:52.107455 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:52.108300  3908 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:52.108843  3501 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.016s	sys 0.010s
I20260709 12:18:52.108858  3908 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:52.110473  3501 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.059s	user 0.016s	sys 0.012s
I20260709 12:18:52.110817  3908 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.113242 32112 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
W20260709 12:18:52.078086 31993 debug-util.cc:398] Leaking SignalData structure 0x7b0800133e00 after lost signal to thread 30823
I20260709 12:18:52.077102  3882 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.122630 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:52.125480 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:52.128681  3742 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:52.136209  3882 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:52.136224  3892 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:52.142505  3910 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:18:52.138665 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.145136 31971 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6.
I20260709 12:18:52.147490  3913 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.148847  3882 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:52.152199 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:52.153199 31964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1951) took 3166 ms (client timeout 2194 ms). Trace:
I20260709 12:18:52.155495  3755 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.165526  3755 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.011526 31938 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:58058 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 424)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 434)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 451)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208)
W20260709 12:18:52.160835 31964 rpcz_store.cc:269] 0709 12:18:48.986531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.153002 (+3166471us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:52.153113 (+   111us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:18:52.173056 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.174079 31964 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:52.176049 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.161154 31971 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:52.177577 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.188860 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.240722 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.241678 31971 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
W20260709 12:18:52.267952 32623 leader_election.cc:336] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.998s (SENT)
W20260709 12:18:52.269268 32623 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.029s (SENT)
W20260709 12:18:52.269910 32623 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.029s (SENT)
I20260709 12:18:52.270761 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.278591 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.279349 31964 raft_consensus.cc:2376] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12. Re-sending same reply.
I20260709 12:18:52.280202 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.280782 31964 raft_consensus.cc:2376] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:18:52.281307 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.281664 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.282135 31971 raft_consensus.cc:2410] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:52.282465 31964 raft_consensus.cc:2368] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9.
I20260709 12:18:52.283293 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.284356 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.284974 31964 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:52.267305  3892 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:52.273450  3742 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:52.270654  3755 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:52.290481  3882 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.283058 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:52.293567 32118 raft_consensus.cc:606] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.168119  3913 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.297897 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.304371 31964 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:52.305182 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.306056 31964 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:52.306917 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.307482 31964 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:52.308581 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.309293 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.309855 31971 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.312005 31964 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:18:52.315270  3892 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:52.318740 32010 connection.cc:256] RPC call timeout handler was delayed by -2.4465390420000404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:18:52.319303 32010 connection.cc:256] RPC call timeout handler was delayed by -2.4330390420000185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
W20260709 12:18:52.319713 32010 connection.cc:256] RPC call timeout handler was delayed by -1.9074390420000782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260709 12:18:52.324832 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:52.325718 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53484.7us
W20260709 12:18:52.326388 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.219s (SENT)
I20260709 12:18:52.326282 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.327911 32118 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.326982  3921 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.325994  3898 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:52.328783 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.249s (SENT)
I20260709 12:18:52.314127  3913 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:52.336512  3920 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:18:52.330660 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.249s (SENT)
I20260709 12:18:52.346740 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.347819  3723 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:52.328284  3818 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:52.331128  3378 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.332231  3795 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:52.332746 31938 connection.cc:256] RPC call timeout handler was delayed by -2.68260720499984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 12:18:52.372082 31938 connection.cc:256] RPC call timeout handler was delayed by -2.6358072049999919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
I20260709 12:18:52.371968  3818 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.333494 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.333374  3898 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:52.334462 32010 connection.cc:256] RPC call timeout handler was delayed by -1.722861506000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
W20260709 12:18:52.374130 32010 connection.cc:256] RPC call timeout handler was delayed by -1.5788615060000666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 12:18:52.374507 32010 connection.cc:256] RPC call timeout handler was delayed by -1.5708390419999887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2519s for a response
W20260709 12:18:52.374800 32010 connection.cc:256] RPC call timeout handler was delayed by -1.5194615059999705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
I20260709 12:18:52.374917 31964 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
W20260709 12:18:52.375123 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4690615060001164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 12:18:52.375404 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4609615060001033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:18:52.375705 32010 connection.cc:256] RPC call timeout handler was delayed by -1.4375615060000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
W20260709 12:18:52.375983 32010 connection.cc:256] RPC call timeout handler was delayed by -1.3782706720000988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24900000000000003s for a response
W20260709 12:18:52.376240 32010 connection.cc:256] RPC call timeout handler was delayed by -1.3727615059999607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 12:18:52.376470 32010 connection.cc:256] RPC call timeout handler was delayed by -1.3647390420001102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27480000000000004s for a response
W20260709 12:18:52.376703 32010 connection.cc:256] RPC call timeout handler was delayed by -1.357461505999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 12:18:52.376956 32010 connection.cc:256] RPC call timeout handler was delayed by -1.2674615060000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 12:18:52.377199 32010 connection.cc:256] RPC call timeout handler was delayed by -1.2512615059999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 12:18:52.377439 32010 connection.cc:256] RPC call timeout handler was delayed by -1.2305615059999582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
I20260709 12:18:52.334103  3921 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.334954  3908 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:52.349308 32623 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.302s (SENT)
I20260709 12:18:52.351990 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:52.381081 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:52.372596 31938 connection.cc:256] RPC call timeout handler was delayed by -2.553007204999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 12:18:52.332937  3755 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:52.378672 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.384040  3755 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.386436 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:52.382320 31938 connection.cc:256] RPC call timeout handler was delayed by -2.4144072049998613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 12:18:52.387761 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:52.388080 31938 connection.cc:256] RPC call timeout handler was delayed by -2.3181072049999329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260709 12:18:52.380331 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.388376 32038 raft_consensus.cc:2376] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12. Re-sending same reply.
I20260709 12:18:52.380357 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:52.388931 32623 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.302s (SENT)
W20260709 12:18:52.388402 31938 connection.cc:256] RPC call timeout handler was delayed by -1.5423072050000428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29s for a response
I20260709 12:18:52.355778  3528 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:52.389258 32044 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
W20260709 12:18:52.391120 32623 leader_election.cc:336] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 3.083s (SENT)
W20260709 12:18:52.389845 31938 connection.cc:256] RPC call timeout handler was delayed by -1.1544072049998704s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3331s for a response
I20260709 12:18:52.390643 32118 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:52.392347 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:52.393193 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.393334 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.393977 31971 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:18:52.393934 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:52.395686 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.395804 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.396641 32118 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.397511 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.397852 32043 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:52.398101 32039 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:52.398157 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.398882 32040 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:52.397755 32044 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:52.373617  3378 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.394531  3723 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.403302 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:52.403894 31993 debug-util.cc:398] Leaking SignalData structure 0x7b08002c8280 after lost signal to thread 31904
I20260709 12:18:52.403985 31964 raft_consensus.cc:2368] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9.
I20260709 12:18:52.404546 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.405212 31971 raft_consensus.cc:2368] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7.
I20260709 12:18:52.407661  3921 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.387482  3908 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:52.408927 32010 leader_election.cc:336] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.546s (SENT)
I20260709 12:18:52.409035  3700 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:52.409739 32010 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.561s (SENT)
I20260709 12:18:52.374258  3795 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:52.410533 32010 leader_election.cc:336] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.697s (SENT)
W20260709 12:18:52.403654 32623 leader_election.cc:336] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:58058 dropped due to backpressure: service queue is full with 50 items
I20260709 12:18:52.410941 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.414853 32118 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:52.411639 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.416695 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:52.418375 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.419404 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.420136 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.421865 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.393117  3742 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:52.422015 32010 leader_election.cc:336] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.763s (SENT)
I20260709 12:18:52.423756 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.427527 32043 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:52.410171  3908 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:52.434898 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.435830 31971 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:52.435191 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:52.402794  3755 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.403113  3818 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:52.438045 32010 leader_election.cc:336] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.819s (SENT)
I20260709 12:18:52.438160 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.402287  3378 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:52.445055 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.438474  3723 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:52.441855 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:52.443765 32010 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.828s (SENT)
I20260709 12:18:52.441450  3892 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:52.443501 32044 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
W20260709 12:18:52.463680 32010 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.854s (SENT)
I20260709 12:18:52.439491 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.445657 32112 raft_consensus.cc:2376] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12. Re-sending same reply.
W20260709 12:18:52.445761 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.445207  3926 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:52.486819 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.487442 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.463610  3818 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:52.489012  3818 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.491669  3818 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:52.492187  3818 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.446480 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.446179 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.464884 31964 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:18:52.470552 32010 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.926s (SENT)
I20260709 12:18:52.447337  3755 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.444715  3910 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:18:52.482491  3669 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.499405  3910 consensus_queue.cc:1243] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:52.499584  3669 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.500360 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.500586 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.499423  3755 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.501574 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.503039 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.503453  3755 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.504714 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.501595 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.505873 32114 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:52.505877 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.507964 32120 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:52.509292 32120 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.509632 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 533) took 1456 ms. Trace:
I20260709 12:18:52.503734 31972 raft_consensus.cc:2376] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12. Re-sending same reply.
I20260709 12:18:52.442435  3868 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:52.511432 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.441843  3633 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 12:18:52.486059  3773 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.497s	user 0.016s	sys 0.020s
I20260709 12:18:52.487370  3795 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.494828 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:52.479156 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.513548 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49045.4us
W20260709 12:18:52.514173 32010 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.943s (SENT)
I20260709 12:18:52.498499 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.514587 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.515111 31964 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:18:52.515565 32010 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.043s (SENT)
I20260709 12:18:52.515951 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.516575 31964 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:18:52.517339 32010 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.084s (SENT)
I20260709 12:18:52.517474 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.517755 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.511494  3868 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.518769  3868 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:52.518993  3926 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.519763  3926 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.518709  3892 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:52.512287  3773 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.438366  3742 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.510138 32117 rpcz_store.cc:276] 0709 12:18:51.052958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.053130 (+   172us) service_pool.cc:224] Handling call
0709 12:18:52.508973 (+1455843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.509212 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.509250 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:52.509306 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:52.509372 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.509444 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.509568 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.523551 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.524118  3742 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.524487 31968 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:18:52.524403 32010 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.220s (SENT)
W20260709 12:18:52.524768 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 132498us
I20260709 12:18:52.518070 31964 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:52.525388 32010 leader_election.cc:336] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.221s (SENT)
I20260709 12:18:52.525867 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.526109 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:52.526525  3773 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:52.526696 32654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 693) took 1977 ms (client timeout 1935 ms). Trace:
I20260709 12:18:52.527269  3892 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:52.527406  3795 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.528800  3795 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:52.531265  3669 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:52.531996  3801 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:52.531280 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.532858  3801 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:52.533224  3801 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:52.533910  3868 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.536351 31972 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:52.537289 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.532869 32114 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:52.537843  3528 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:18:52.541268  3378 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:52.541945 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.527129 32654 rpcz_store.cc:269] 0709 12:18:50.549607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.526511 (+1976904us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:52.526615 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:18:52.542565 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.542743 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:52.525440 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.543032 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.544010 31971 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:18:52.544836 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.545501 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.546065 31967 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:18:52.546886 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.547883 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.492632  3818 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.548650 31967 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:18:52.549925 31968 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 7.
I20260709 12:18:52.542692  3898 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.552050 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:52.552516 32121 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:52.553517 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.553825 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:52.523640 32117 raft_consensus.cc:1217] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Deduplicated request from leader. Original: 11.4->[13.5-15.6]   Dedup: 15.6->[]
I20260709 12:18:52.554078 32121 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:52.554492  3378 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:52.526630 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.555694 31964 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:52.540599 32653 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:18:52.556210 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.550024 32042 raft_consensus.cc:1217] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[9.8-15.11]   Dedup: 9.8->[11.9-15.11]
I20260709 12:18:52.563553 32042 pending_rounds.cc:85] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 8
I20260709 12:18:52.563832 32042 pending_rounds.cc:107] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:18:52.564059 32042 raft_consensus.cc:2889] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.564358 32042 pending_rounds.cc:107] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:18:52.564582 32042 raft_consensus.cc:2889] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.556515  3862 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.568467 31967 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:18:52.568667 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:52.556502 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.558851 32043 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:18:52.567507 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:52.570117 32043 pending_rounds.cc:85] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 6
I20260709 12:18:52.570441 32043 pending_rounds.cc:107] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:52.570700 32043 raft_consensus.cc:2889] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.555418 32121 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:52.559258  3725 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.549s	user 0.020s	sys 0.000s
I20260709 12:18:52.541631 31972 raft_consensus.cc:2376] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13. Re-sending same reply.
I20260709 12:18:52.559736 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.559975 32114 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.573246 32659 raft_consensus.cc:2410] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:18:52.554605 32117 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [13.5-15.6]
I20260709 12:18:52.561977 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:52.542909 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.572041  3926 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:18:52.569212  3755 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:52.569213  3862 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.575932 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.581595 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:52.576663 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.562455  3868 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:52.534017  3801 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.541719  3528 consensus_queue.cc:1243] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:18:52.578841 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.590809 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.591821 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:52.592154  3801 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.592360 32043 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:52.580420 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 408) took 1725 ms (client timeout 1925 ms). Trace:
W20260709 12:18:52.593641 31969 rpcz_store.cc:269] 0709 12:18:50.855022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.855147 (+   125us) service_pool.cc:224] Handling call
0709 12:18:52.580197 (+1725050us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400069354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:52.580342 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450135296}
I20260709 12:18:52.594738 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.595263 31969 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:52.596041 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.596817 31969 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:52.597534 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.597548 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:52.580999 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.598945 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.549976  3818 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.580873  3892 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:52.579308  3926 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:52.599831  3892 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.600569  3892 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.587204  3919 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.587492 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 734) took 2070 ms. Trace:
I20260709 12:18:52.579845  3755 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:52.599812 31969 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.603644 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.604271 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.588589  3633 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:52.580662 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.601176 32660 rpcz_store.cc:276] 0709 12:18:50.517402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.517648 (+   246us) service_pool.cc:224] Handling call
0709 12:18:50.518051 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.518344 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:18:50.521883 (+  3539us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.587000 (+2065117us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b1800381ac0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a
0709 12:18:52.587176 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:52.587224 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:52.587265 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.587336 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.587442 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3186,"prepare.queue_time_us":3998,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":1210,"raft.queue_time_us":77,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"spinlock_wait_cycles":40434048,"thread_start_us":56957,"threads_started":1,"wal-append.queue_time_us":56950,"wal-append.run_cpu_time_us":1644,"wal-append.run_wall_time_us":1067395}
I20260709 12:18:52.606729 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.607460 31964 raft_consensus.cc:2368] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12.
I20260709 12:18:52.570779 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:52.599139 32040 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:52.599973  3926 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:52.574963  3938 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:52.587742 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.588205 32039 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:52.609009 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.620442 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.612257 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:52.616297 31969 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:52.622467 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.623263 31969 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.624060 32043 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:52.625448 32040 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:52.625912 32038 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:52.627606 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.628134 32040 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: [11.13-11.13]
I20260709 12:18:52.628217 32038 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:52.626585 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.629748 31967 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:52.629485 32040 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.633584 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.634881 31964 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:18:52.628142 32043 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:52.632149 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:52.626760 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:52.637348 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.636067 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.605273  3919 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.639418 32041 raft_consensus.cc:2410] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:52.639633 32114 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:52.640498 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:52.642019  3898 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.642618 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.643325 31971 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:52.644712 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.644665  3755 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.645844 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.646718 31971 raft_consensus.cc:2410] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:52.643394 32114 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:52.647480 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.648036 31971 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:52.648864 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.649355  3378 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.651233  3378 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.651693  3378 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.649439 31971 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.655563 31964 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:18:52.629071  3908 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:52.657989 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:52.658214  3908 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:52.658579  3908 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:52.659176 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.659376  3908 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.660259 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.660712  3908 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.661293 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.661587 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:52.661988 32044 raft_consensus.cc:2368] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9.
I20260709 12:18:52.635504  3935 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:52.662412 31993 debug-util.cc:398] Leaking SignalData structure 0x7b0800395720 after lost signal to thread 31996
W20260709 12:18:52.662743 31993 debug-util.cc:398] Leaking SignalData structure 0x7b0800355000 after lost signal to thread 31997
I20260709 12:18:52.662948  3722 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.663573  3894 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.663437  3722 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.617596  3818 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:52.665143 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:52.580374  3725 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.669502 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.673825 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.674489 32040 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:52.675626 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.676122 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:52.676944 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.677565 32044 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:52.678439 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.678855 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.678993 32044 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:52.679418 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:52.670188 31967 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 8.
I20260709 12:18:52.684278 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.665676  3722 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.681907 32039 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:52.683008 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.683074 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.686460 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.654844  3528 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:52.671676 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.684841 31967 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:52.629400  3913 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:18:52.697062 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:52.698172 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.685159 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.636221  3938 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:52.685338 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.694674  3910 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:52.707540  3910 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.707971  3910 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.696784 32040 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:52.710896 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.711704 32040 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:52.678202 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:52.684482  3818 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:52.699019 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:52.700222 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.701922 31969 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:52.713853 32117 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 8: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:52.676887  3924 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:52.714493 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.695628 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:52.703017 31935 connection.cc:256] RPC call timeout handler was delayed by -3.0442897850000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 12:18:52.705428 31967 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:52.715189 31971 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
W20260709 12:18:52.735275 31935 connection.cc:256] RPC call timeout handler was delayed by -2.8912897850000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 12:18:52.735682 31935 connection.cc:256] RPC call timeout handler was delayed by -2.8084897849998924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 12:18:52.736013 31935 connection.cc:256] RPC call timeout handler was delayed by -2.5735897849999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260709 12:18:52.736086 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:52.736543 31935 connection.cc:256] RPC call timeout handler was delayed by -1.7977897850000772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29s for a response
I20260709 12:18:52.737473 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.705827 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:52.738138 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.697914  3749 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:52.706794  3935 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.711979 32044 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:52.695582  3725 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.702811  3801 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.702385 31964 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:52.740530 31967 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:52.744952 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.745790 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.746495 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:52.747068 31966 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 9.
I20260709 12:18:52.748590 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 429) took 1207 ms. Trace:
I20260709 12:18:52.748075 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.749207  3944 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.742898 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.750707 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:52.743214  3801 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.753165  3801 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.753866 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.754786 31967 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:52.685724  3926 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.755543 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.756156  3926 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.723809  3793 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:52.756239 31967 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:18:52.703118  3528 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:52.772745 32653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 728) took 4193 ms (client timeout 2097 ms). Trace:
I20260709 12:18:52.711006  3866 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:52.759021 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:52.774742 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:52.740717  3749 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.747239 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.778692 31963 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:52.749073 31966 rpcz_store.cc:276] 0709 12:18:51.541377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.745567 (+1204190us) service_pool.cc:224] Handling call
0709 12:18:52.748534 (+  2967us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.779027 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:52.779484 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.775285 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:52.773298 32653 rpcz_store.cc:269] 0709 12:18:48.579313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.582438 (+  3125us) service_pool.cc:224] Handling call
0709 12:18:52.772474 (+4190036us) spinlock_profiling.cc:243] Waited 3.93 s on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:52.772636 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9036111104}
I20260709 12:18:52.780567 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.781569 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:52.762718 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:52.782547 32652 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.782784 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.764362 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.786896 32660 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:18:52.788071 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2195) took 1244 ms. Trace:
I20260709 12:18:52.788626 31963 rpcz_store.cc:276] 0709 12:18:51.543885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.747018 (+1203133us) service_pool.cc:224] Handling call
0709 12:18:52.787979 (+ 40961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.743618  3742 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:52.741556  3378 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:52.753334  3892 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.744087 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.766659 31971 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:52.776830 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.777047 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.781641 32658 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:52.742506  3938 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.764441  3633 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:52.740797  3935 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.758838  3793 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:52.742174  3818 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.793367 31966 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:52.826781  3749 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:52.966938 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2190) took 1472 ms. Trace:
I20260709 12:18:52.967075 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.967352 31966 rpcz_store.cc:276] 0709 12:18:51.494501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.792756 (+1298255us) service_pool.cc:224] Handling call
0709 12:18:52.793138 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.966660 (+173522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.966882 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.968344 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.968711 31966 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:52.827611  3800 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.827503 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:52.970918 31971 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:52.970173  3800 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.971133 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.971510 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 430) took 1429 ms. Trace:
I20260709 12:18:52.971837 31971 rpcz_store.cc:276] 0709 12:18:51.541659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.970410 (+1428751us) service_pool.cc:224] Handling call
0709 12:18:52.970651 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.971290 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.971461 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.972189 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:52.973345 31971 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:52.973915 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2191) took 1431 ms. Trace:
I20260709 12:18:52.857828  3722 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:52.815097  3378 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.974391 31971 rpcz_store.cc:276] 0709 12:18:51.542757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.972902 (+1430145us) service_pool.cc:224] Handling call
0709 12:18:52.973152 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.973671 (+   519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.973843 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.712626  3501 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:52.975781 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.956243 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:18:52.976806 31967 raft_consensus.cc:606] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:52.977245 31967 raft_consensus.cc:614] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:52.823272 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:52.978559 31967 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:52.979176 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2193) took 1435 ms. Trace:
I20260709 12:18:52.856074  3528 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:52.793174  3742 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:52.980690  3742 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:52.894760 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.981727  3742 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:52.982082 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.969352 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:52.801740 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.983458 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.983781 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.966463  3948 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.781183  3866 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:53.032548 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:53.042969  3866 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.043577 32657 raft_consensus.cc:2376] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7. Re-sending same reply.
I20260709 12:18:53.044265  3935 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.034508 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.044910 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:53.045914 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.029294  3910 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.046751 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:52.990854  3633 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.047770 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.048333 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.048462 32045 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:18:53.049422 32044 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:52.781613  3944 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:53.048434  3378 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:53.050719 32659 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:53.051282  3800 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:52.814942 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.052445 31963 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:53.052619 32038 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:53.053957 31963 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:53.054662 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2199) took 1509 ms. Trace:
I20260709 12:18:53.055575  3892 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.031211 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:53.056146 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:53.056532 32047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 1998) took 3937 ms (client timeout 1928 ms). Trace:
I20260709 12:18:53.029242  3793 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.057027 32038 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:18:53.057672 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.059119 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:53.059798 32655 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.983886 31964 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:53.061084 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2196) took 1516 ms. Trace:
I20260709 12:18:53.061344 32117 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:53.061170 32011 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.061614 31964 rpcz_store.cc:276] 0709 12:18:51.544124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.983381 (+1439257us) service_pool.cc:224] Handling call
0709 12:18:52.983687 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.060822 (+ 77135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.061007 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.046108 32657 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:53.062628 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:53.062903 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.063184 31964 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.063621 32121 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.065055  3898 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:53.067226 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:53.068310 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2279) took 1016 ms. Trace:
I20260709 12:18:53.068747 32116 rpcz_store.cc:276] 0709 12:18:52.051917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.066872 (+1014955us) service_pool.cc:224] Handling call
0709 12:18:53.068222 (+  1350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.069625  3303 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:53.055027 31963 rpcz_store.cc:276] 0709 12:18:51.544969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.053169 (+1508200us) service_pool.cc:224] Handling call
0709 12:18:53.053681 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.054386 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.054598 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.074891 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.075207 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.076412 31963 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:18:53.077360 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.151s	user 0.059s	sys 0.059s
W20260709 12:18:53.077853 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.166s	user 0.059s	sys 0.059s
I20260709 12:18:52.793300  3801 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.286s
W20260709 12:18:53.057096 32047 rpcz_store.cc:269] 0709 12:18:49.119116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.056313 (+3937197us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:53.056455 (+   142us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:18:52.826027 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:53.079742  3793 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:53.080446 32653 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:52.815214 31969 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:53.031824  3862 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.080276  3793 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:53.083374  3793 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 11.13, Last appended by leader: 13, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.083966  3495 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.084401  3910 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.084656  3793 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.091344 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.092252  3908 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.979543 31967 rpcz_store.cc:276] 0709 12:18:51.543312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.978080 (+1434768us) service_pool.cc:224] Handling call
0709 12:18:52.978350 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.978968 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.979130 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.094952 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:52.969251 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 427) took 1428 ms. Trace:
I20260709 12:18:53.096454 31966 rpcz_store.cc:276] 0709 12:18:51.540801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.968184 (+1427383us) service_pool.cc:224] Handling call
0709 12:18:52.968433 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.969048 (+   615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.969206 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:52.840445 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:53.033731 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 531) took 2168 ms. Trace:
I20260709 12:18:53.099167 31966 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:53.100008 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2200) took 1554 ms. Trace:
I20260709 12:18:53.098948 32119 rpcz_store.cc:276] 0709 12:18:50.865168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.865285 (+   117us) service_pool.cc:224] Handling call
0709 12:18:50.865549 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.996755 (+131206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.996785 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.996827 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:50.996859 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.033460 (+2036601us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002f9c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:53.033511 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.033667 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301325184}
I20260709 12:18:53.100577 31966 rpcz_store.cc:276] 0709 12:18:51.545265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.098136 (+1552871us) service_pool.cc:224] Handling call
0709 12:18:53.098775 (+   639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.099720 (+   945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.099948 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:53.101065 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.974674  3722 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:53.106178 32656 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:18:53.111274 31971 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 7. Current term is 9. Ops: []
I20260709 12:18:53.111925 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2192) took 1568 ms. Trace:
I20260709 12:18:53.112224 31971 rpcz_store.cc:276] 0709 12:18:51.543051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.975470 (+1432419us) service_pool.cc:224] Handling call
0709 12:18:53.111114 (+135644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.111718 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.111879 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.069713 32652 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:53.115274 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.118054 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.096313 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 11 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.077651 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.143898 31963 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:53.145136 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.145325 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:53.147392 31970 raft_consensus.cc:2376] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 6. Re-sending same reply.
I20260709 12:18:53.149718 31963 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:53.148960 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 421) took 1644 ms (client timeout 1662 ms). Trace:
I20260709 12:18:53.149025 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2203) took 1602 ms. Trace:
I20260709 12:18:53.150017 31964 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
W20260709 12:18:53.150557 31970 rpcz_store.cc:269] 0709 12:18:51.504471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.532708 (+ 28237us) service_pool.cc:224] Handling call
0709 12:18:53.126363 (+1593655us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911950a0 00007f2291194984 00007f228a70f6d4 00007f228a70f662
0709 12:18:53.148907 (+ 22544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928799488}
I20260709 12:18:53.151335 31971 rpcz_store.cc:276] 0709 12:18:51.546154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.147535 (+1601381us) service_pool.cc:224] Handling call
0709 12:18:53.148038 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.148490 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:53.148539 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:53.148609 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:53.148674 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.148781 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.148942 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2241,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176}
I20260709 12:18:53.152623 31964 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:53.155292 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2204) took 1608 ms. Trace:
I20260709 12:18:53.156076 31964 rpcz_store.cc:276] 0709 12:18:51.933771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.152141 (+1218370us) service_pool.cc:224] Handling call
0709 12:18:53.152446 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.155062 (+  2616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.155239 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:53.156860 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 382201us
I20260709 12:18:53.157495 31964 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:53.157900 31964 pending_rounds.cc:85] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 6
I20260709 12:18:53.155730 31971 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:53.157797  3956 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:53.159250 31964 pending_rounds.cc:107] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 12.7
W20260709 12:18:53.160154 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0860637920002318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2157s for a response
I20260709 12:18:53.160607 31964 raft_consensus.cc:2889] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:53.158259 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 383049us
I20260709 12:18:53.161875 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.159970  3956 consensus_queue.cc:1243] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 16.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:53.164088 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.164877 32047 raft_consensus.cc:2410] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:18:53.166306 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.159996 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 441) took 1327 ms. Trace:
I20260709 12:18:53.166612 32655 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:53.157927 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.166816 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.167523 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:53.109881  3722 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.151599 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2202) took 1605 ms. Trace:
I20260709 12:18:53.193137 31963 rpcz_store.cc:276] 0709 12:18:51.545867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.144648 (+1598781us) service_pool.cc:224] Handling call
0709 12:18:53.149561 (+  4913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.151309 (+  1748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.151538 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.153004 31970 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:18:53.167841 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:53.166744 31971 rpcz_store.cc:276] 0709 12:18:51.832719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.154281 (+1321562us) service_pool.cc:224] Handling call
0709 12:18:53.154573 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.159753 (+  5180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.159943 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.169822 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.195205 32047 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:53.195609 31971 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:53.196023 31970 pending_rounds.cc:85] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 8
I20260709 12:18:53.196079 31971 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:53.197185 31971 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 15.6, Last appended by leader: 6, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.198906 31971 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:53.170246 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.194533 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.198580  3765 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:53.196178 32121 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:18:53.202294 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.208235 31965 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:18:53.208729 31963 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:53.166100 31967 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:53.239894 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.240425 31963 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:53.240782 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.212210  3919 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.212589 32039 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:53.212929 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.213502 32038 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:53.215866  3722 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.218694 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:53.247633 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:53.219065 32112 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:18:53.247565 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:53.220000 32117 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:53.173058  3956 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:53.225145 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 736) took 2435 ms. Trace:
I20260709 12:18:53.248364 32118 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:53.226735 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.227778 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:53.210249 31972 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:53.250015 32046 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:53.203969  3765 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:53.204077 31993 debug-util.cc:398] Leaking SignalData structure 0x7b0800214780 after lost signal to thread 32071
I20260709 12:18:53.249092 32658 rpcz_store.cc:276] 0709 12:18:50.789789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.790401 (+   612us) service_pool.cc:224] Handling call
0709 12:18:52.778449 (+1988048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.969842 (+191393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.224927 (+255085us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b540061e750. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:53.225079 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4541750656}
W20260709 12:18:53.252353 31993 debug-util.cc:398] Leaking SignalData structure 0x7b0800285bc0 after lost signal to thread 32072
I20260709 12:18:53.228741 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.204252  3938 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:53.205447 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.205518 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:53.212610 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.253438 32113 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.253762 31968 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 7.
I20260709 12:18:53.254952 31968 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:53.247215  3862 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:53.255473 32038 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:53.256193 31968 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:53.257362 31968 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:53.257933  3722 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.231360 32120 raft_consensus.cc:1217] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[5.7-5.7]   Dedup: 5.7->[]
I20260709 12:18:53.231863 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.259279 32657 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:53.261426 32120 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:18:53.261607 31969 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 6.
I20260709 12:18:53.232885 31971 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 6. Preceding OpId from leader: term: 16 index: 6. (term mismatch)
I20260709 12:18:53.259599 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.262380 31971 pending_rounds.cc:85] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 5
I20260709 12:18:53.262714 31971 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 15.6
I20260709 12:18:53.262950 31971 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:53.263164 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57826us
I20260709 12:18:53.261327 32117 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:53.206398 31966 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:53.264406 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2208) took 1315 ms. Trace:
I20260709 12:18:53.264708 31966 rpcz_store.cc:276] 0709 12:18:51.949077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.200836 (+1251759us) service_pool.cc:224] Handling call
0709 12:18:53.201174 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.264180 (+ 63006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.264344 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.263587 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2207) took 1314 ms. Trace:
I20260709 12:18:53.265717 31971 rpcz_store.cc:276] 0709 12:18:51.948791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.195180 (+1246389us) service_pool.cc:224] Handling call
0709 12:18:53.195480 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.263356 (+ 67876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.263541 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.266665 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.267505 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.209345 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2205) took 1275 ms. Trace:
I20260709 12:18:53.196472 31970 pending_rounds.cc:107] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:18:53.237913 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:53.202597 32084 connection.cc:256] RPC call timeout handler was delayed by -2.1558390409998083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 12:18:53.246279 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 79055.7us
I20260709 12:18:53.246654 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2201) took 1701 ms. Trace:
I20260709 12:18:53.219478 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:53.248528 31963 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.241067  3575 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.247166  3801 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:18:53.207639 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.279244 32047 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:53.230616 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 524) took 2375 ms. Trace:
I20260709 12:18:53.259016 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.259214  3919 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:53.252766  3866 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.282114  3919 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:53.282652 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.283164  3919 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.272661  3782 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.284401  3782 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:53.284242  3862 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:53.284839  3782 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.285748 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.286357 32041 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.286639 32654 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:53.247238  3749 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:18:53.271621 32116 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:53.270720 32084 connection.cc:256] RPC call timeout handler was delayed by -2.1558390409998083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 12:18:53.274471 31970 raft_consensus.cc:2889] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:53.290441  3765 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:53.291042 32084 connection.cc:256] RPC call timeout handler was delayed by -2.1536204579999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
W20260709 12:18:53.291419 32084 connection.cc:256] RPC call timeout handler was delayed by -2.1536204579999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 12:18:53.291528 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 439) took 1459 ms. Trace:
W20260709 12:18:53.291724 32084 connection.cc:256] RPC call timeout handler was delayed by -2.0382753470000807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24950000000000003s for a response
W20260709 12:18:53.292062 32084 connection.cc:256] RPC call timeout handler was delayed by -2.0382753470000807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24950000000000003s for a response
I20260709 12:18:53.269927 31964 rpcz_store.cc:276] 0709 12:18:51.934163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.156866 (+1222703us) service_pool.cc:224] Handling call
0709 12:18:53.157123 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.197805 (+ 40682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.209280 (+ 11475us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.291936 31970 rpcz_store.cc:276] 0709 12:18:51.832012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.152370 (+1320358us) service_pool.cc:224] Handling call
0709 12:18:53.152661 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.291311 (+138650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.291477 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.279709 32115 rpcz_store.cc:276] 0709 12:18:50.855520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.855630 (+   110us) service_pool.cc:224] Handling call
0709 12:18:50.855872 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.906159 (+ 50287us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:50.911320 (+  5161us) log.cc:844] Serialized 14 byte log entry
0709 12:18:53.078833 (+2167513us) log.cc:844] Serialized 14 byte log entry
0709 12:18:53.140006 (+ 61173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:53.140070 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:53.144844 (+  4774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.230365 (+ 85521us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b540010ead4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:53.230416 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.230557 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14222,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":3623,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":175,"spinlock_wait_cycles":115037952,"thread_start_us":2227349,"threads_started":2,"wal-append.queue_time_us":178926,"wal-append.run_cpu_time_us":945,"wal-append.run_wall_time_us":1000981}
I20260709 12:18:53.202713  3633 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.283699  3866 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.290535 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.269995 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:53.298812 31965 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2180) took 2203 ms (client timeout 1763 ms). Trace:
I20260709 12:18:53.268533  3697 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:18:53.263972  3495 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:53.278299 32113 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
W20260709 12:18:53.302161 31965 rpcz_store.cc:269] 0709 12:18:51.095158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.096016 (+   858us) service_pool.cc:224] Handling call
0709 12:18:53.298655 (+2202639us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000f5d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:53.298751 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3158655360}
I20260709 12:18:53.299254 31963 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:53.307446 32041 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:18:53.307562 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2286) took 1254 ms. Trace:
I20260709 12:18:53.274858 31967 rpcz_store.cc:276] 0709 12:18:51.545587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.142768 (+1597181us) service_pool.cc:224] Handling call
0709 12:18:53.163007 (+ 20239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.246429 (+ 83422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.246603 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.247473  3944 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:18:53.258787  3795 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:18:53.259843  3669 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:18:53.312920 32660 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:53.313444 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.311017  3801 consensus_queue.cc:1243] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:18:53.310586 31993 debug-util.cc:398] Leaking SignalData structure 0x7b080039a7e0 after lost signal to thread 32145
W20260709 12:18:53.316946 31993 debug-util.cc:398] Leaking SignalData structure 0x7b0800321300 after lost signal to thread 32146
W20260709 12:18:53.318116 31993 debug-util.cc:398] Leaking SignalData structure 0x7b08003fa500 after lost signal to thread 32684
I20260709 12:18:53.318331 31972 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:53.298179  3938 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:53.285403  3952 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.308115 32113 rpcz_store.cc:276] 0709 12:18:52.053628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.068323 (+ 14695us) service_pool.cc:224] Handling call
0709 12:18:53.228664 (+1160341us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5c000eaf0c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228fb39161 00007f228fcdaa79 00007f22911949b6 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:53.307510 (+ 78846us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476171520}
I20260709 12:18:53.322741 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:18:53.283859  3681 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:18:53.293747  3894 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:18:53.294291  3722 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:18:53.321615  3862 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:53.309850  3793 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:53.333431 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 65993us
I20260709 12:18:53.312528  3795 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:53.298537  3575 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.335144 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.339258 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.340214 32653 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:53.345451 31935 leader_election.cc:400] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9.
I20260709 12:18:53.345786 31935 leader_election.cc:403] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:53.346297  3801 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 8. Current term is 9.
W20260709 12:18:53.346645  3818 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:18:53.347515 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:53.348232 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55547.7us
I20260709 12:18:53.348531 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.348997  3818 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:53.349386  3818 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.349870  3818 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.349478  3801 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:53.350467 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:53.350777  3801 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:53.351724  3801 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.352442 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:53.353526 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.351338  3749 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:53.354599  3749 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:53.354993  3749 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:53.347674 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 106978us
I20260709 12:18:53.360014  3755 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:53.360504  3755 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:53.360919  3755 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:53.361460 32084 connection.cc:256] RPC call timeout handler was delayed by -1.7256591540001409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22850000000000004s for a response
W20260709 12:18:53.378216 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6838407010000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22850000000000004s for a response
I20260709 12:18:53.360075 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.379115 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:53.381841 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:53.383291 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.385417 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.344604  3495 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:53.333882  3952 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.342857  3938 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:53.342928  3862 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:53.412940 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90276.9us
W20260709 12:18:53.419539 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.417656 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57659.6us
I20260709 12:18:53.413110 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2127) took 1090 ms. Trace:
I20260709 12:18:53.377362  3965 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:18:53.420917 32120 rpcz_store.cc:276] 0709 12:18:52.322994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.533205 (+210211us) service_pool.cc:224] Handling call
0709 12:18:53.231232 (+698027us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.261861 (+ 30629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.412894 (+151033us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b540010ead0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:53.413053 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603975552}
I20260709 12:18:53.390292  3697 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:53.392719  3764 raft_consensus.cc:2764] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:53.392091  3944 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:53.431514 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.434535 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.435539 32047 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.421108  3938 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.424206  3924 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:53.343394  3956 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:18:53.419284  3968 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:53.441254 31869 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 11, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.443272 31869 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.401893  3669 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 12:18:53.440176 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.447577  3952 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.441159  3862 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.450995 32047 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:53.454905 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:53.451474  3801 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.434625  3965 consensus_queue.cc:1243] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:53.463526 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:53.455420 31869 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.474505 31869 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 9 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.476689 31869 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.425647 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.478857 31869 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 9, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 9 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.483973  3697 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:53.483783 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50581.7us
I20260709 12:18:53.485258  3697 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:53.485924 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.485834 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 132369us
I20260709 12:18:53.480786 31869 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 12, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:53.542101 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.544106 31869 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 7, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 7 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:53.548203  3801 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:53.549733  3633 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.549604  3749 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.545830 31869 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 10, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:53.551770  3749 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.551151 31869 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 11, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:53.552850 31869 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 12, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:53.558290  3742 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.004s
W20260709 12:18:53.574137 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 86834.6us
W20260709 12:18:53.574474 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 148901us
I20260709 12:18:53.575297 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:53.575558 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:53.581872 31869 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:53.583537 31869 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 10, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 10 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:53.588685 32047 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:53.585098 31869 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 11, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:53.595724 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:53.462781  3924 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:53.592125  3801 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:18:53.596751 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.617986 32047 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:53.489014  3722 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:53.616748  3924 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.596309 31869 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 5 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.556520  3968 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:53.594048  3749 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:53.612879 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:53.614423  3894 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:53.615166 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.634133 31869 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 10, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.587348  3965 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:53.595237  3942 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:18:53.476038  3970 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.161s
I20260709 12:18:53.631075  3969 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:18:53.627063  3722 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:53.626284  3908 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.592816  3964 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:53.587019  3782 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.641795 31869 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:53.668223 31869 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:53.663033 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.665011 32046 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:53.671077 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.671972 32045 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.671077 32047 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:53.679976 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:53.680583 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.681263 32039 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.680665 32041 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:53.670104 31869 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:53.665920  3944 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:53.687353 31869 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 4 to 5, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 5 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:53.695252  3976 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:18:53.715101 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.652057  3755 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.705480 32045 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:53.643764  3982 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:18:53.702052 32039 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:53.685998  3965 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:53.752744  3965 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:53.753621  3965 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.754938  3965 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.691358  3944 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:53.774367 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 286459us
I20260709 12:18:53.741567  3782 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.786149 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.787058 31967 raft_consensus.cc:2376] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:18:53.791080 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.791756 31967 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
W20260709 12:18:53.796335 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.798225 32660 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:53.800020  3988 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:53.800395 32113 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:53.802506 32113 raft_consensus.cc:606] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:53.802961 32113 raft_consensus.cc:614] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:53.804780 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:53.807548 31967 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:53.813648 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:53.814985 31963 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:53.816362 31965 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:53.816867 31965 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:53.817780 31965 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.819875 31965 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:53.819869 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 207103us
I20260709 12:18:53.823006 31963 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:53.775460  3944 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.826532 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.832386 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:53.839005 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:53.862085 31970 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:18:53.839489  3782 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:53.840018  3980 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:53.863201  3782 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:53.821148  3976 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:18:53.850411 31965 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:18:53.854950 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:53.865744 32045 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.855566 32113 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:53.856137 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.858232 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.858801 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:53.859901 32121 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:53.860008 32115 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:53.827348 31972 raft_consensus.cc:2376] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13. Re-sending same reply.
I20260709 12:18:53.774917  3722 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.880276  3378 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:53.880779  3378 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:53.881652  3378 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.884874  3938 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:53.885293  3755 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.887765 32657 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 12:18:53.902571 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 127369us
I20260709 12:18:53.889010  3908 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:53.889529  3972 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:18:53.892431 32658 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:53.888527 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:53.899106  3924 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.868283  3993 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:18:53.906765  3378 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.926461  3779 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:53.926919  3779 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:53.928064  3779 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.929437  3779 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.912156  3924 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.903416  3979 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:18:53.934919  3979 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:53.935287  3979 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:53.935554  3924 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.936116  3979 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:53.937386  3979 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.939431  3944 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.909900  3782 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.941030  3968 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:53.941437  3968 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:53.948938  3818 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.000s
I20260709 12:18:53.948773  3935 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:53.949486  3935 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:53.949461  3818 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:53.941582  3944 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.868543 31965 pending_rounds.cc:85] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 6
I20260709 12:18:54.036707 31965 pending_rounds.cc:107] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:18:54.037195 31965 raft_consensus.cc:2889] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:53.994846 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 398981us
I20260709 12:18:54.041153 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:54.043076  3495 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:18:53.996562 32038 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:54.044224  3924 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:53.998857 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.893205 32118 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:53.997803 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 165356us
I20260709 12:18:54.020411 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:54.054608 31972 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.021760 32121 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:54.022172 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:54.025187 32116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:54.028899  3935 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.030799  3755 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.032083  3782 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.030933  3818 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:53.942188  3968 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.035346  3575 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:54.030836 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.079349 32116 raft_consensus.cc:606] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:54.097891 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43982.6us
W20260709 12:18:54.098444  4006 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:18:54.099234 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:54.100622  3782 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.104038  3782 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.107268 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:54.107609  3995 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:54.108229  3995 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:54.121366  4004 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:54.122210 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67498us
I20260709 12:18:54.125377 31964 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:54.125612 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:54.126649 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.127182 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.127703 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.126663 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.109951  3995 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.143522 32658 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:54.145514  3974 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.146116  3974 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:54.146582  3974 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.146698 32120 pending_rounds.cc:85] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 3
I20260709 12:18:54.147046 32120 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:18:54.147306 32120 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:54.147681 32120 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:18:54.145529  3681 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.152344  3681 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.155437  3993 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:18:54.159219  3990 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:54.166977  3935 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:54.168570 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.168990 32120 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:54.176993 31972 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:54.186092  3681 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:54.191306 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 92322.4us
I20260709 12:18:54.192642 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:54.195827 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.144805  3995 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:54.195636 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67397.3us
I20260709 12:18:54.189971 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:54.204480 31970 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:18:54.210270 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:54.206703 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:54.221400 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.229326 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 751) took 1104 ms. Trace:
I20260709 12:18:54.229961 32653 rpcz_store.cc:276] 0709 12:18:53.033613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.081223 (+ 47610us) service_pool.cc:224] Handling call
0709 12:18:53.340035 (+258812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.340645 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.102160 (+761515us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54003a07d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:54.137487 (+ 35327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594533248}
I20260709 12:18:54.226572 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:54.253477 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:54.254897 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:54.267902 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.265837 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57931.9us
I20260709 12:18:54.283595 32045 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:54.287786  3951 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.288592  3951 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:54.289180  3951 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.295871 32114 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:54.295624  3633 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:54.302896  3782 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.303453  3782 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:54.303903  3782 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:54.308432  4012 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:18:54.309322 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:54.310464 32119 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:54.313421  3898 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:54.313982 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:54.315912 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:54.206451  4004 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.295735  3919 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.330253 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:54.332173  3919 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.341143 32121 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:54.349445  3378 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:18:54.342226  4011 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:18:54.353196  4011 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:54.353597  4011 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:54.354830  4011 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.356309  4011 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.357336 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:54.346014  3919 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.317080  3924 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:18:54.441335  3862 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.441906  3862 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:54.443466 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 129750us
W20260709 12:18:54.443868  4020 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:18:54.444370  4004 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:54.454167  3779 raft_consensus.cc:2764] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:18:54.454085  3681 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:54.453594 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 125980us
I20260709 12:18:54.458151 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:54.458374 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 76130.9us
I20260709 12:18:54.452642  3862 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.466800 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:54.475562 32116 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:54.470599  3528 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:54.376461  3974 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.490711  3378 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:54.491369 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.495040  3919 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.497138  3378 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.499262  3919 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.499414  3378 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:54.499192 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.493559  4004 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:54.523440  4004 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:54.524752  4004 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.504441  3951 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.527825  3974 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:54.531265  3378 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:54.529803  3951 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:54.530660 32656 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:54.534097 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:54.535605  4015 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.426s	user 0.004s	sys 0.000s
I20260709 12:18:54.537084  3935 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.539369 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:54.281057 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:54.544950  3782 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.549597  3951 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:54.550793  3984 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.551298  3984 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.551764  3984 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.551705 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:54.554571 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:54.554982  3728 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.555485  3728 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.555864  3728 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.556286 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:54.557372  3528 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:54.557927  3528 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:54.561452  3528 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.558593  3951 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:54.566847 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.567631  3993 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.568482  3993 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:54.571879 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.575551  3951 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:54.576881  3951 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.580097  3993 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.581286  3935 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.582072  3935 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.585508  3782 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.558866  3378 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.594197  3378 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.595072  3974 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:54.596776  3378 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.597337  3974 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:54.597500  3378 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.605221 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:54.606242 31966 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:54.762862 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.613986  4011 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.155s
I20260709 12:18:54.781581 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:54.651638  3378 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.841838  3866 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:54.842037  3954 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.842733  3974 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:54.847791  3946 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.856750  3974 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:54.857295  3974 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:54.856994  3946 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.852361 31966 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:54.859807  3946 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.855763  3954 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:54.864198 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.865082 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:54.864603  3995 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.001s
I20260709 12:18:54.868702 32113 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:54.869073  3984 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:54.875172  3974 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.877235  3935 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:54.877964  3974 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:54.879030  3728 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.882736  3984 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:54.884153 31965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.029s	sys 0.031s
W20260709 12:18:54.884624 31965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.030s	sys 0.032s
I20260709 12:18:54.885864  3378 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:54.863212  3866 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:54.887002 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:54.918905 31970 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
W20260709 12:18:55.328775 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.460142 31969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.003s	sys 0.012s
W20260709 12:18:55.460662 31969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.003s	sys 0.012s
W20260709 12:18:55.721262  4021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.000s	sys 0.055s
W20260709 12:18:55.721899  4021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.000s	sys 0.055s
W20260709 12:18:55.783181 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.834408 31963 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.703s	user 0.031s	sys 0.020s
W20260709 12:18:55.834828 31963 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.741s	user 0.032s	sys 0.020s
I20260709 12:18:55.836236 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 468) took 1815 ms. Trace:
W20260709 12:18:55.841400 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 1.75548e+06us
W20260709 12:18:55.845984  4008 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.729s	user 0.000s	sys 0.019s
I20260709 12:18:55.850515 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:54.890430  3946 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:55.860668 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:55.836614 31963 rpcz_store.cc:276] 0709 12:18:54.020474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.029102 (+  8628us) service_pool.cc:224] Handling call
0709 12:18:54.029816 (+   714us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:54.035754 (+  5938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.035796 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:54.036781 (+   985us) log.cc:844] Serialized 52 byte log entry
0709 12:18:54.081543 (+ 44762us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:55.835307 (+1753764us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.835703 (+   396us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.835875 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.835959 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.836004 (+    45us) raft_consensus.cc:1685] finished
0709 12:18:55.836016 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.836161 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5104,"prepare.queue_time_us":258780,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":6426,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":273,"spinlock_wait_cycles":188800,"thread_start_us":1785183,"threads_started":2,"wal-append.queue_time_us":166003}
I20260709 12:18:55.866051  3946 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:55.867970  3946 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.870183  3986 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.871363  3986 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.874020  3986 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:55.874879 31869 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.679s	user 0.009s	sys 0.041s
W20260709 12:18:55.875255 31869 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.679s	user 0.009s	sys 0.041s
I20260709 12:18:55.875124  3986 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:55.876933  3986 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:55.877466  3986 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:55.878547  3986 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.892813  4037 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:55.894297 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.2564e+06us
I20260709 12:18:55.898471  4037 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.901187  4037 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:55.905230 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:55.910966  3986 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:55.911554  3986 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.913093  3986 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:55.913514  3986 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.913911  3986 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.918147  4038 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.918861  3986 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:55.919394  3986 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.919252  4038 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.919955  3986 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.921998  3986 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:55.922529  3986 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.923147  3986 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.925294  3986 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:55.925769  3986 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.931173  4041 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.932428 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:55.935324 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:55.927912  4038 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:55.937734 31972 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:55.939280 31972 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:55.940591 31972 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:55.942865 31966 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.942970 31965 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.944381 31965 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:55.945205 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:55.944049 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:55.956238  4040 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:55.957057  4040 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.944120  4037 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:55.962708  4037 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:55.965797 32068 debug-util.cc:398] Leaking SignalData structure 0x7b080016b8e0 after lost signal to thread 30820
W20260709 12:18:55.966562 32068 debug-util.cc:398] Leaking SignalData structure 0x7b0800095fe0 after lost signal to thread 30823
I20260709 12:18:55.966709  4042 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.967562  4042 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.968287  3378 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.969732 31963 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:55.971213 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:55.971849 31963 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.960523  4038 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.969293  3378 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.980042  3378 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:55.980806  3378 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.981391  3378 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.974874  4038 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.977793  4042 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:55.985316  4038 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:55.986824 32084 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.827s (SENT) [suppressed 33 similar messages]
I20260709 12:18:55.955263  4041 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:55.995935 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:55.996641 31972 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:55.988065  3986 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:55.997613  3954 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:55.999907  3986 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.000976 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:56.001732 31969 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:56.007042 31963 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:56.008028 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:56.009617 31963 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:56.010633 31963 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:18:56.011762 31963 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:56.012822 31963 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:56.014407 31972 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.015292 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:56.020265 31963 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:56.024206 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 475) took 1449 ms. Trace:
W20260709 12:18:56.024235 32084 leader_election.cc:336] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.160s (SENT)
I20260709 12:18:56.024566 31963 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:56.024958 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:56.026207  3991 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.024722 31971 rpcz_store.cc:276] 0709 12:18:54.574632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.899677 (+1325045us) service_pool.cc:224] Handling call
0709 12:18:55.900056 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.926180 (+ 26124us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:55.926699 (+   519us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.936006 (+  9307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.943468 (+  7462us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.943616 (+   148us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:55.943827 (+   211us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.943997 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.945098 (+  1101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.023866 (+ 78768us) raft_consensus.cc:1685] finished
0709 12:18:56.023890 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.024020 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2996,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":97,"raft.queue_time_us":130160,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":3165,"thread_start_us":25268,"threads_started":2,"wal-append.queue_time_us":66259}
W20260709 12:18:56.029071 32084 connection.cc:256] RPC call timeout handler was delayed by -2.4244450289997985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.272s for a response
I20260709 12:18:56.029212 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 557) took 1664 ms. Trace:
I20260709 12:18:56.029701 32113 rpcz_store.cc:276] 0709 12:18:54.364929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.368194 (+  3265us) service_pool.cc:224] Handling call
0709 12:18:54.440135 (+ 71941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.869087 (+428952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.028966 (+1159879us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54002fa654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:56.029141 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984598912}
W20260709 12:18:56.030731 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9807450289999906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32130000000000003s for a response
I20260709 12:18:56.026141 31963 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
W20260709 12:18:56.031119 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9258450289999018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3274s for a response
I20260709 12:18:56.031498 31963 pending_rounds.cc:85] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 9
I20260709 12:18:56.031759  3991 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.031925 31963 pending_rounds.cc:107] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:56.032450 31963 raft_consensus.cc:2889] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:56.032480  3991 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.035490  3991 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.036369  3991 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.036950  3991 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.039219  3991 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:56.040138 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3393663100000595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 12:18:56.041087 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3195663100000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:18:56.042138  3973 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.042963  3973 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.040601  3991 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:56.044763 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3033663100000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:18:56.047029  3973 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.047797  3973 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.021232  4041 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.048528  3973 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.050066  3378 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.052570  3973 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.054368  3973 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:56.060583  3973 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:56.044363  3991 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.069278  3973 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:56.069909  3991 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.075224  4040 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.086436 31972 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:56.070035  3973 consensus_queue.cc:1243] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:18:56.090461 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68171.3us
I20260709 12:18:56.101696  3378 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.102411  3378 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:56.105232  4030 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.221s	user 0.002s	sys 0.008s
I20260709 12:18:56.112172  3378 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.110180 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2161) took 1968 ms. Trace:
I20260709 12:18:56.115758 32120 rpcz_store.cc:276] 0709 12:18:54.141487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.145587 (+  4100us) service_pool.cc:224] Handling call
0709 12:18:54.146150 (+   563us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:18:54.182994 (+ 36844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.183027 (+    33us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:18:54.183977 (+   950us) log.cc:844] Serialized 104 byte log entry
0709 12:18:54.318954 (+134977us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:54.338126 (+ 19172us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.108889 (+1770763us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b1800344b00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f228bd85d70 00007f228bd79293 00007f228bd6ecea 00007f228bd6f23a 00007f228c1d9a2a 00007f228c1e90bf 00007f228c1e9038
0709 12:18:56.109349 (+   460us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.109609 (+   260us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.109744 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.109903 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.109950 (+    47us) raft_consensus.cc:1685] finished
0709 12:18:56.109966 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.110119 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1666711,"prepare.queue_time_us":11211,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":1643,"raft.queue_time_us":9323,"spinlock_wait_cycles":19978752,"thread_start_us":134015,"threads_started":1,"wal-append.queue_time_us":151023}
I20260709 12:18:56.112787  3973 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:56.093273  4040 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.122583 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:56.123464 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 478) took 1262 ms. Trace:
I20260709 12:18:56.123821 31968 rpcz_store.cc:276] 0709 12:18:54.861053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.122115 (+1261062us) service_pool.cc:224] Handling call
0709 12:18:56.123406 (+  1291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:56.121241  4040 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.126438  3681 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:56.127018  3681 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.128602  4040 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.129364  3681 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.129398  3954 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:56.130139  3954 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.131110  4050 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:56.132428  3954 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:56.132843 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.133582 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.136224 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.137274  3973 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.137639 32120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:56.138264 32120 raft_consensus.cc:606] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:56.137919  3973 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.115332  4037 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.139716 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 551) took 2027 ms. Trace:
I20260709 12:18:56.140152 32117 rpcz_store.cc:276] 0709 12:18:54.111818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.122807 (+ 10989us) service_pool.cc:224] Handling call
0709 12:18:54.123221 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.347196 (+223975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.347261 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:54.349580 (+  2319us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:54.349636 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.139370 (+1789734us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:56.139418 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.139542 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500454656}
I20260709 12:18:56.141098 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.141418 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.141845 32652 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:56.143316 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.144014 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.150183 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.151111 32115 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:56.151914 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.152348  3866 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:56.153497 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.154726 32652 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:56.159585 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.162916 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.163604 32115 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:56.164598 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.168527 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57239.4us
I20260709 12:18:56.176673 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.177484 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.178854 32113 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:56.179651 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.140368  3973 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.185608 32113 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:56.182778 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:56.184653 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:56.180773 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.194298 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.194903 32113 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:56.192819  4037 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:56.196527  4037 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.196936  4037 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.198077 32115 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:56.205435 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.207007 32117 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 16 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 5.
I20260709 12:18:56.211869 32654 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.212903 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.214421 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.215723 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.216328 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:18:56.217236 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.220340 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.221027  4052 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.221478 32115 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:56.221623  4052 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.223241 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.231631 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.227985 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:56.235167 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.237108 32115 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:56.242522 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.243552 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.243950 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.244303 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.245014 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.245092 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.246273 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.245426 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.254634 32115 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:56.245815 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.256485 32115 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:18:56.245115 32113 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:56.256496 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.256913 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:56.253762 32117 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
W20260709 12:18:56.257565 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 2.0073e+06us
I20260709 12:18:56.254993 32112 raft_consensus.cc:2376] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11. Re-sending same reply.
I20260709 12:18:56.259758 32121 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:56.240048  4052 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.258481 32115 pending_rounds.cc:85] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 10
I20260709 12:18:56.260990 32119 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:18:56.262497 32115 pending_rounds.cc:107] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:18:56.262887 32115 raft_consensus.cc:2889] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:56.262174 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.267314 32117 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:56.270473 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.271993 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.273018  4042 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:56.274019  3973 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.274660  3973 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.277127  3973 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.277026  4058 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:18:56.278982  4004 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:56.279579  4004 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.280058  4004 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:56.281983  4052 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:56.282618  4052 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.284852  4052 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:56.285633  4052 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:56.271888  3986 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:56.290791  3986 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.291261  3986 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.270066  4041 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:18:56.300567 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.301143 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.302361 32654 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:56.300664 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.308319 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.306771  4038 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:56.299810  4042 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:56.277899  3973 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:18:56.314054 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.318943 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.328531 32119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:18:56.329190 32119 raft_consensus.cc:606] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:18:56.330495 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.326591 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.333493 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.306989  4040 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.337070 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.338097 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 708) took 2112 ms. Trace:
I20260709 12:18:56.312757  4038 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:56.338716 32656 rpcz_store.cc:276] 0709 12:18:54.224891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.229728 (+  4837us) service_pool.cc:224] Handling call
0709 12:18:54.230171 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.532542 (+302371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.332465 (+1799923us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54004e4b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:56.335194 (+  2729us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690242304}
I20260709 12:18:56.312132  4037 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:56.347517 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 105261us
I20260709 12:18:56.342677  4038 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.349926 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.356328  4004 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:56.375178 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.379245  4004 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.349303  4040 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.383473  4037 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.384208  4037 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:56.384531  4040 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.384747  4067 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:56.385145  4037 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:56.385280  4040 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:56.385744  4037 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.349773  3378 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:56.387964 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.388316 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:56.376959  4042 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.392257 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:56.395023 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 473) took 1821 ms. Trace:
I20260709 12:18:56.395392 31967 rpcz_store.cc:276] 0709 12:18:54.573931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.141765 (+1567834us) service_pool.cc:224] Handling call
0709 12:18:56.142198 (+   433us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:18:56.142794 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.144527 (+  1733us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:18:56.145480 (+   953us) log.cc:844] Serialized 104 byte log entry
0709 12:18:56.343319 (+197839us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:56.343742 (+   423us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.351490 (+  7748us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.394157 (+ 42667us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.394450 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.394590 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.394658 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.394693 (+    35us) raft_consensus.cc:1685] finished
0709 12:18:56.394706 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.394854 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2624,"prepare.queue_time_us":7245,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":800,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":139,"thread_start_us":189321,"threads_started":3,"wal-append.queue_time_us":196048}
I20260709 12:18:56.385995  4040 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:56.400146 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.385313  4067 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.404037  4067 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.404031 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:56.404915 32656 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.413662  4067 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.415010  3968 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.415140  3986 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.390667  3378 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.422904 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.423743 32047 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:56.424966  4036 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.425516  4036 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.416288  4067 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.386184  4037 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.431286  4068 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.434759  3946 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:18:56.439538  3765 raft_consensus.cc:2764] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:56.440600 31966 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:18:56.417474  3378 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.453495 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.455962 32047 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:56.454778  4040 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.436249  3954 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:18:56.462916 32039 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:56.463093  3954 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:56.463722  3954 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:56.464999  3954 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.468954  3954 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:56.472940  3862 consensus_meta.cc:287] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 2.010s	user 0.007s	sys 0.016s
I20260709 12:18:56.473399  3862 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.475209 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.486217 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.487269 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:56.492070 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.493083 32118 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:18:56.494937 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.495563 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.496418 32113 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.502835 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.503706 32047 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:56.504797 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.505102 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.505457 32047 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.509662 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.510670 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:18:56.515944 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:56.495947 32043 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:56.505997 32112 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.521247 32656 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
W20260709 12:18:56.523221 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 134981us
I20260709 12:18:56.531627 32659 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:56.535933  3681 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:56.496726 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.538817 32042 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.539974 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.541298 32039 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:56.541977  3986 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.543088 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:56.549818 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.551430 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.534058 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 764) took 1921 ms. Trace:
I20260709 12:18:56.562558 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.576402 32041 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.574522 32659 rpcz_store.cc:276] 0709 12:18:54.612660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.612905 (+   245us) service_pool.cc:224] Handling call
0709 12:18:54.613374 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.532102 (+1918728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.533826 (+  1724us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:56.534004 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4380055424}
I20260709 12:18:56.577219 32652 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:18:56.576779 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.587877 32116 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:56.587978 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.585431 32047 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:56.581261  3681 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:56.600224 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.602047 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.602475 32113 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:56.603128 32117 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:18:56.606442 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90690.5us
I20260709 12:18:56.611985 32112 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:56.612759 32116 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:56.562775  4038 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.613587 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.614298 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.605988  4042 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:56.599488  3991 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:18:56.620033 32116 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:56.620487 32047 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:56.621261 32116 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:56.622298 32042 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:56.630261 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.632247  3862 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.619282  3970 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.633244 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.634478 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.636114 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.618386  4042 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.638000 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:56.638835 32116 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.639863 32116 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.642071 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2115) took 2978 ms. Trace:
I20260709 12:18:56.642434 32038 rpcz_store.cc:276] 0709 12:18:53.664087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.664202 (+   115us) service_pool.cc:224] Handling call
0709 12:18:53.664526 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.044497 (+379971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.641814 (+2597317us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54000339d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:18:56.641999 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761912576}
I20260709 12:18:56.643781 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.644337 32038 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:56.638000 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.645109 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:56.650048 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.656769 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:56.657085 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:56.657408 31967 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:18:56.657655 31970 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:56.657869 31969 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:56.650782  3991 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:18:56.650925 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.660068 32114 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:56.660946  4065 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:56.661917 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:56.663785 31970 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:56.655318 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.664680 32047 raft_consensus.cc:2376] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12. Re-sending same reply.
I20260709 12:18:56.665508 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.666559 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.667492 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.648306 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:56.668385 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.669173 32047 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:18:56.669359  3990 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.669898 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.661700  4065 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.670882 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:56.652726 32043 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:56.660993 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.655339 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.637593 32041 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:56.658026 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.685560 32116 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:56.688897 31970 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 8.
I20260709 12:18:56.691044  4041 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.691658  4041 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.639261  4042 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:56.694653 32047 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:56.684451  3946 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:56.695694  3946 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.696061  3946 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.696692 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.697737 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.698624  3782 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:56.699182  3782 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:56.700047 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:56.695839 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.700570 32660 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.699029  4038 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.707918 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:56.725867 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77598.4us
W20260709 12:18:56.726264  4069 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:18:56.707171  3986 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:56.726253 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.729352 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.743742 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.700260  3782 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.711768 32038 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:56.752424 32116 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:56.756837 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:56.761468 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:56.763610 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.766043 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:56.779589 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67549.4us
I20260709 12:18:56.780138 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:56.782610 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.753715  3495 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:56.782791  3528 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.791847 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.792906  3984 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:56.793433  3984 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:56.795531  4071 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:18:56.795169  3984 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.767584  3968 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.796959  4041 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.797792 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.798125 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.798731 32117 raft_consensus.cc:2410] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:18:56.799053  4041 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.799602  3926 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.800738 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.800957 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.801556  3968 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:56.782662 32041 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:56.789896 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:56.768023  3986 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:56.783350 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.806043 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.803494 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:56.790720 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:56.785658  3935 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.790375  3970 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:18:56.790830  3866 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:18:56.819267  4041 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.819752  3378 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.819896  4041 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.781649  4038 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:56.819268  3970 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:56.824596  3970 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:56.825421  3970 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.761278  4037 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.826777  3970 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:56.827019  3978 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:56.827627  3978 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:56.828375  4037 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.803427 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.829094  4036 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.829620  4036 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:56.829828 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.829970  4036 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.830394  4036 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.830600  4064 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:56.828301  4041 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.831139  4064 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:56.832319  4064 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.832813  3986 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.832270  4076 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:18:56.834475 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.835340 32121 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:56.828426  3935 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.837049 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.844458  4040 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.848258 32045 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:56.846228 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:56.822052  3378 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.785480  4066 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:56.852566 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:56.784683  3495 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.810101  4067 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.816479  4065 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:56.852152  4070 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:18:56.879213  3946 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.872309  4042 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.851063  3866 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:18:54.882117  3728 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:56.863153 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:56.884671 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 103625us
I20260709 12:18:56.802086  4068 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:18:56.800119  3926 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.889479  3926 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:56.828579  3978 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.855384  4066 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:56.890688  4066 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.886906 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.894618 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.899080 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:56.899875 32121 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:18:56.861052  3495 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:56.898588  4068 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:56.906414 32653 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:56.927038 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.929402 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.934893 32121 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:56.937752 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.905114  3378 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:56.951519 32660 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:56.952672 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.953281 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:56.940428 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.943202  3954 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:56.943608  4065 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.939189  3779 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:18:56.941596  3986 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:56.928061  3991 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:18:56.908149  4068 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.979372  3954 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:56.980713  3954 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:56.958226 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:56.952075  4050 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:18:56.959164  4064 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:56.959689  3782 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:56.985510  3782 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:56.985966  3782 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.961078  3995 raft_consensus.cc:2764] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:56.966586  3378 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:56.934018  4083 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:56.991634  4083 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:56.991345  3378 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:56.992164  4083 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.939070  3728 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:56.998224 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:57.017330 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58792.3us
I20260709 12:18:56.934608  4076 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:18:57.021206 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:57.023102 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:56.939656  3990 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:18:56.998275  4068 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:57.030928 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.007361  4064 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:57.038882  3728 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:57.039142  3984 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:57.039533  3990 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:57.040992  3495 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.020542  4065 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:57.046044  4042 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.046177  4064 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.046788  3946 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:57.048697  4078 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:18:57.049202  3681 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.068795 31969 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:57.051851  3970 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:57.055052 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:57.047017 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:57.044116  3728 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.050045  3782 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.055749  3990 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.064584  3984 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:57.088315 32068 debug-util.cc:398] Leaking SignalData structure 0x7b080016a4a0 after lost signal to thread 31904
I20260709 12:18:57.082615  3681 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:57.063277  3986 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:57.091770 32068 debug-util.cc:398] Leaking SignalData structure 0x7b0800167040 after lost signal to thread 31996
W20260709 12:18:57.137728 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 210763us
W20260709 12:18:57.147851 32068 debug-util.cc:398] Leaking SignalData structure 0x7b0800167080 after lost signal to thread 31997
I20260709 12:18:57.136103 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:57.149266 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08003910a0 after lost signal to thread 32071
I20260709 12:18:57.061965  4065 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:18:57.149637 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08002ebf20 after lost signal to thread 32072
W20260709 12:18:57.152084 32068 debug-util.cc:398] Leaking SignalData structure 0x7b080033e980 after lost signal to thread 32145
I20260709 12:18:57.068244  4095 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:57.154024 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08003261a0 after lost signal to thread 32146
W20260709 12:18:57.155721 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08003973a0 after lost signal to thread 32684
W20260709 12:18:57.156201 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08001ddd80 after lost signal to thread 3669
I20260709 12:18:57.159030  3986 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.134881  3970 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.171296  3970 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.158707  4065 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:57.171878  3986 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:57.153788  4040 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:57.164124  3991 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:57.144347  4042 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:57.195245  4088 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:18:57.197451  4085 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:18:57.162703  4036 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.201226  4083 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.175123  4037 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:18:57.187708  4040 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.203910  4040 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.206459  3984 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.207782  4036 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:57.190761  3991 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.209002  3991 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.211586  4083 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.181831  3728 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.174516  3990 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.212095  3782 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:57.213639  3728 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.186569  4093 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:18:57.213645  3990 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.215608  3495 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:57.181362  4095 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:57.210145  3991 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:57.205826  4068 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.188555  4066 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.190570  3779 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:18:57.238106  4068 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:18:57.236071 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67717.9us
I20260709 12:18:57.242290  4066 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.238260  4040 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.254057  4040 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.255606  3681 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.260533 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:57.261225 32117 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:57.269008  4065 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.272185  4065 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:57.272246 32660 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:57.269342  3681 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.274834  3991 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:18:57.276837  3970 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:57.277184  3970 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.277916  3970 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:57.288123 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.341167  4050 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:57.341830  4050 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.343078  4050 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.347266  4050 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:57.344489 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.356284 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.356940 32116 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.362437 32119 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.366645  4101 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:18:57.374620 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.385174 32116 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:18:57.399894 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.401680  3986 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:57.402257  3986 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.404410  3986 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:57.405989 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:57.407821 31970 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:57.411201 31968 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:57.415642 31968 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:57.417320 31970 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.420557 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.421686 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.422143 32118 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.422426 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:57.422674 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.423274 32116 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.423766 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:57.424620 32119 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:18:57.425403 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.430231 32116 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
W20260709 12:18:57.434156  4087 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 12:18:57.440443 31968 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:57.449934 32118 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
W20260709 12:18:57.484436 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 443499us
I20260709 12:18:57.485383 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:57.487010 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:57.487208 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:57.488566 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:57.490202 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:57.491715  4050 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:57.492523  4050 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:57.492928 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.493069  4050 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.498490  4100 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:57.500578  4100 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:57.501103  4100 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.499059  4099 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:57.494268  4050 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.515466 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2003) took 1574 ms. Trace:
I20260709 12:18:57.515878 31966 rpcz_store.cc:276] 0709 12:18:55.941453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.943767 (+  2314us) service_pool.cc:224] Handling call
0709 12:18:57.515265 (+1571498us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54001e8cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:57.515374 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123475712}
I20260709 12:18:57.527707  4100 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.534945 31965 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.544811  4100 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:57.546695 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 806) took 1407 ms. Trace:
I20260709 12:18:57.547266 32653 rpcz_store.cc:276] 0709 12:18:56.139321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.695538 (+556217us) service_pool.cc:224] Handling call
0709 12:18:57.546399 (+850861us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400544054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:18:57.546586 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470842112}
I20260709 12:18:57.554910 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:57.555584 31972 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:57.571143 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08003f9620 after lost signal to thread 3976
W20260709 12:18:57.571771 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08002734e0 after lost signal to thread 3998
W20260709 12:18:57.572103 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6780 after lost signal to thread 4007
W20260709 12:18:57.572449 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08002270a0 after lost signal to thread 4014
I20260709 12:18:57.572345  3924 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.573148  3924 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.580159  3924 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.578640 31972 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:18:57.581224 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 570) took 1139 ms. Trace:
W20260709 12:18:57.588464  4081 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.560s	user 0.001s	sys 0.003s
I20260709 12:18:57.581668 32112 rpcz_store.cc:276] 0709 12:18:56.441467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.612643 (+171176us) service_pool.cc:224] Handling call
0709 12:18:56.612963 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.613340 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.613367 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.617063 (+  3696us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.846125 (+229062us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:56.846174 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.846248 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:57.580991 (+734743us) raft_consensus.cc:1685] finished
0709 12:18:57.581013 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.581161 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":1692544,"thread_start_us":207801,"threads_started":1,"wal-append.queue_time_us":260122}
I20260709 12:18:57.607388  3970 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.623940  3378 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:57.624676  3378 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:57.626149 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.632112 32117 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
W20260709 12:18:57.634564  4057 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.399s	user 0.007s	sys 0.006s
I20260709 12:18:57.636399 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2170) took 1500 ms. Trace:
I20260709 12:18:57.636876  4064 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:57.636799 32120 rpcz_store.cc:276] 0709 12:18:56.139322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.139458 (+   136us) service_pool.cc:224] Handling call
0709 12:18:56.139782 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.140183 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.140215 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.142837 (+  2622us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.210485 (+ 67648us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:56.210524 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.210588 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:57.634057 (+1423469us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:18:57.636006 (+  1949us) raft_consensus.cc:1685] finished
0709 12:18:57.636037 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.636322 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45520,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":4684544,"thread_start_us":23528,"threads_started":1,"wal-append.queue_time_us":52898}
I20260709 12:18:57.657711 31967 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:57.664198 31971 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.664253 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:57.673557 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:57.674180 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:57.675300 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:57.676340 32656 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:57.676488 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:57.678454 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:57.679088 32659 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:57.682762 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:57.637696  4064 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.683591 32658 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:57.689028 32658 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:57.689383 32658 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:57.690153 32658 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.691406 32658 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.703145 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:57.721652 32658 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:18:57.725808 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.3995e+06us
I20260709 12:18:57.727376  4091 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:57.728979  4091 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:57.733173 32120 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:57.733662  4066 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.733907  3765 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:57.736457  3765 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:57.736830  3984 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:57.737422  3984 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.739395  3495 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:57.734216  4066 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:57.735076 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 70900.3us
I20260709 12:18:57.735713  3944 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.751909 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:57.752672 31967 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:57.745028  3495 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.746304 31964 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:57.746964 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:57.754251 31971 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:18:57.757711 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:57.765311 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:57.774895 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.782390 31967 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:57.788820 31964 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:57.793285 31936 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:57.804024 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:57.807890 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:18:57.810108 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.809859 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 155464us
I20260709 12:18:57.811635 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:57.813939 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.808772  3935 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:57.820863 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89653.8us
I20260709 12:18:57.825348 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:57.809329 31967 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.738976  3765 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:57.830590 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:57.838630 32113 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:57.781699  4065 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:57.843308 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:57.787674  3944 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:18:57.845557 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.844995 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:57.859581 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46873.5us
I20260709 12:18:57.861785 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:57.843951 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:57.863205 32041 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:57.806344  3782 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:57.864372 32120 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:57.865419 32115 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:57.866261 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.866577 32117 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:57.867462 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:57.868623 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:57.869633 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.873257 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.876852 32039 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.831140  4097 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.869118 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:57.860059  4078 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:18:57.904587 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:57.896872 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.907333 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.908099 32041 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:57.894459 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 115212us
I20260709 12:18:57.831403  3378 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:57.842634  4036 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:57.909754 32043 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:57.911957 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.912644 32045 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:57.910441 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 77840.1us
I20260709 12:18:57.844131  4096 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:57.916102 32084 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.928s (SENT)
I20260709 12:18:57.916538 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:57.911689 32042 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:57.873864  4092 raft_consensus.cc:2764] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:57.928226 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.878831  3779 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:57.928580 32658 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:57.928880 32044 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:57.931036 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 105770us
W20260709 12:18:57.936434 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93069.8us
I20260709 12:18:57.936684 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.937737 32038 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.936632 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.939378 32043 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.937940 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:57.873275  3528 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:18:57.944108  4036 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:57.944923 32041 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:18:57.946575 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:57.947184 32042 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:57.951265 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.844743  4067 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:18:57.946017 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:57.953733 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.953768 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84421.2us
I20260709 12:18:57.910204  3995 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:57.957563 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:57.958387 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:18:57.961232 32039 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:57.964349 31964 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:57.965459 32041 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:57.971653 32039 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:57.973624 32045 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:57.970026 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:57.954226  3728 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:57.913491  3970 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 12:18:58.015704 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.933563  3779 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:57.980944 32039 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:57.945602  4036 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:57.983243 32041 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:58.040979 32039 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:57.929327  3978 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:18:57.972715  4042 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:57.965775  3765 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.965420  4040 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:57.968257  3995 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:18:57.993302 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.043273  3779 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.983948  3984 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:58.045385 32042 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:57.978434  3991 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:18:57.938835  4101 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:18:57.929723  4096 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.054252 32038 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:57.985270  4065 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:58.048811 32039 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:58.077795 32084 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.040s (SENT)
I20260709 12:18:58.084009 32042 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:58.042503  4100 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:18:57.978837  4091 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:18:58.086273 32039 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 7 FOLLOWER]: Advancing to term 10
I20260709 12:18:58.056816  4040 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:58.074548  3681 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:58.093397  4065 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:58.094184  4080 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
W20260709 12:18:58.096238 31997 heartbeater.cc:646] Failed to heartbeat to 127.30.25.62:33695 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.30.25.62:33695 timed out after 15.000s (SENT)
W20260709 12:18:58.088127 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 119422us
I20260709 12:18:57.985821  4092 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:58.078984  3995 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:57.974730  3944 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.083843  3495 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:58.101161 32044 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:58.086781  3984 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.085532 32042 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.091523  3978 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:18:58.131865 32084 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.157s (SENT)
I20260709 12:18:58.132879  3978 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:58.134990 32084 leader_election.cc:336] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.192s (SENT)
W20260709 12:18:58.136032 32084 leader_election.cc:336] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.194s (SENT)
I20260709 12:18:58.136337  4066 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:58.136415 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:58.137434  3990 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:58.138866 32084 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.231s (SENT)
W20260709 12:18:58.140664  4119 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:18:58.143676  4124 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:18:58.097781  4040 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:58.162505  4096 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:58.164674 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:18:58.169045 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 201159us
I20260709 12:18:58.169813 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:58.174160 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.176033 32039 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:58.178233 32042 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.087091  4068 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:58.109658  3681 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:58.122920  4080 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:58.088164  4067 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
W20260709 12:18:58.215188 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 168765us
I20260709 12:18:58.216593 32660 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:58.226179 31967 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:58.117697  4092 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.202263  4080 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:58.222668  3495 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:58.244598 31966 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:58.244741 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74998.7us
W20260709 12:18:58.244645 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 328907us
I20260709 12:18:58.250465 31972 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:58.251087 32118 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:58.252554 32042 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:58.254703 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:58.257228 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:58.255901 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:58.263566 31971 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:58.230538  3991 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:58.245754  3495 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:58.293869  3495 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.261301 31964 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:58.288913  3984 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:58.289722  4137 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:58.290328  3986 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:58.290580  4140 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:58.290800  4067 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:58.267834 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:58.302126 32118 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:58.293741  3991 consensus_queue.cc:1243] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:58.303737  4143 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:18:58.265838  4135 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:18:58.310729  4135 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:58.311097  4135 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:58.313969  3984 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:58.316637 32660 pending_rounds.cc:85] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 5
I20260709 12:18:58.316978 32660 pending_rounds.cc:107] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:58.317220 32660 raft_consensus.cc:2889] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:58.267860  4101 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 12:18:58.320029 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72703.4us
I20260709 12:18:58.335978 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:58.338617 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.332475  3779 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.353217 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:58.360219  4042 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:58.361011  4042 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.379283  3898 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:58.372695  4141 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 12:18:58.381008  3908 raft_consensus.cc:2764] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:18:58.391934 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.379901  3898 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.399991  3898 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:58.404194  4093 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:58.404728  4093 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:58.411984  4120 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.473s	user 0.003s	sys 0.000s
I20260709 12:18:58.415692 31972 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:58.417941 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80595.2us
W20260709 12:18:58.433092 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 82254us
I20260709 12:18:58.266160  3681 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.455549 32121 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:58.346216  3991 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:58.508285 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89475.3us
I20260709 12:18:58.509194 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:58.517382 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73362.2us
W20260709 12:18:58.520628  4117 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:18:58.536044 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca0e0 after lost signal to thread 30823
I20260709 12:18:58.546463  4128 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.550522  4089 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:18:58.553751 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6f00 after lost signal to thread 31904
W20260709 12:18:58.560004 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080009c520 after lost signal to thread 31996
W20260709 12:18:58.560432 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6da0 after lost signal to thread 31997
W20260709 12:18:58.561760 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080033c780 after lost signal to thread 32071
W20260709 12:18:58.566407 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47775.8us
W20260709 12:18:58.562134 32142 debug-util.cc:398] Leaking SignalData structure 0x7b0800341060 after lost signal to thread 32072
W20260709 12:18:58.570158 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.568027 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08003ae3a0 after lost signal to thread 32145
W20260709 12:18:58.571439 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080020b8e0 after lost signal to thread 32146
W20260709 12:18:58.576393 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08000a16c0 after lost signal to thread 32684
W20260709 12:18:58.576798 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080012bf80 after lost signal to thread 32685
W20260709 12:18:58.585983 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.525727  3779 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:58.592525  4064 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:58.588932 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.589028 32658 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:18:58.591430 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:58.528110  3728 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:18:58.563195  4092 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:58.573783  4095 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:58.593216  4042 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.636585 32651 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:58.619837  4095 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:58.547281  4128 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.551246  4089 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.632746  3991 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:18:58.657239 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 148139us
W20260709 12:18:58.678215 31969 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.035s	sys 0.056s
I20260709 12:18:58.311957  4135 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.636364  4037 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:58.680819 31969 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.035s	sys 0.056s
W20260709 12:18:58.681028 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.405726  4093 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.296846  4080 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.648092  4095 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.623570  4092 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:18:58.646693  4137 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 12:18:58.708359  4097 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.801s	user 0.013s	sys 0.000s
W20260709 12:18:58.710433  4154 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:18:58.688486  4037 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.704704  4093 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:58.745539 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54306.3us
I20260709 12:18:58.760185 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:58.762843 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.763396 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.756860  4150 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:18:58.749325  4097 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:58.759673  3990 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:58.764413  3728 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.801172 32119 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:58.804136  3995 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
I20260709 12:18:58.756170  3991 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:58.809764 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 115999us
I20260709 12:18:58.796123  3990 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.814811  3995 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.833266  3995 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:18:58.705474  4100 consensus_queue.cc:1175] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f05e4741d35c4cf3a31358c56e65b30c" responder_term: 14 status { last_received { term: 13 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:58.834751  4037 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:58.838148 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2127) took 2397 ms. Trace:
I20260709 12:18:58.838563 32040 rpcz_store.cc:276] 0709 12:18:56.440455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.450914 (+ 10459us) service_pool.cc:224] Handling call
0709 12:18:56.451345 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.650625 (+2199280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.650656 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.692149 (+ 41493us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.773358 (+ 81209us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:58.773393 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.773455 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:58.837851 (+ 64396us) raft_consensus.cc:1685] finished
0709 12:18:58.837870 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.837990 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1922927,"thread_start_us":336489,"threads_started":2,"wal-append.queue_time_us":104289}
W20260709 12:18:58.845852  4040 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.003s
I20260709 12:18:58.846354  4040 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.803820  3728 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:58.854341  3765 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.001s
I20260709 12:18:58.853083  4040 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:58.857549  3728 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:58.850767  4128 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:58.863399 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.864156 32084 leader_election.cc:336] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.827s (SENT)
W20260709 12:18:58.866438  4142 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.606s	user 0.003s	sys 0.001s
I20260709 12:18:58.853334  4089 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:58.870132 32084 leader_election.cc:336] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.892s (SENT)
W20260709 12:18:58.879591 32084 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.950s (SENT)
I20260709 12:18:58.880008 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:58.881121 32084 leader_election.cc:336] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.964s (SENT)
I20260709 12:18:58.881500 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:58.882682  3765 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:58.859721  3495 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.006s
I20260709 12:18:58.883328 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:58.883776  3495 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.883348  3765 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:58.870060  4095 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:58.887633  4101 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:58.888381  4101 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:58.888695  4101 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:58.889480  4101 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.891774  4101 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.895431  4143 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:58.895856  4143 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:18:58.896101  4095 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:58.896772  4143 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.897923  4143 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:58.895603  3495 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:58.899621  4147 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.381s	user 0.002s	sys 0.000s
W20260709 12:18:58.905944 32084 leader_election.cc:336] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.033s (SENT)
I20260709 12:18:58.906466 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:58.906888  4126 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:58.907331  4126 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:18:58.907514 32084 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.065s (SENT)
I20260709 12:18:58.908027 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:58.908257  4126 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.909488  4126 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.911231  3765 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:58.918670 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.919570  4134 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:18:58.919761 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.920168  4134 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:58.920833 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.921234 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.923233 31870 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 8 to 9, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 9 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:58.927280 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.928642 32651 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 12:18:58.930891 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.930372  4134 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:58.931341 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:58.932080 32113 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:18:58.925037 31870 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 5 to 7, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 7 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:58.937402  3978 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:58.938408  3978 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:58.944794  4152 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.377s	user 0.001s	sys 0.001s
W20260709 12:18:58.956344 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 201423us
I20260709 12:18:58.961716 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:58.973358 32084 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.837s (SENT)
I20260709 12:18:58.976572 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:58.975660 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:58.981467 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:58.982726 32120 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:58.983485 32116 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:58.987665 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:18:58.990126 32112 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 7: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:18:58.992455 32084 leader_election.cc:336] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.165s (SENT)
I20260709 12:18:58.993050 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:58.998634  4037 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.009495 32084 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.167s (SENT)
W20260709 12:18:59.010866 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 96640.9us
I20260709 12:18:59.012722 31870 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 8 to 12, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 12 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.013689  4042 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:59.015275  4153 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.448s	user 0.003s	sys 0.000s
I20260709 12:18:59.014986 31870 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 9 to 10, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 10 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.016959 31870 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:18:59.018579  3986 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.019145  3986 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:18:59.019630 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08003962e0 after lost signal to thread 3926
W20260709 12:18:59.020125 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080022adc0 after lost signal to thread 3993
W20260709 12:18:59.020576 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080032c5a0 after lost signal to thread 4055
W20260709 12:18:59.020828 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08003502a0 after lost signal to thread 4056
W20260709 12:18:59.021075 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080023e4e0 after lost signal to thread 4059
I20260709 12:18:59.025580  3986 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:59.027501  3926 raft_consensus.cc:2764] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:18:59.029597  3926 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.031152  4056 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:18:59.031675  4056 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.002755  4148 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:59.030953  4059 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:59.035499  4059 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:59.035887  4059 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:18:59.034181  4189 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:18:59.001224  3991 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:58.999172  4037 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.034071  4056 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:59.044219 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.046370 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:59.052635 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:59.045202 31967 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:59.061231 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.061978 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:59.042510 32084 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.285s (SENT)
I20260709 12:18:59.064826 31964 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:59.069730 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.070446 32043 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:18:59.072466 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.075306 32039 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:59.075474 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.076010 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.076297 32040 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:18:59.076622 32042 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:18:59.075958  4146 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.077461 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.077281  4146 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:59.077941 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.078892 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.079147 31965 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
W20260709 12:18:59.079464 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.079118 32039 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:59.067965 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:59.077536 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 832404us
I20260709 12:18:59.092381 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:59.096957 32623 leader_election.cc:336] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.784s (SENT)
I20260709 12:18:59.030102  3926 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.106189 31969 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:18:59.098237 32043 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:59.107221 32038 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:59.104611  4140 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:59.104657 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.109042 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:59.110014 31970 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:59.110229 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.103911  4067 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:59.112536 32040 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:59.113235  3990 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:59.113626 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.114140 32040 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.115028  3926 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:59.036749  4059 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.119082  4064 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:59.119634  4059 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.120581  4160 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.120954 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2128) took 2630 ms. Trace:
I20260709 12:18:59.120698 32039 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.122237 32114 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:59.121333 32046 rpcz_store.cc:276] 0709 12:18:56.490851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.560477 (+ 69626us) service_pool.cc:224] Handling call
0709 12:18:56.561001 (+   524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:58.644258 (+2083257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.644302 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:58.681047 (+ 36745us) log.cc:844] Serialized 52 byte log entry
0709 12:18:58.748840 (+ 67793us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:58.781925 (+ 33085us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.120651 (+338726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.120722 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.120762 (+    40us) raft_consensus.cc:1685] finished
0709 12:18:59.120776 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.120902 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2155209,"prepare.queue_time_us":93876,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":2473606,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":66,"thread_start_us":334400,"threads_started":3,"wal-append.queue_time_us":160064}
I20260709 12:18:59.122608  4161 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.123145  4136 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:59.124344  3991 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.123450  4161 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.124816  3991 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.126866  3991 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:59.127108  4146 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:59.128005 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.128871 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:18:59.123364 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.112339 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.130863 32041 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.121094  4160 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.137383 32042 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:59.139144 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.139606 32040 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:18:59.140161 32043 raft_consensus.cc:2376] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11. Re-sending same reply.
I20260709 12:18:59.143409 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:18:59.111511  4145 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:18:59.098515  3898 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:59.146659 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:59.147065 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.148598 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:59.151465 31970 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:59.151901 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.152504 32038 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:18:59.163164 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:59.164054 32653 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:18:59.169209 32041 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:18:59.169467 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.130041  3908 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:59.172916 32039 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:59.176757 32040 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:59.177126 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:59.177289 32040 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:59.142964  3924 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:59.179085 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:18:59.181751 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.152490  4140 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:59.180863 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:18:59.190131 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.184036 32040 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.181881 31965 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:59.198931 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:18:59.199579 31968 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:59.215880 32039 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:59.221697 31970 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:59.183063  4140 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:59.224486 32040 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.237131 32040 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:18:59.237461 32040 pending_rounds.cc:85] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 6
I20260709 12:18:59.237663 32040 pending_rounds.cc:107] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:18:59.237905 32040 raft_consensus.cc:2889] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.238554 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:59.201993  3926 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:59.239846 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 101846us
I20260709 12:18:59.242794 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:59.211789  4036 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:59.201488  3978 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:59.250869  4155 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:59.250662  3978 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.251294  4080 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.252754  3978 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:18:59.255229 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:18:59.206807  4100 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:59.259477  4040 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:18:59.260831  4121 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.120s
I20260709 12:18:59.210851  4161 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:59.262926  4193 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:59.265331  3935 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:59.268474 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:59.240195  4140 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:59.276163  4128 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:59.276707  4128 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:59.277459  4037 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:59.277786  4036 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:59.278757  4036 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 11.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.180183  3908 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.280977  4155 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:59.281383  3908 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:59.281903  4155 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.244947 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:59.203724  3924 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:59.290403 32084 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 3.318s (SENT)
I20260709 12:18:59.290875 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:59.292774 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.293387 32038 raft_consensus.cc:2376] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12. Re-sending same reply.
W20260709 12:18:59.294564 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.282608 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.303774 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61068us
I20260709 12:18:59.304842  4135 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.309820 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:59.311628 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.312316 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.313522 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.314110 32046 raft_consensus.cc:2376] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14. Re-sending same reply.
I20260709 12:18:59.277539  4128 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:59.315901 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71008.9us
I20260709 12:18:59.318707 31968 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:59.317085 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.318465 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.320854 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.321537 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.321792 32043 raft_consensus.cc:2368] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
I20260709 12:18:59.323278 32040 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:59.323264 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.324254 32038 raft_consensus.cc:2376] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12. Re-sending same reply.
I20260709 12:18:59.320559  3990 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:59.327845 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:59.328672 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:59.329499  3782 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:59.329582 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.330157  3782 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:59.330672 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:59.330499  3986 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:59.331121  3782 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.332449  3986 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:59.333079  3986 consensus_queue.cc:1243] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:59.319187  3984 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:59.334784  3984 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.335165  3984 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.336897  3986 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:59.337320  3986 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:59.337692  3986 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.334190  3944 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:59.338586  3924 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:59.338676  3944 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:18:59.339162  3924 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:59.339581 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 101475us
I20260709 12:18:59.339871  3924 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:59.340274  3924 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:59.339984  3944 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.340670  3924 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:59.341372 32084 leader_election.cc:336] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.495s (SENT)
W20260709 12:18:59.343146 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.343554 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.347210 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:59.348543 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.355489 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:18:59.360893 32040 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:59.382887 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:18:59.383744 31965 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
W20260709 12:18:59.385419 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.386629 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:18:59.390283 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:18:59.391595  4095 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:59.391979 31964 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:59.392076 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2160) took 1576 ms. Trace:
I20260709 12:18:59.392393  4093 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:59.377560  4148 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:18:59.392547 32047 rpcz_store.cc:276] 0709 12:18:57.815645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.831709 (+ 16064us) service_pool.cc:224] Handling call
0709 12:18:57.865885 (+ 34176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.058558 (+1192673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.058593 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.059057 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.196799 (+137742us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:59.196837 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.196903 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.391842 (+194939us) raft_consensus.cc:1685] finished
0709 12:18:59.391861 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.391991 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1180428,"raft.queue_time_us":1200855,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":16416,"thread_start_us":137416,"threads_started":1,"wal-append.queue_time_us":327098}
I20260709 12:18:59.394685 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.395000  4148 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:59.395496 32119 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:59.384586  4137 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:59.396165  4137 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:18:59.395442  4148 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:59.397367  4148 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.396545  4137 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.398088 32112 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:59.401558 32116 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:59.402791 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.403028 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.403620 32119 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 16 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 5.
W20260709 12:18:59.404445 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49081.2us
I20260709 12:18:59.405198 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.405277 32656 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 7.
I20260709 12:18:59.362870  4121 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:59.363749  4042 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:59.418182 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:18:59.419962 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.421533 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.424014 32118 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:18:59.426170 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:59.427448 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:59.428465 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:59.430845 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.431454 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.431746 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.432046 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.434537 32040 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:59.407857  4148 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.422709  4160 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:18:59.445644 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45556.6us
I20260709 12:18:59.446285  4037 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:59.446686  4195 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:18:59.448374 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.466635 32047 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:59.449120 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.467636 32039 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:59.449867 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.451025 32046 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 12:18:59.468420 32045 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:59.451748 32042 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:59.405575  4089 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:18:59.419636  4093 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:59.440627  4097 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:59.457562  3924 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.415216  4042 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:18:59.464406 31970 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:59.470836 32118 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.472460 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:18:59.482932 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:18:59.474267 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.459707  3765 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:59.475440 32041 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.477807 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.479305 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.488909 32047 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:59.440148  3990 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:59.486095 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.490072 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:59.493721 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.497400 31870 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 6 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 10 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.501649 32040 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:18:59.503199 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.505762 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.512130 31870 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 7 to 9, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 9 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:59.504283  4130 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.172s
I20260709 12:18:59.517903 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.518216 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.518960 32047 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:59.519943 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.520664 32046 raft_consensus.cc:2368] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15.
I20260709 12:18:59.521375 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.521998 32046 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:59.481163  4097 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.522187 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.514287 31870 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 10 to 11, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 11 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.476094  4093 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:18:59.484874  4042 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.523990 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.527542 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.490620  3978 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:59.531976 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.536954 32046 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:59.532116 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.532982 32040 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:18:59.532887 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.538689 32045 raft_consensus.cc:2368] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11.
I20260709 12:18:59.488129  3765 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:59.495075  4095 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:59.571269 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.570750 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.575318 32042 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:18:59.534210 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.568912 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.576422 32047 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:18:59.539085 32040 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:18:59.503058  4121 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.564913 31870 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 6 to 9, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 9 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:59.581931 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.528327  4097 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:59.590950 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.585757 32047 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:59.592500 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.589706 31870 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 8 to 11, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 11 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.585337 32046 pending_rounds.cc:85] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 4
I20260709 12:18:59.489643  4037 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:18:59.593547 32046 pending_rounds.cc:107] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 10.5
W20260709 12:18:59.592823 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 110873us
I20260709 12:18:59.593961 32046 raft_consensus.cc:2889] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.593344 32118 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:18:59.587152 32040 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:59.599140 32042 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:59.600188 32119 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:18:59.541648  3978 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:18:59.561395  3779 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.570271  3765 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:59.589072  3924 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:18:59.601995 32045 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:18:59.681792 32045 pending_rounds.cc:85] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 6
I20260709 12:18:59.682130 32045 pending_rounds.cc:107] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:18:59.682441 32045 raft_consensus.cc:2889] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.604135 32041 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:18:59.683720 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.610975 32039 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:59.684677 31870 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 10 to 11, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 11 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:59.610548 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:18:59.593564  4195 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:18:59.689324 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:18:59.715763 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.717705 32045 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:59.718242 32045 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:59.719761 32042 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:59.610538  4064 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.742492 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 325639us
I20260709 12:18:59.675630  3978 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:59.745143 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 60646.2us
W20260709 12:18:59.752015 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.744203 32045 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.590991  4121 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:18:59.782483 31869 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 230) took 16953 ms (client timeout 15000 ms). Trace:
I20260709 12:18:59.785725 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.786914 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:18:59.786629  3984 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.788264 32118 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:59.789474  3984 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:59.793607 32041 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:59.798367 32043 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.785774 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.779072 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:18:59.826798 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 47853.6us
I20260709 12:18:59.711894  4093 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.832304 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.835642 32041 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:18:59.682195  3779 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.851869 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.852706 32042 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:18:59.853750 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.854501 32039 raft_consensus.cc:2376] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14. Re-sending same reply.
I20260709 12:18:59.859366 31870 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 7 to 13, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 13 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:59.862200 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.863004 32041 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:18:59.866657 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 461494us
W20260709 12:18:59.867792 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 142676us
W20260709 12:18:59.868405 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.875881 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.876444 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.881990 32623 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.482s (SENT)
W20260709 12:18:59.887075 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.888007 32084 leader_election.cc:336] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.523s (SENT)
W20260709 12:18:59.889261 32084 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.724s (SENT)
I20260709 12:18:59.833304  3926 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.891561 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.938751 32044 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:18:59.900775 31870 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:18:59.891594 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:18:59.776710  4130 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
W20260709 12:18:59.944921 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.945118 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:59.945420 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.945814 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.945736 32041 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:18:59.866158  3782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.001s	sys 0.019s
I20260709 12:18:59.894394 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.949458 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2235) took 1632 ms. Trace:
I20260709 12:18:59.894762 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:18:59.956759 31870 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 5 to 6, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 6 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:59.956070 32115 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.018s	sys 0.032s
W20260709 12:18:59.958552 32115 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.018s	sys 0.032s
W20260709 12:18:59.961905  3986 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.014s
I20260709 12:18:59.962476  3986 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:18:59.965232  3986 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:18:59.955515 32117 rpcz_store.cc:276] 0709 12:18:58.316633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.391187 (+ 74554us) service_pool.cc:224] Handling call
0709 12:18:58.391559 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.679685 (+288126us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:58.708334 (+ 28649us) log.cc:844] Serialized 14 byte log entry
0709 12:18:58.948836 (+240502us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b48002301d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:18:59.029770 (+ 80934us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18002ff2c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:18:59.183636 (+153866us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.196287 (+ 12651us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.196376 (+    89us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.203240 (+  6864us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.203382 (+   142us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:59.203415 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.203483 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.262019 (+ 58536us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18002ff2c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:59.949243 (+687224us) raft_consensus.cc:1685] finished
0709 12:18:59.949277 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.949398 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56033,"prepare.queue_time_us":27852,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":116,"raft.queue_time_us":148593,"raft.run_cpu_time_us":2969,"raft.run_wall_time_us":444401,"spinlock_wait_cycles":25214464,"thread_start_us":298348,"threads_started":2,"wal-append.queue_time_us":551253}
I20260709 12:18:59.972221 31870 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 8 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 10 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.835137  4136 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:59.979230 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.779142  4064 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:18:59.987665 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:18:59.989217 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 97723.4us
I20260709 12:18:59.997464 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:18:59.997630 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56054.4us
W20260709 12:19:00.001689 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.972242  4156 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:19:00.009938 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:00.017082 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:18:59.886039  3924 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 12:18:59.877827  4137 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:00.051292  4157 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:18:59.956377  3926 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.098165 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:00.099522 32041 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
W20260709 12:19:00.064677 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.016428  4128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.008s	sys 0.017s
W20260709 12:19:00.021915  3495 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.013s	sys 0.034s
I20260709 12:19:00.072419 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:00.073215 32042 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:00.074203 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:00.132453 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:00.016002  4126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.007s	sys 0.017s
I20260709 12:19:00.077966 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:00.147511 32041 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:00.148056 32041 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:18:59.970911  3378 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:18:59.873641  4093 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:00.078698  3990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.014s	sys 0.015s
W20260709 12:19:00.137240 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.004s	sys 0.041s
I20260709 12:19:00.059855  3986 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:00.137676 32038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.003s	sys 0.027s
W20260709 12:19:00.138020 32013 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.017s	sys 0.029s
W20260709 12:18:59.783054 31869 rpcz_store.cc:269] 0709 12:18:42.771290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.772507 (+  1217us) service_pool.cc:224] Handling call
0709 12:18:43.496003 (+723496us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1d7f3a 00007f2291f8de9c 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:18:59.724321 (+16228318us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:54.452843 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:54.481738 (+ 28895us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:54.481797 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:18:54.481843 (+    46us) tablet.cc:662] Decoding operations
0709 12:18:54.618057 (+136214us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:54.647925 (+ 29868us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:54.647969 (+    44us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:18:54.649777 (+  1808us) tablet.cc:701] Row locks acquired
0709 12:18:54.649835 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:18:54.650053 (+   218us) write_op.cc:270] Start()
0709 12:18:54.650208 (+   155us) write_op.cc:276] Timestamp: P: 1783599534649974 usec, L: 0
0709 12:18:54.650228 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:18:54.870878 (+220650us) log.cc:844] Serialized 7779 byte log entry
0709 12:18:55.848954 (+978076us) op_driver.cc:464] REPLICATION: finished
0709 12:18:55.856577 (+  7623us) write_op.cc:301] APPLY: starting
0709 12:18:55.857145 (+   568us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:55.871445 (+ 14300us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:55.871574 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:55.931202 (+ 59628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:55.946662 (+ 15460us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:18:55.955484 (+  8822us) write_op.cc:312] APPLY: finished
0709 12:18:56.015866 (+ 60382us) log.cc:844] Serialized 180 byte log entry
0709 12:18:56.026588 (+ 10722us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:56.026808 (+   220us) write_op.cc:454] Released schema lock
0709 12:18:56.026904 (+    96us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:56.045842 (+ 18938us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:56.045876 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":243747,"reported_tablets":51,"spinlock_wait_cycles":373635328,"child_traces":[["op",{"apply.queue_time_us":7319,"apply.run_cpu_time_us":31378,"apply.run_wall_time_us":189202,"cfile_cache_hit":78,"cfile_cache_hit_bytes":81432,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":2,"lbm_read_time_us":1383,"lbm_reads_lt_1ms":12,"num_ops":20,"prepare.queue_time_us":162389,"prepare.run_cpu_time_us":63808,"prepare.run_wall_time_us":1358441,"raft.queue_time_us":494338,"raft.run_cpu_time_us":6949,"raft.run_wall_time_us":8347,"replication_time_us":1198558,"thread_start_us":2542879,"threads_started":4,"wal-append.queue_time_us":479490,"wal-append.run_cpu_time_us":5205,"wal-append.run_wall_time_us":1668256}]]}
W20260709 12:19:00.089736  4100 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.014s	sys 0.016s
W20260709 12:19:00.109177 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.148918 31971 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.007s	sys 0.064s
W20260709 12:19:00.109328  4128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.009s	sys 0.017s
W20260709 12:19:00.161768 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51215.8us
W20260709 12:19:00.130503  3495 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.016s	sys 0.034s
W20260709 12:19:00.167092 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.004s	sys 0.041s
W20260709 12:19:00.173683 32038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.027s
W20260709 12:19:00.171063 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93206.4us
W20260709 12:19:00.174036 32013 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.017s	sys 0.029s
W20260709 12:19:00.176910 31971 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.007s	sys 0.065s
W20260709 12:19:00.177433 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.111531  4136 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:00.146692  4126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.007s	sys 0.019s
W20260709 12:19:00.187965 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 690386us
I20260709 12:19:00.187724 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:00.196696 32013 connection.cc:256] RPC call timeout handler was delayed by -1.5969092459999956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 12:19:00.166240  3990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.015s	sys 0.016s
I20260709 12:19:00.189059  3984 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:00.204730  3984 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:00.205902  3984 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.138841  4155 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:19:00.210350 32045 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:00.212220  4148 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.000s
I20260709 12:19:00.215978  3926 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:00.218132  4208 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:19:00.223443  4160 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.224218  4160 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:00.226727  4207 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
W20260709 12:19:00.178252  4203 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.449s	user 0.001s	sys 0.000s
I20260709 12:19:00.228116  4064 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.229156  3779 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.229233  4160 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:00.231963  4097 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 12:19:00.232632  4097 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:00.234771  4209 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:19:00.235455  4097 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:00.186129  4204 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:00.236932  4206 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:19:00.237200  4204 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.239760  4204 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.180666 32041 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:00.246183 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.246789 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.247298 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:00.170396  4095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.011s	sys 0.035s
W20260709 12:19:00.176741  4100 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.015s	sys 0.017s
I20260709 12:19:00.168274  3986 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.251260 32041 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.185845  3924 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.259451 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:00.231999  4080 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.980s	user 0.007s	sys 0.004s
W20260709 12:19:00.265933 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.243376  3944 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:00.271222 32046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.010s	sys 0.024s
I20260709 12:19:00.267302 32120 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:19:00.255105  4140 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.980s	user 0.012s	sys 0.001s
W20260709 12:19:00.252566  4095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.011s	sys 0.035s
I20260709 12:19:00.254364 32045 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:00.253510  4067 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:00.282855 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:00.201797  4130 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 12:19:00.275455 32046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.010s	sys 0.024s
I20260709 12:19:00.256054  3986 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:00.287531 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.013s	sys 0.018s
W20260709 12:19:00.300565 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.013s	sys 0.018s
W20260709 12:19:00.300897 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 518679us
I20260709 12:19:00.258512  3990 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:00.288049 32039 raft_consensus.cc:2376] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11. Re-sending same reply.
I20260709 12:19:00.273063  4143 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.284680 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:00.312546 32045 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:00.314801  4137 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.333823  4160 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:00.315505  3926 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.316751 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:00.317826  4211 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:00.323371  3782 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.327479  4121 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:00.329154 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:00.329502  4042 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.005s
I20260709 12:19:00.330799 31972 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:00.262552  4128 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
W20260709 12:19:00.332979 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55850.2us
I20260709 12:19:00.267860  3924 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:00.347815  3765 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 12:19:00.348595 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:00.348971  3990 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:00.351999 32113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.012s	sys 0.024s
I20260709 12:19:00.352348 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:00.365895 32045 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:00.366334 32114 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:00.263875  4037 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:19:00.374943 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:00.375129 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:00.356575 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:00.282793  4095 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:00.309288  4067 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:00.379125 32040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.008s	sys 0.019s
W20260709 12:19:00.379622 32040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.008s	sys 0.020s
I20260709 12:19:00.380154 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:00.384150 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:00.386708 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.362740 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.367193 32113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.013s	sys 0.024s
I20260709 12:19:00.376402 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:00.388216 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.388324 32117 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:00.375656 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:00.287468  4140 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.376911 32045 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:00.398362 32656 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:00.313432 32041 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:00.352291  4143 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.406809 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:00.377825  4137 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.358414  3926 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.393460  4130 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:00.360095  3782 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.482858  3782 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.391815  4222 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.484002  4222 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.360456  4211 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.433223 32114 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:00.361553  4136 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.436496  4140 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:00.436966  4201 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:00.437580  4067 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.438709 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:00.442368  4214 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:00.442713  3986 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.418473 32041 pending_rounds.cc:85] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 6
I20260709 12:19:00.568555 32658 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 12:19:00.568931 32041 pending_rounds.cc:107] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:19:00.569386 32041 raft_consensus.cc:2889] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:00.450155 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.453650 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2035) took 1005 ms. Trace:
W20260709 12:19:00.571703 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.570264 32038 rpcz_store.cc:276] 0709 12:18:59.448361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.460461 (+ 12100us) service_pool.cc:224] Handling call
0709 12:18:59.477293 (+ 16832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.490092 (+ 12799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.490126 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.505085 (+ 14959us) log.cc:844] Serialized 26 byte log entry
0709 12:19:00.186583 (+681498us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:00.253237 (+ 66654us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.253452 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.253528 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:00.320163 (+ 66635us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b18003c3220. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:00.321479 (+  1316us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b18003c3220. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:00.453385 (+131906us) raft_consensus.cc:1685] finished
0709 12:19:00.453418 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.453582 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35582,"prepare.queue_time_us":66907,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":6697,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":32991,"spinlock_wait_cycles":26942080,"thread_start_us":608191,"threads_started":1,"wal-append.queue_time_us":920363}
I20260709 12:19:00.457578  4095 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.463338  3924 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.468408  4143 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:00.470824  3926 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.471314  4137 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.586274  4137 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:00.473047  3978 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.003s
I20260709 12:19:00.587065  3978 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.589876  3978 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:00.372555  3765 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.591792  3765 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:00.403276  4040 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:19:00.370875  4042 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.597069  4042 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:00.393280  3779 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:00.597887  3779 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.600005  3779 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:00.478760 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54907.7us
W20260709 12:19:00.479703 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63688.4us
I20260709 12:19:00.602057 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:00.605291 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:00.489368 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:00.507795 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 123693us
I20260709 12:19:00.572853 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:00.606133 32656 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:19:00.606380 32658 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:00.608251 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:00.609615 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:00.609993 32119 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:00.610832 32121 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:00.620223 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.620782 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.633471 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2337) took 1032 ms. Trace:
W20260709 12:19:00.646862 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45377.3us
I20260709 12:19:00.634112 32119 rpcz_store.cc:276] 0709 12:18:59.601302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.604534 (+  3232us) service_pool.cc:224] Handling call
0709 12:18:59.604726 (+   192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.610373 (+1005647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.633085 (+ 22712us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540013a954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:00.633379 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2311444224}
I20260709 12:19:00.569870  4155 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:00.653995 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48790.1us
W20260709 12:19:00.654028 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.653854  4130 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:19:00.657919  4211 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:00.647821  4042 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:00.653345 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2326) took 1187 ms. Trace:
I20260709 12:19:00.652982  4036 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.658486  3926 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:00.664333 32121 rpcz_store.cc:276] 0709 12:18:59.465952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.467555 (+  1603us) service_pool.cc:224] Handling call
0709 12:18:59.467827 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.632422 (+1164595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.648729 (+ 16307us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400138654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:00.653286 (+  4557us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2628102912}
I20260709 12:19:00.670770 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:00.646991  4064 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:19:00.658900  4222 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:00.676635 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.676965 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:00.678334 32047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.023s	sys 0.034s
W20260709 12:19:00.678730 32047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.023s	sys 0.034s
W20260709 12:19:00.672437 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.017s	sys 0.010s
I20260709 12:19:00.665043  4042 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.692464 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:00.693389 32119 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:00.693727  4037 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:00.694407 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:00.695042 32656 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:00.695878 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.653241  3924 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:19:00.655454  4204 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:19:00.694904  4042 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.769843 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:00.770730 32120 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
W20260709 12:19:00.707139 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.705992 32112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.021s	sys 0.025s
W20260709 12:19:00.773080 32112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.022s	sys 0.025s
W20260709 12:19:00.709920 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.736307 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.774030 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.751061 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:00.775887 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:00.776118 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:00.752198 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:00.676496  3926 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:00.754288  4130 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:00.787462 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.787393  4130 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.754374  4137 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:00.755203  4064 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:00.760174  3924 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:00.694149  4037 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.676476  4155 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:00.783766 32119 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:00.847404 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.011s	sys 0.020s
W20260709 12:19:00.847782 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.011s	sys 0.020s
I20260709 12:19:00.786613  3926 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.789507  3782 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:00.796623 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.861033 31937 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.018s	sys 0.023s
W20260709 12:19:00.862398 31937 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.019s	sys 0.023s
I20260709 12:19:00.827644 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2255) took 1431 ms. Trace:
I20260709 12:19:00.794759 32120 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:19:00.685303 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.018s	sys 0.010s
W20260709 12:19:00.827913 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.018s	sys 0.011s
I20260709 12:19:00.799634  4204 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:00.874722 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.020s	sys 0.011s
W20260709 12:19:00.875301 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.875958 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.865876 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2252) took 1470 ms. Trace:
W20260709 12:19:00.861411 31971 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.010s	sys 0.020s
I20260709 12:19:00.848527  4037 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.854025  3782 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:00.828047  4064 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:00.879691  3924 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:00.918941 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.004163 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.940166 31971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.011s	sys 0.020s
I20260709 12:19:00.824143  4137 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:00.971805  4222 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.972309  3935 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:00.978374 32656 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:00.979249  4234 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.983276  3765 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:00.987072  4233 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:19:00.990639 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 45725.9us
I20260709 12:19:00.994570  4240 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:01.002449  3782 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:01.005852 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 506) took 1655 ms. Trace:
I20260709 12:19:01.019253  3978 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:01.017114 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.019073  4100 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.932539 32115 rpcz_store.cc:276] 0709 12:18:59.395516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.399928 (+  4412us) service_pool.cc:224] Handling call
0709 12:18:59.417004 (+ 17076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.417267 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:59.958952 (+541685us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.865575 (+906623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.865621 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:00.865664 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.865736 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.865830 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":102563,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":79,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":10553,"thread_start_us":1440384,"threads_started":2}
I20260709 12:19:01.035265 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:01.001660 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.111s	user 0.015s	sys 0.020s
I20260709 12:19:01.019644  4201 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:00.899427  4204 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:01.040573 31971 rpcz_store.cc:276] 0709 12:18:59.350091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.350232 (+   141us) service_pool.cc:224] Handling call
0709 12:18:59.350609 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.350842 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:00.188448 (+837606us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.005503 (+817055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.005558 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:01.005602 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.005669 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.005796 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68434,"prepare.queue_time_us":96310,"prepare.run_cpu_time_us":21,"prepare.run_wall_time_us":21,"raft.queue_time_us":79,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":5085,"thread_start_us":1585362,"threads_started":2}
I20260709 12:19:01.044502 32656 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 7.
I20260709 12:19:01.063951 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:01.065073 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.175s	user 0.017s	sys 0.020s
W20260709 12:19:01.065820 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.066497 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.059100 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.004s	sys 0.034s
W20260709 12:19:01.067920 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.004s	sys 0.034s
W20260709 12:19:01.068359 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 845328us
I20260709 12:19:01.081634 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:01.083738 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:01.085675 31965 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:01.089519 31965 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:01.029824  4130 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:01.095422 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:01.109575 31968 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:01.111470 31971 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:01.097749 32625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.096s	user 0.009s	sys 0.022s
W20260709 12:19:01.112517 32625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.009s	sys 0.022s
I20260709 12:19:01.033120  4235 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:01.114102 32652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.006s	sys 0.014s
W20260709 12:19:01.127624 32652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.007s	sys 0.014s
I20260709 12:19:01.123189 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:01.110635  3991 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.003s	sys 0.020s
I20260709 12:19:01.124542 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:01.039206  4097 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:01.116117 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:01.131685 31972 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:01.128573 32044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.011s	sys 0.007s
I20260709 12:19:01.128960 32660 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:01.031477  3944 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:19:01.140076 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:01.131024 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 845) took 1302 ms. Trace:
I20260709 12:19:01.029168  4222 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:01.026988  3924 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:01.128461  3991 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.003s	sys 0.020s
W20260709 12:19:01.097698  4210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.011s	sys 0.018s
W20260709 12:19:01.144730 32623 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.012s	sys 0.016s
W20260709 12:19:01.152017 32623 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.012s	sys 0.016s
W20260709 12:19:01.152390 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 871209us
W20260709 12:19:01.154243 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.140086 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:01.129678 31970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.011s	sys 0.012s
I20260709 12:19:01.142603 32652 rpcz_store.cc:276] 0709 12:18:59.828425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.828632 (+   207us) service_pool.cc:224] Handling call
0709 12:18:59.829393 (+   761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.829666 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:00.363184 (+533518us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.130596 (+767412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.130668 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:01.130735 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.130816 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.130953 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2417,"prepare.queue_time_us":1024362,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"thread_start_us":1260384,"threads_started":2}
W20260709 12:19:01.132596  4126 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.021s	sys 0.020s
I20260709 12:19:01.141119 32653 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:01.089680  4232 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:01.033995  4234 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:01.039161  3765 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:01.156919 31970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.012s	sys 0.012s
I20260709 12:19:01.059126  3978 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:01.088346  4100 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:01.171703 32044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.011s	sys 0.007s
I20260709 12:19:01.112036  4201 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:01.028347  4155 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:01.156270  4126 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.022s	sys 0.020s
W20260709 12:19:01.156373  3495 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.003s	sys 0.040s
I20260709 12:19:01.043807  4240 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:01.157697  4210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.012s	sys 0.018s
W20260709 12:19:01.178242 32042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.006s	sys 0.014s
W20260709 12:19:01.182495 31870 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.012s	sys 0.022s
W20260709 12:19:01.183712 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.184914 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2053) took 1665 ms. Trace:
I20260709 12:19:01.159857  4232 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:01.189618 32042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.006s	sys 0.014s
W20260709 12:19:01.190677 32117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.005s	sys 0.031s
I20260709 12:19:01.189131 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:01.191283  3984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.000s	sys 0.028s
W20260709 12:19:01.191747 32117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.005s	sys 0.031s
I20260709 12:19:01.190042 32046 rpcz_store.cc:276] 0709 12:18:59.519824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.537568 (+ 17744us) service_pool.cc:224] Handling call
0709 12:18:59.585080 (+ 47512us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:59.596488 (+ 11408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.596534 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:59.597089 (+   555us) log.cc:844] Serialized 52 byte log entry
0709 12:19:00.294773 (+697684us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:00.294814 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.294883 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.184651 (+889768us) raft_consensus.cc:1685] finished
0709 12:19:01.184676 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.184840 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59531,"raft.queue_time_us":1198,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":81401,"thread_start_us":697268,"threads_started":1,"wal-append.queue_time_us":1105924}
W20260709 12:19:01.193583 31870 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.013s	sys 0.022s
I20260709 12:19:01.142356  4097 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:01.176968  3990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.024s	sys 0.074s
I20260709 12:19:01.194536 31972 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:19:01.205004 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.198421 32113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.019s	sys 0.006s
I20260709 12:19:01.199659 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:01.206553 31971 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:01.200891 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:01.208321 32113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.019s	sys 0.006s
W20260709 12:19:01.185356 32118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.003s	sys 0.017s
W20260709 12:19:01.188830  3495 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.003s	sys 0.041s
W20260709 12:19:01.209483 32118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.003s	sys 0.017s
I20260709 12:19:01.131727  4235 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:01.168187  3991 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 12:19:01.225461 32043 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.019s	sys 0.094s
I20260709 12:19:01.215548 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:01.237174 32043 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.019s	sys 0.095s
I20260709 12:19:01.237457 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:01.212644 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.238241 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.191778  3984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.000s	sys 0.028s
W20260709 12:19:01.236387 32045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.016s	sys 0.023s
I20260709 12:19:01.190259  3986 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:01.203387  3990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.025s	sys 0.074s
W20260709 12:19:01.256846 32045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.016s	sys 0.024s
I20260709 12:19:01.258585 32116 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.195114  4232 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:01.264633 32038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.004s	sys 0.021s
W20260709 12:19:01.243547  4095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.004s	sys 0.018s
W20260709 12:19:01.267341 32038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.005s	sys 0.021s
W20260709 12:19:01.253984  4242 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.342s	user 0.001s	sys 0.000s
W20260709 12:19:01.275004  4236 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:19:01.275377  4238 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 12:19:01.275962  3779 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.279182 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:01.410185 32046 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.863682 32112 rpcz_store.cc:276] 0709 12:18:59.396385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.421099 (+ 24714us) service_pool.cc:224] Handling call
0709 12:18:59.439807 (+ 18708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.440024 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:59.846474 (+406450us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.781957 (+935483us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.827144 (+ 45187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.827218 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:00.827307 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.827402 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.827567 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50892,"prepare.queue_time_us":88886,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":101,"raft.queue_time_us":469513,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":15851,"thread_start_us":1307204,"threads_started":2,"wal-append.queue_time_us":41233}
W20260709 12:19:01.266706  4243 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:19:01.287953 32039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.000s	sys 0.025s
I20260709 12:19:01.290468 32652 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:19:01.414067 32039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.000s	sys 0.025s
W20260709 12:19:01.345264 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.020s	sys 0.012s
W20260709 12:19:01.367268 32041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.024s	sys 0.012s
W20260709 12:19:01.281208  4241 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
W20260709 12:19:01.391435 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.408428 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.282068  4064 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:01.367610  4067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.003s	sys 0.034s
W20260709 12:19:01.415088 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.020s	sys 0.013s
W20260709 12:19:01.415129 32041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.024s	sys 0.012s
W20260709 12:19:01.427798 32047 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.024s	sys 0.012s
W20260709 12:19:01.443610 32047 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.024s	sys 0.012s
I20260709 12:19:01.445971  4204 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:01.441190  4067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.003s	sys 0.034s
W20260709 12:19:01.379176  4095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.005s	sys 0.019s
W20260709 12:19:01.473080 32659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.012s	sys 0.011s
W20260709 12:19:01.473793 32659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.012s	sys 0.011s
W20260709 12:19:01.473017 32114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.002s	sys 0.030s
W20260709 12:19:01.475696 32114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.002s	sys 0.030s
W20260709 12:19:01.472949 32658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.011s	sys 0.011s
W20260709 12:19:01.477939 32658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.012s	sys 0.011s
W20260709 12:19:01.482755  4121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.004s	sys 0.008s
W20260709 12:19:01.483865  4121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.004s	sys 0.009s
W20260709 12:19:01.486045 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 346148us
I20260709 12:19:01.486907 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:01.487782 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 298693us
I20260709 12:19:01.492563 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:01.494030 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.493705 32652 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 12:19:01.503053 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.505798 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2201) took 1242 ms. Trace:
W20260709 12:19:01.492511 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.493180 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 473336us
W20260709 12:19:01.531184 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44323.9us
I20260709 12:19:01.533818 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:01.545141 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2258) took 1231 ms. Trace:
I20260709 12:19:01.508955 32039 rpcz_store.cc:276] 0709 12:19:00.262667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.410353 (+147686us) service_pool.cc:224] Handling call
0709 12:19:00.425919 (+ 15566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.426135 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:00.555077 (+128942us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.502383 (+947306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.502911 (+   528us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.503364 (+   453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.504424 (+  1060us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.505410 (+   986us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":141910,"prepare.run_cpu_time_us":2278,"prepare.run_wall_time_us":10687,"thread_start_us":946322,"threads_started":2}
W20260709 12:19:01.486712 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.571957 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.574369 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.592608 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 931979us
W20260709 12:19:01.606575 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.639410 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.662750 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.670712 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.573779  4136 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.010s	sys 0.013s
I20260709 12:19:01.694459 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2065) took 2108 ms. Trace:
I20260709 12:19:01.714615 32040 rpcz_store.cc:276] 0709 12:18:59.591480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.595921 (+  4441us) service_pool.cc:224] Handling call
0709 12:18:59.603356 (+  7435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.684099 (+ 80743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.716866 (+ 32767us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.775915 (+ 59049us) log.cc:844] Serialized 26 byte log entry
0709 12:19:00.380004 (+604089us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:00.380048 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.380118 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.694212 (+1314094us) raft_consensus.cc:1685] finished
0709 12:19:01.694233 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.694399 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54110,"raft.queue_time_us":81522,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":60647,"thread_start_us":510194,"threads_started":1,"wal-append.queue_time_us":1022434}
W20260709 12:19:01.694365 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 454301us
I20260709 12:19:01.545996 32118 rpcz_store.cc:276] 0709 12:19:00.334791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.334917 (+   126us) service_pool.cc:224] Handling call
0709 12:19:00.335201 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.335388 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:00.346223 (+ 10835us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.544682 (+1198459us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.544749 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:01.544803 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.544901 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.545065 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10693,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":45,"raft.queue_time_us":221158,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":6075,"spinlock_wait_cycles":16256,"thread_start_us":1197995,"threads_started":2}
I20260709 12:19:01.459414  4143 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:01.439487  3779 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:01.472949  4222 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:01.734519 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:01.452869  4130 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
I20260709 12:19:01.739166 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:01.581120  4241 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:19:01.711308  4136 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.010s	sys 0.013s
I20260709 12:19:01.597769  3924 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:01.577096  4160 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:01.792265 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:01.795385 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2196) took 1537 ms. Trace:
I20260709 12:19:01.799328  4126 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:01.799563  4255 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 12:19:01.801043  4258 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:19:01.804534 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2200) took 1542 ms. Trace:
W20260709 12:19:01.806103  4236 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:19:01.778818 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:01.806974  4260 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.358s	user 0.000s	sys 0.000s
W20260709 12:19:01.807757  4245 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.358s	user 0.000s	sys 0.000s
I20260709 12:19:01.819216 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2218) took 1197 ms. Trace:
I20260709 12:19:01.888711 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 854) took 1198 ms. Trace:
I20260709 12:19:01.889048 32039 rpcz_store.cc:276] 0709 12:19:00.621600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.588779 (+967179us) service_pool.cc:224] Handling call
0709 12:19:01.819148 (+230369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.864594 32041 rpcz_store.cc:276] 0709 12:19:00.262343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.403761 (+141418us) service_pool.cc:224] Handling call
0709 12:19:00.418230 (+ 14469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:00.653454 (+235224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.653482 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.654408 (+   926us) log.cc:844] Serialized 52 byte log entry
0709 12:19:01.441509 (+787101us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:01.462183 (+ 20674us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.465482 (+  3299us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.544060 (+ 78578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.545784 (+  1724us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.804276 (+258492us) raft_consensus.cc:1685] finished
0709 12:19:01.804303 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.804462 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34405,"prepare.queue_time_us":295099,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":25027,"raft.queue_time_us":138046,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":838542,"threads_started":2,"wal-append.queue_time_us":1137648}
W20260709 12:19:01.761173  3990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.018s	sys 0.025s
I20260709 12:19:01.827293  4100 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:01.693548  3765 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:01.711898  4137 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:01.711838  4234 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:01.845678 32042 rpcz_store.cc:276] 0709 12:19:00.257799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.259008 (+  1209us) service_pool.cc:224] Handling call
0709 12:19:00.271920 (+ 12912us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:00.331739 (+ 59819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.331769 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.332406 (+   637us) log.cc:844] Serialized 52 byte log entry
0709 12:19:01.190907 (+858501us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.237657 (+ 46750us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.549043 (+311386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.549144 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.795175 (+246031us) raft_consensus.cc:1685] finished
0709 12:19:01.795194 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.795329 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46462,"prepare.queue_time_us":60350,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2489,"raft.queue_time_us":48418,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":1169074,"threads_started":2,"wal-append.queue_time_us":1460603}
W20260709 12:19:01.870479 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.921510 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:01.922425 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.865221 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 331695us
I20260709 12:19:01.867096 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2222) took 1197 ms. Trace:
I20260709 12:19:01.923125 32040 rpcz_store.cc:276] 0709 12:19:00.670230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.769226 (+1098996us) service_pool.cc:224] Handling call
0709 12:19:01.867037 (+ 97811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.923441 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:01.784193  3978 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:01.881731 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:01.884809 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.863476  4296 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:01.862790  4297 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:01.891908 31971 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.685s	user 0.011s	sys 0.000s
I20260709 12:19:01.947598 31971 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
W20260709 12:19:01.893463 31970 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.008s	sys 0.042s
W20260709 12:19:01.948803 31970 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.009s	sys 0.042s
W20260709 12:19:01.834381 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.774020  4248 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:01.889149 32658 rpcz_store.cc:276] 0709 12:19:00.690267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.690847 (+   580us) service_pool.cc:224] Handling call
0709 12:19:00.691231 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.691630 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.691669 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.703107 (+ 11438us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.478427 (+775320us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:01.478472 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.478553 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.888444 (+409891us) raft_consensus.cc:1685] finished
0709 12:19:01.888467 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.888646 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10410,"raft.queue_time_us":637,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":55016,"thread_start_us":719595,"threads_started":1,"wal-append.queue_time_us":1102341}
I20260709 12:19:01.827533  4097 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:01.883394  3495 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.024s	sys 0.028s
I20260709 12:19:01.924072 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:01.771001  4143 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:01.929488 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:01.898989  3990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.018s	sys 0.026s
W20260709 12:19:01.982650 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 59302.9us
I20260709 12:19:01.862350  4284 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:01.914676  4279 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:19:01.943902 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62245.2us
W20260709 12:19:01.954767  4155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.001s	sys 0.010s
I20260709 12:19:01.946141 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:01.889003  4136 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:01.899012  4059 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:01.969421 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.982808  3495 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.024s	sys 0.028s
I20260709 12:19:01.956708 32040 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:01.957319 32011 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:01.957657 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2064) took 2378 ms. Trace:
I20260709 12:19:01.956688  3978 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.957888 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:01.965648 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:02.076649 32120 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:01.954962 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 248087us
I20260709 12:19:01.984153 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:01.983582 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.009s	sys 0.001s
W20260709 12:19:02.077406 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.009s	sys 0.002s
I20260709 12:19:01.983659 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:01.985821 32113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.003s	sys 0.006s
W20260709 12:19:02.080240 32113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.003s	sys 0.007s
I20260709 12:19:02.080989 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2266) took 1726 ms. Trace:
W20260709 12:19:02.083415 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.988623 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2328) took 2488 ms. Trace:
W20260709 12:19:02.015906 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.085335 32120 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:19:02.085814 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.081395 32113 rpcz_store.cc:276] 0709 12:19:00.354702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.423109 (+ 68407us) service_pool.cc:224] Handling call
0709 12:19:00.423491 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:00.426155 (+  2664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.426207 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.430911 (+  4704us) log.cc:844] Serialized 52 byte log entry
0709 12:19:01.208935 (+778024us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:01.242909 (+ 33974us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.080626 (+837717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.080711 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.080768 (+    57us) raft_consensus.cc:1685] finished
0709 12:19:02.080787 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.080926 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119558,"prepare.queue_time_us":33545,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2796,"raft.queue_time_us":221706,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":120022,"thread_start_us":1614853,"threads_started":2,"wal-append.queue_time_us":1151379}
I20260709 12:19:02.087445 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2089) took 1003 ms. Trace:
I20260709 12:19:02.087918 32042 rpcz_store.cc:276] 0709 12:19:01.083850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.944232 (+860382us) service_pool.cc:224] Handling call
0709 12:19:02.087371 (+143139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:02.073760 32047 rpcz_store.cc:276] 0709 12:18:59.579092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.602339 (+ 23247us) service_pool.cc:224] Handling call
0709 12:18:59.602627 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.602982 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:59.690540 (+ 87558us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.717704 (+ 27164us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b18004664e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:18:59.839632 (+121928us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b18004664e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:19:01.444099 (+1604467us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.684209 (+240110us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.684432 (+   223us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:01.684487 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.846720 (+162233us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.957440 (+110720us) raft_consensus.cc:1685] finished
0709 12:19:01.957458 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.957593 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14907,"prepare.queue_time_us":14436,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":83070,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":67055,"spinlock_wait_cycles":114653184,"thread_start_us":1595291,"threads_started":2,"wal-append.queue_time_us":1434999}
W20260709 12:19:02.044059  4037 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.024s	user 0.011s	sys 0.000s
I20260709 12:19:01.932808  3765 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:02.096652 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:02.096817  3765 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.097905 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:02.098537 32039 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:02.099400 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:02.099483  3765 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:01.946041 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:01.950268  4234 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:02.053260 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.100323  4234 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:02.100762  4234 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.084139 32116 rpcz_store.cc:276] 0709 12:18:59.468532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.469777 (+  1245us) service_pool.cc:224] Handling call
0709 12:18:59.470102 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.259121 (+1789019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.956924 (+697803us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54002b5cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:01.957072 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4112439552}
I20260709 12:19:02.107530  4037 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.109719  4037 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:01.880384  4126 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:02.059329 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.969543 32660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.008s	sys 0.000s
W20260709 12:19:02.123965 32660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.008s	sys 0.000s
W20260709 12:19:01.984388  4258 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:02.089892 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:02.066861  4155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.001s	sys 0.010s
W20260709 12:19:02.111467 32084 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.840s (SENT) [suppressed 34 similar messages]
W20260709 12:19:02.128561 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 385570us
I20260709 12:19:02.100059 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2221) took 1439 ms. Trace:
I20260709 12:19:01.981252  4248 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.144104 32040 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:02.152760 32039 rpcz_store.cc:276] 0709 12:19:00.665506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.099169 (+1433663us) service_pool.cc:224] Handling call
0709 12:19:02.100008 (+   839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:02.159672 32040 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:02.160425 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2208) took 1890 ms. Trace:
I20260709 12:19:02.160960 32040 rpcz_store.cc:276] 0709 12:19:00.269880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.158810 (+1888930us) service_pool.cc:224] Handling call
0709 12:19:02.159253 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.160124 (+   871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.160360 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:02.153496 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2058) took 1062 ms. Trace:
W20260709 12:19:02.156446 32623 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.694s (SENT)
I20260709 12:19:02.164433 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:02.165712 32623 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.700s (SENT)
W20260709 12:19:02.166427 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 209170us
I20260709 12:19:02.166711 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2207) took 1897 ms. Trace:
I20260709 12:19:02.167127 32039 rpcz_store.cc:276] 0709 12:19:00.269596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.158799 (+1889203us) service_pool.cc:224] Handling call
0709 12:19:02.166155 (+  7356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.166389 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.166417 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.166457 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:02.166495 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.166545 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.166640 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.954487  4297 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:02.189008 32046 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.778s	user 0.007s	sys 0.000s
I20260709 12:19:02.189422 32046 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
W20260709 12:19:02.168255 32623 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.931s (SENT)
W20260709 12:19:02.170056 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 186518us
I20260709 12:19:02.171916 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:02.181699 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.169248 32653 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:02.190523 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:02.193670 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:02.122529  4273 raft_consensus.cc:2764] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:02.202143 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2088) took 1108 ms. Trace:
I20260709 12:19:02.202800 32046 rpcz_store.cc:276] 0709 12:19:01.081647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.261233 (+179586us) service_pool.cc:224] Handling call
0709 12:19:02.189942 (+928709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:02.122404  3765 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:02.206127 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2199) took 1947 ms. Trace:
I20260709 12:19:02.149957  4067 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.206951 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:02.058564  4284 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:02.185498  4042 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.423s	user 0.005s	sys 0.009s
I20260709 12:19:02.246407 32044 rpcz_store.cc:276] 0709 12:19:00.258824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.264807 (+  5983us) service_pool.cc:224] Handling call
0709 12:19:00.265186 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.265602 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.265643 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.266190 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.176419 (+910229us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:01.184777 (+  8358us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.621446 (+436669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.621570 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.205940 (+584370us) raft_consensus.cc:1685] finished
0709 12:19:02.205959 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.206076 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107388,"prepare.queue_time_us":12028,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":763388,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":29030,"thread_start_us":1301136,"threads_started":2,"wal-append.queue_time_us":1162504}
I20260709 12:19:01.963151  4296 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.168737 31965 rpcz_store.cc:276] 0709 12:19:01.090946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.091406 (+   460us) service_pool.cc:224] Handling call
0709 12:19:01.091733 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.091990 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:01.487183 (+395193us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.153100 (+665917us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.153163 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:02.153214 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.153295 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.153434 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51910,"prepare.queue_time_us":714609,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":91312,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":46587,"thread_start_us":1007882,"threads_started":3,"wal-append.queue_time_us":664300}
W20260709 12:19:02.225230 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.218550 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.218806 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.156323  4286 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.227207 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.155936  4250 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:02.276487 32043 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.054s	sys 0.094s
I20260709 12:19:02.082962  3978 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:02.284081 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77233.1us
W20260709 12:19:02.285720 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 140724us
W20260709 12:19:02.291368 32043 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.054s	sys 0.095s
I20260709 12:19:02.292133 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.122978  4136 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.301694 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:02.316469 32041 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:02.295224 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:02.205526  4283 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:02.297868 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:02.332265 32113 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.170660  4291 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:19:02.303973 32120 raft_consensus.cc:2410] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:02.205438  3924 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.156534  4247 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:02.245918  3495 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:02.337361 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:02.301050 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.012s	sys 0.016s
I20260709 12:19:02.290902  4250 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:02.368201  4204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.012s	sys 0.015s
I20260709 12:19:02.374543  4205 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.417865  4205 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:02.291927  4059 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.418673  4059 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.421080  4059 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.373210  3924 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.422483  3924 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:02.424629  3782 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.346s	user 0.009s	sys 0.002s
I20260709 12:19:02.428416  4205 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.432675 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 862) took 1068 ms. Trace:
I20260709 12:19:02.432993  4093 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.433224  4297 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.434669 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2202) took 2167 ms. Trace:
I20260709 12:19:02.435245  3978 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:02.436255  4235 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:02.435122 32045 rpcz_store.cc:276] 0709 12:19:00.266866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.440156 (+173290us) service_pool.cc:224] Handling call
0709 12:19:00.440526 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.440905 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.440937 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.441486 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.257576 (+816090us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.257944 (+   368us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.694148 (+436204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.694259 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.434384 (+740125us) raft_consensus.cc:1685] finished
0709 12:19:02.434436 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.434597 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100504,"prepare.queue_time_us":18054,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":686949,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":5047,"spinlock_wait_cycles":12672,"thread_start_us":1250585,"threads_started":2,"wal-append.queue_time_us":1988184}
I20260709 12:19:02.371716  3779 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:02.440629 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.441090  4293 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.330s	user 0.000s	sys 0.002s
I20260709 12:19:02.441310  4284 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.301165  4155 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:02.425040  3782 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.444286  4248 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.445133  4304 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.445633  4304 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:02.447729  4275 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.589s	user 0.000s	sys 0.001s
I20260709 12:19:02.447805  4304 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.449373 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2273) took 1757 ms. Trace:
W20260709 12:19:02.449721  4232 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.170s	user 0.007s	sys 0.000s
I20260709 12:19:02.278950  4067 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.433785  4093 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:02.377501 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.013s	sys 0.016s
I20260709 12:19:02.300693  4126 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.284708  4042 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:02.373215 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 78206us
I20260709 12:19:02.297675 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:02.387651 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50422.5us
W20260709 12:19:02.389202  4299 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:19:02.389978  4143 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.390146  4294 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:02.391333  3495 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:02.385854  3926 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.509s	user 0.013s	sys 0.001s
W20260709 12:19:02.394882  4253 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 12:19:02.395478 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.449827 32114 rpcz_store.cc:276] 0709 12:19:00.691843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.692121 (+   278us) service_pool.cc:224] Handling call
0709 12:19:00.692759 (+   638us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.693209 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.693245 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.693771 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.476172 (+782401us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:01.476226 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.476313 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.446951 (+970638us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:02.449033 (+  2082us) raft_consensus.cc:1685] finished
0709 12:19:02.449062 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.449302 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":345,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":10201472,"thread_start_us":781954,"threads_started":1,"wal-append.queue_time_us":1164761}
I20260709 12:19:02.396898  4302 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:02.397330 32113 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:19:02.492504 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 324240us
I20260709 12:19:02.398972 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:02.215412  4301 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.217s
I20260709 12:19:02.401328  4283 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.420193  4240 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:02.403889  4247 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:02.410758  4269 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.493s	user 0.000s	sys 0.001s
W20260709 12:19:02.411830  4271 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:19:02.412420  4267 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
I20260709 12:19:02.402508  4273 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:02.413667  4296 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.256628  3765 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.415197 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 856) took 1718 ms. Trace:
I20260709 12:19:02.416142  4160 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.455094 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.452104 32047 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 12. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 12:19:02.519917 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2198) took 2261 ms. Trace:
W20260709 12:19:02.486037 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.486783 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.392046  4250 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.516237 32659 rpcz_store.cc:276] 0709 12:19:00.697216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.697357 (+   141us) service_pool.cc:224] Handling call
0709 12:19:00.753840 (+ 56483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.766987 (+ 13147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.767016 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.767496 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.474332 (+706836us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.474387 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.474480 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.414982 (+940502us) raft_consensus.cc:1685] finished
0709 12:19:02.415003 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.415138 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12730,"raft.queue_time_us":35237,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":24895,"thread_start_us":706497,"threads_started":1,"wal-append.queue_time_us":1544367}
I20260709 12:19:02.489100 32116 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:02.501374  4201 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.397984  4288 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:19:02.493477 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.007s (SENT)
W20260709 12:19:02.536041 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 137147us
I20260709 12:19:02.536325 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:02.494827 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2209) took 2224 ms. Trace:
I20260709 12:19:02.493075  3926 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.433120 32660 rpcz_store.cc:276] 0709 12:19:01.364383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.364565 (+   182us) service_pool.cc:224] Handling call
0709 12:19:01.365036 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.365448 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.365487 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.366097 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.124422 (+758325us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:02.124479 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.124567 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.432468 (+307901us) raft_consensus.cc:1685] finished
0709 12:19:02.432488 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.432602 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83367,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6975,"thread_start_us":690362,"threads_started":1,"wal-append.queue_time_us":1034007}
W20260709 12:19:02.409353  4261 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.717s	user 0.009s	sys 0.012s
W20260709 12:19:02.487349  4204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.013s	sys 0.016s
W20260709 12:19:02.499220 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.541752 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.492630  4284 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:02.451946  4234 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.500180  4155 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.518775 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.503774  4304 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:02.521723 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.525760 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.450233  4232 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.520691 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2214) took 2161 ms. Trace:
I20260709 12:19:02.563802 32047 rpcz_store.cc:276] 0709 12:19:00.359530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.451459 (+2091929us) service_pool.cc:224] Handling call
0709 12:19:02.451762 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.520383 (+ 68621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.520626 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:02.520483 32043 rpcz_store.cc:276] 0709 12:19:00.258513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.260945 (+  2432us) service_pool.cc:224] Handling call
0709 12:19:00.262986 (+  2041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.273719 (+ 10733us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:01.237755 (+964036us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.302563 (+1064808us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.330529 (+ 27966us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.330707 (+   178us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:02.331008 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.391858 (+ 60850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.391951 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.440687 (+ 48736us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b240010b4b8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f228912109d 00007f228bd5310e 00007f228bd6a88c 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221
0709 12:19:02.519675 (+ 78988us) raft_consensus.cc:1685] finished
0709 12:19:02.519703 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.519855 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113373,"prepare.queue_time_us":97145,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":517151,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":7685,"spinlock_wait_cycles":92431232,"thread_start_us":1925197,"threads_started":3,"wal-append.queue_time_us":692224}
I20260709 12:19:02.490262 32120 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:02.528126  3495 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:02.529937 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:02.532155 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.512830  4297 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:02.545025 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.547241 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2217) took 2168 ms. Trace:
I20260709 12:19:02.551879 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2203) took 2283 ms. Trace:
I20260709 12:19:02.558055 32114 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:02.566324 32653 raft_consensus.cc:2410] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 12.
W20260709 12:19:02.568363 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.568183 32041 rpcz_store.cc:276] 0709 12:19:00.378749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.492440 (+2113691us) service_pool.cc:224] Handling call
0709 12:19:02.501926 (+  9486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.502183 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:02.511973 (+  9790us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.546918 (+ 34945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.546975 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:02.547023 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.547083 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.547186 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10122,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":7591,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":34586,"threads_started":1}
W20260709 12:19:02.571717 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78263.2us
W20260709 12:19:02.556529  4289 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:19:02.568385 32038 rpcz_store.cc:276] 0709 12:19:00.268412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.578751 (+310339us) service_pool.cc:224] Handling call
0709 12:19:00.579122 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.579542 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:00.579825 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.486725 (+906900us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.547908 (+1061183us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b2c0005c228. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:02.548368 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.548586 (+   218us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:02.548878 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.549080 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.549185 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.551613 (+  2428us) raft_consensus.cc:1685] finished
0709 12:19:02.551642 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.551805 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43491,"prepare.queue_time_us":85948,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":1378,"raft.queue_time_us":592256,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":75677,"spinlock_wait_cycles":186926976,"thread_start_us":906311,"threads_started":2,"wal-append.queue_time_us":1201569}
I20260709 12:19:02.573709  4291 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:02.574290  4303 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:02.574234  4291 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.575085  4303 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:02.562938  4304 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:02.575943  4252 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.576321  4304 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:02.576855  4291 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:02.576915  4274 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:02.577618  4274 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.578850  4232 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.502820  4283 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.580744  4274 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.575146  4136 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.544162  4284 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:02.562577  4288 consensus_queue.cc:1243] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:19:02.584579 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54727.7us
I20260709 12:19:02.535127  4160 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.565990  3984 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:02.593887 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 508) took 2268 ms. Trace:
I20260709 12:19:02.567212  4297 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:02.597275 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.598667 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.594273 31970 rpcz_store.cc:276] 0709 12:19:00.321941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.322107 (+   166us) service_pool.cc:224] Handling call
0709 12:19:00.322512 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:00.322874 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.322906 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.323562 (+   656us) log.cc:844] Serialized 52 byte log entry
0709 12:19:01.166028 (+842466us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:01.949239 (+783211us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.949479 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.949580 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.589959 (+640379us) raft_consensus.cc:1685] finished
0709 12:19:02.589983 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.590149 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65,"prepare.queue_time_us":640783,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":2344,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":27573,"thread_start_us":1586650,"threads_started":2,"wal-append.queue_time_us":2114288}
W20260709 12:19:02.601514  4323 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:19:02.602741 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2210) took 2329 ms. Trace:
I20260709 12:19:02.603158 32039 rpcz_store.cc:276] 0709 12:19:00.273153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.168049 (+1894896us) service_pool.cc:224] Handling call
0709 12:19:02.168329 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.168629 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.168655 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.169076 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.339846 (+170770us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:02.339887 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.339978 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.602492 (+262514us) raft_consensus.cc:1685] finished
0709 12:19:02.602516 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.602681 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6909,"raft.queue_time_us":30612,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":40106,"thread_start_us":94135,"threads_started":1,"wal-append.queue_time_us":254330}
I20260709 12:19:02.584242  4308 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.565104  4155 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.610949 32043 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:02.590821  4140 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.612646 32039 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:02.613369 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2224) took 1815 ms. Trace:
I20260709 12:19:02.613781 32039 rpcz_store.cc:276] 0709 12:19:00.798457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.611992 (+1813535us) service_pool.cc:224] Handling call
0709 12:19:02.612304 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.613146 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.613319 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:02.624693 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2226) took 1647 ms. Trace:
I20260709 12:19:02.625226 32039 rpcz_store.cc:276] 0709 12:19:00.977209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.623630 (+1646421us) service_pool.cc:224] Handling call
0709 12:19:02.624048 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.624273 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.624307 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.624364 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:02.624415 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.624492 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.624622 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:02.627925 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.591864  4059 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.024s
W20260709 12:19:02.637961 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48897.3us
I20260709 12:19:02.613197  3986 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.640535 31936 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:02.614609  3378 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:02.643033  4306 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:19:02.643517  3765 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:02.615065  4288 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:19:02.644507 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.640079  3782 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.599566  3984 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.637454  4067 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:02.645635  4319 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:19:02.654502 32653 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:02.677588 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:02.678241 32659 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.631459  4284 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.650879  3986 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.619828  4308 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.685338  4336 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:02.688069 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.690363 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.690761  4234 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.695997 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2212) took 2353 ms. Trace:
W20260709 12:19:02.697606 32084 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.016s
W20260709 12:19:02.721194 32084 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.000s	sys 0.017s
W20260709 12:19:02.721544 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 610045us
W20260709 12:19:02.698087 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.722594 32084 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 3.301s (SENT)
I20260709 12:19:02.698691 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:02.723564 32038 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:02.703965  4286 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:02.709841 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2219) took 2056 ms. Trace:
I20260709 12:19:02.653715  3378 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.718585 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:02.680202  3926 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.733004 32656 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.725924 32047 rpcz_store.cc:276] 0709 12:19:00.653177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.564389 (+1911212us) service_pool.cc:224] Handling call
0709 12:19:02.564662 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.565018 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.565056 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.567008 (+  1952us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.593464 (+ 26456us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:02.610205 (+ 16741us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.610427 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.610499 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.709624 (+ 99125us) raft_consensus.cc:1685] finished
0709 12:19:02.709643 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.709767 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1371,"prepare.queue_time_us":16471,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":9135,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":18035,"thread_start_us":14448,"threads_started":1,"wal-append.queue_time_us":67804}
I20260709 12:19:02.652002  4297 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.6, Last appended by leader: 6, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.721067 32046 rpcz_store.cc:276] 0709 12:19:00.342862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.205016 (+1862154us) service_pool.cc:224] Handling call
0709 12:19:02.273594 (+ 68578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.334649 (+ 61055us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:02.404688 (+ 70039us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.544305 (+139617us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.559433 (+ 15128us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.559611 (+   178us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:02.559680 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.559770 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.695803 (+136033us) raft_consensus.cc:1685] finished
0709 12:19:02.695822 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.695942 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66346,"prepare.queue_time_us":68098,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":153871,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":9781,"thread_start_us":131757,"threads_started":1,"wal-append.queue_time_us":203301}
W20260709 12:19:02.734939  4328 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:19:02.734681  4335 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.677031  4205 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.768896 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:02.768968 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:02.769378 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:02.779137 32046 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.779091 32047 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:02.711365  4042 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.771445 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:02.785671  4232 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.726327  4093 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.786937 32659 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:02.771749 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:02.784792 32656 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:02.788110 32038 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.806963  4093 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:02.810653 32652 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:02.811142  3984 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.811949  4160 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.812564  4313 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:19:02.812772  4093 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:02.814762 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:02.819024  4232 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:02.819449  4232 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:02.819808  4232 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:02.822942  4326 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:19:02.824611  4270 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.966s	user 0.004s	sys 0.000s
I20260709 12:19:02.824895 32038 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:02.830358 32039 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:02.830770 32039 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:02.831920  4205 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.834203  3378 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.815426  4297 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.837162 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2220) took 2182 ms. Trace:
I20260709 12:19:02.837517 32043 rpcz_store.cc:276] 0709 12:19:00.654760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.565633 (+1910873us) service_pool.cc:224] Handling call
0709 12:19:02.565881 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.611361 (+ 45480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.836907 (+225546us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b5400058cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:02.837088 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102886656}
I20260709 12:19:02.735519  3765 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:19:02.773769  4335 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:02.786692  4155 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.733397  3924 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.831693 32039 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:02.866214 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.851492 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:02.850121 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.844430 32044 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:02.869455 32656 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:02.850135 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:02.869272 32651 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:02.871601 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:02.851854 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.872777 32655 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:02.874912 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:02.876302 32039 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:02.838290  4250 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:02.879801 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2223) took 2203 ms. Trace:
I20260709 12:19:02.855733 32043 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:02.886567 32041 rpcz_store.cc:276] 0709 12:19:00.676502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.586334 (+1909832us) service_pool.cc:224] Handling call
0709 12:19:02.594237 (+  7903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.609925 (+ 15688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.609973 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.610549 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.879484 (+268935us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:02.879525 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.879590 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.879639 (+    49us) raft_consensus.cc:1685] finished
0709 12:19:02.879649 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.879751 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15596,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5474,"thread_start_us":244986,"threads_started":1,"wal-append.queue_time_us":259933}
I20260709 12:19:02.852296 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:02.881145 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 165349us
W20260709 12:19:02.893014 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 173093us
I20260709 12:19:02.846472 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:02.894706 32042 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:02.907869 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61462.3us
I20260709 12:19:02.908778 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:02.912794 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 140129us
I20260709 12:19:02.914681  4296 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:02.915119  4296 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.915664  4296 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:02.897897 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:02.914016 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:02.927345 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:02.884904 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:02.901010 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.987824 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.987030 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.987039 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:02.989944 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 632) took 1063 ms. Trace:
I20260709 12:19:02.990150 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2093) took 1366 ms. Trace:
I20260709 12:19:02.990365 32115 rpcz_store.cc:276] 0709 12:19:01.926968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.927089 (+   121us) service_pool.cc:224] Handling call
0709 12:19:01.946202 (+ 19113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.336106 (+389904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.336152 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.336209 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:02.336255 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.989743 (+653488us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54002f9c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:02.989789 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.989893 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896156032}
W20260709 12:19:02.989354 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75426.8us
I20260709 12:19:02.990895 32039 rpcz_store.cc:276] 0709 12:19:01.623919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.626544 (+1002625us) service_pool.cc:224] Handling call
0709 12:19:02.627016 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.830230 (+203214us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b54000caba0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d11c0 00007f228c1d2231 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c
0709 12:19:02.988936 (+158706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.988979 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.989034 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:02.989079 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.989167 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.990076 (+   909us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41041664}
I20260709 12:19:02.910132 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:02.839736  4308 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.845918  3924 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:02.842228  4335 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:03.000222 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73001.8us
I20260709 12:19:02.840842  3986 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:02.865455  3765 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:02.843990  4283 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:03.001401 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 91347.7us
I20260709 12:19:03.013712 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:03.016347 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.016839 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.931185  4240 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:02.931613  3991 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:03.022680 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:02.931139  4309 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:03.026868 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:03.000097  4290 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:19:03.028255 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:02.926113  4296 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.038712 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.010638  4155 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:03.038712 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.010612  4097 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.039825 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.040784 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:03.043114 32115 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:03.022948  4040 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:03.023140  4064 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.030416  3991 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.030103  4240 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:03.031319  4250 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.002357  4304 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.056937 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:03.044755  4222 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:03.050685  4093 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.036679  4309 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:03.045051  4283 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:03.052269  4040 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.054422  4288 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.029646  3765 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:03.064391 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 50776.6us
I20260709 12:19:03.066124 32038 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:03.047783  3924 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:03.047585  4274 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:03.064463 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:03.071535  4126 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:03.071717  4304 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:03.054567  3991 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:03.056509  4240 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.048516  4097 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:03.065011  4040 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.059741  4222 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:03.080377 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.061313  4309 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:03.062418  4093 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:03.086274 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.075253  4126 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:03.076121  4155 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.079720  4097 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.096487 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:03.081513  4345 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:03.068914  4283 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.086735  4222 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:03.084342  4037 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:03.073860  4288 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.088497  4093 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.077160  3924 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.081665  4064 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:19:03.077342  4274 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.132696 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.133494  4274 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.111068 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.133850 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.134702 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:03.135306 32115 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:03.138875 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.142216 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:03.130203 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.537256 32040 rpcz_store.cc:276] 0709 12:19:00.270490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.161893 (+1891403us) service_pool.cc:224] Handling call
0709 12:19:02.162241 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.163168 (+   927us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:02.163571 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.494329 (+330758us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.494383 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:02.494435 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.494503 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.494763 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16599,"prepare.queue_time_us":128,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":2689,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5347,"thread_start_us":327966,"threads_started":1,"wal-append.queue_time_us":368029}
I20260709 12:19:03.134619 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:03.145610 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.146413 32040 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:03.120715  4315 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.565s	user 0.000s	sys 0.009s
I20260709 12:19:03.123973  4283 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.126711  4155 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.128970  4288 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:03.131111  4268 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.946s	user 0.004s	sys 0.008s
I20260709 12:19:03.131412 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:03.131543  4037 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.156145  4037 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.158085 32115 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:03.094594  4140 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:03.163769 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2264) took 2812 ms. Trace:
I20260709 12:19:03.096953  4205 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:19:03.164315 32117 rpcz_store.cc:276] 0709 12:19:00.351717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.418785 (+ 67068us) service_pool.cc:224] Handling call
0709 12:19:00.429140 (+ 10355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.486857 (+ 57717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.486904 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.487498 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.192351 (+704853us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.192393 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.866570 (+674177us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b540012b1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:01.866620 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.130538 (+1263918us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:03.163499 (+ 32961us) raft_consensus.cc:1685] finished
0709 12:19:03.163552 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.163699 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":124,"spinlock_wait_cycles":299339008,"thread_start_us":704392,"threads_started":1,"wal-append.queue_time_us":1496796}
I20260709 12:19:03.151732 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:03.172653 32046 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 14. Ops: []
I20260709 12:19:03.172827  4232 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.175081 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2284) took 1221 ms. Trace:
I20260709 12:19:03.173910  4126 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.175967 32115 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:03.177315  4126 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:03.152504 32651 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:19:03.172590 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2205) took 2903 ms. Trace:
I20260709 12:19:03.179579 32042 rpcz_store.cc:276] 0709 12:19:00.269016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.106960 (+1837944us) service_pool.cc:224] Handling call
0709 12:19:02.107256 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.906800 (+799544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.172340 (+265540us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b5400033ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:03.172528 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1810553600}
I20260709 12:19:03.175433 32112 rpcz_store.cc:276] 0709 12:19:01.954063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.954286 (+   223us) service_pool.cc:224] Handling call
0709 12:19:02.044886 (+ 90600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.045278 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.045306 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.045769 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.453155 (+407386us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:02.453190 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.453251 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.120175 (+666924us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:03.174796 (+ 54621us) raft_consensus.cc:1685] finished
0709 12:19:03.174831 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.175027 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1242,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":14998,"spinlock_wait_cycles":14823296,"thread_start_us":371638,"threads_started":1,"wal-append.queue_time_us":468513}
W20260709 12:19:03.189541 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 139520us
I20260709 12:19:03.153622  4249 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:03.239022  3765 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.240054  4296 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.243052  4232 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.241933  4093 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.244422 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.245255  4222 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.248575  4309 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.265501 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.269003 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.277490 32651 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:03.211038  4140 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.264725  4249 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:03.285585 32113 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:03.221220  4327 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:19:03.288785  4249 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.273995  3765 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:19:03.237730  4040 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:03.300355 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 187187us
I20260709 12:19:03.281066  4296 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:19:03.316048 32042 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:03.317531 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:03.324476 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.331859 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:03.318275 32042 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:03.310525  4345 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.344170 31965 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:03.345947 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.346977 31965 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:03.341323 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.347661 32042 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:03.350530 31970 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:03.342412  4297 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.355772 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.358230 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.360384 32042 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:03.361110 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.361583 32042 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:03.362205 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.362821 32042 raft_consensus.cc:2376] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:19:03.364241 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.367210 31965 raft_consensus.cc:2368] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:03.368256 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.369232 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.369611 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.369901 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:03.371214 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.373730 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.339033  4140 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:03.372143 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 181563us
I20260709 12:19:03.376874 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.411247 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.412297 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.413329 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.413957 32047 raft_consensus.cc:2368] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:19:03.379698  4136 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.414578  4136 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.416577  4136 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:03.417367 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.381724 32651 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:03.421196 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.423623 31968 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:03.385524 32042 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:03.390838  4222 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.394969  4093 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.395763  4309 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.397693 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.358343  4297 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.368065  3924 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.424151 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.419543 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.428778 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.430966 32047 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:03.433277 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.433864 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:03.434978 32043 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:03.437775 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 217956us
I20260709 12:19:03.437840 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.439050 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.439553 32042 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:03.441226 31970 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:03.444259 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.447944 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:03.450454 32042 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:03.459554  3991 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.461155 32116 raft_consensus.cc:2368] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:03.463789 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.464560 32658 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:03.474660 32658 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
W20260709 12:19:03.476336 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.477928  3991 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:03.482379  4352 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:19:03.482525  4040 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.440755  4093 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:03.484635  4093 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:03.485042  4093 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:03.486140 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.487335 32043 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:03.489097 32043 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:03.493577 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.498111 32038 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:03.498798 32047 raft_consensus.cc:2368] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:19:03.499576 32042 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:03.428835  4345 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:03.507866  4360 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:19:03.510844 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62986.2us
I20260709 12:19:03.511375  4247 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.511554 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:03.511951  4247 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.512406  4247 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:03.514072  4232 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:03.514889  4232 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:03.515275  4232 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:03.515961 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.516395 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.520985  4093 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.525365 31968 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 17. Ops: []
I20260709 12:19:03.526043 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.527429 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.528710 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.529313 32047 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:03.530934 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.531345 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.531977 31967 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:03.532421 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.539191 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:03.498661  4323 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:19:03.548507  4222 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:03.549320 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.550552 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.551638 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.552620 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.553596 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.513279  4247 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.555446 31969 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:03.542874 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.542992 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.544301 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.557201 32041 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:03.558748  3991 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:03.559636  3991 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:03.559974  3991 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:03.560789  3991 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.562007  3991 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:03.567390  4262 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.709s	user 0.002s	sys 0.002s
I20260709 12:19:03.572248  4284 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:03.576486  4327 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:03.436647  4249 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.581018 32115 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:03.580457  3765 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:03.582738 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:03.584039 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.595305 31967 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:03.545722 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.584842 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.614216  4126 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.616523 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.617678 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:03.586632  4335 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 12:19:03.626368  4335 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.629117  4335 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.605257 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:03.546857 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.631091 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.631654 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.632891 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.606274 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.456928  4240 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.635255 31964 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:19:03.635921  4240 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.609474 32653 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:03.638128 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:03.610602  4249 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:03.604089  4327 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:03.639006  4232 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.639878  4327 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.641032  4232 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.641520 31971 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:03.641602  4097 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
I20260709 12:19:03.619544 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.642097  4097 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.618752 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.616149 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.613866 32658 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.642282 31964 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: [16.6-16.6]
I20260709 12:19:03.643272 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.623045  3765 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.650080 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.651489 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.651875  3765 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.607524  4309 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:03.624257 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.652419  4309 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:03.652555 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.652861  4290 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:03.653249 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.653605 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.653352  4309 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.649912 31965 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:03.654465 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.654626  4309 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:03.655069 32043 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:03.655929 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.656610 32043 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:03.657307 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.653676  4290 consensus_queue.cc:1243] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:03.634111 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:03.598044 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.656453 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.645238 31967 raft_consensus.cc:2368] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:03.643807 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.676992 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.677224 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:03.677945 31971 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:03.666432 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 122998us
I20260709 12:19:03.658254 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.659549 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:03.644220 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.645141 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.660146 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.675868 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:03.667263 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.674763 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.644615 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.649932 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:03.683841 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.685865 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.687978 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:03.689492 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:03.689836 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.686508 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.692090 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.694036 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.694065 32658 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:03.681128 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:03.699004 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101085us
I20260709 12:19:03.699729 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.700964 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.680547 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.702615 31965 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:03.703616 31971 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:19:03.704322 31965 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:03.705690 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.706049 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 9 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.706950 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.708048 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.659190  4302 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:03.710489  4302 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:03.711860  4302 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.714617  3765 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:03.715325  3765 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:03.716388  3765 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.718735 32042 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:03.720594 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.721519 32042 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:03.722484  4240 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:03.723264  4240 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:03.724041  4290 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:03.725003 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50323.4us
I20260709 12:19:03.726418 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2216) took 3349 ms. Trace:
I20260709 12:19:03.724385  4240 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.648465  4093 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:03.727193 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:03.728557 31964 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:19:03.701620 32043 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:03.735643  4306 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:03.736315  4306 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.736717  4306 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:03.736466 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.737115  4306 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.737905 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.734824 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:03.739096 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.644698  4097 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:03.713892  4302 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.722853 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:03.731621 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53084.4us
I20260709 12:19:03.726786 32045 rpcz_store.cc:276] 0709 12:19:00.376552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.452307 (+2075755us) service_pool.cc:224] Handling call
0709 12:19:02.453060 (+   753us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:02.453411 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.453464 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:02.489669 (+ 36205us) log.cc:844] Serialized 52 byte log entry
0709 12:19:02.613265 (+123596us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:02.628364 (+ 15099us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.631849 (+  3485us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.632075 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.632175 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.139837 (+507662us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b1800328f00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22900a358f 00007f22890e2571 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd
0709 12:19:03.599552 (+459715us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b1800328f00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:03.723721 (+124169us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b1800328f00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:03.726185 (+  2464us) raft_consensus.cc:1685] finished
0709 12:19:03.726204 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.726363 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10881,"prepare.queue_time_us":523159,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":34549,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":8999,"spinlock_wait_cycles":19036032,"thread_start_us":105065,"threads_started":1,"wal-append.queue_time_us":139903}
I20260709 12:19:03.743119 32010 leader_election.cc:400] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:03.743485  3495 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:03.743389  4136 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:03.744654  3495 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:03.745141  3495 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.746680  4040 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:03.740403  3990 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:19:03.747105  4040 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.747221  4210 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:03.747596  4210 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.747741  3990 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:03.748273  3990 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:03.748423  4210 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.749528  4210 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:03.749639  4040 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.731635 32112 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:03.751199  4040 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:03.752370 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 60152.7us
I20260709 12:19:03.752583  4036 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:03.743520 32010 leader_election.cc:403] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.753386  4036 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:03.753758  4036 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:03.754475 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.756172  4327 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.749344  3990 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:03.757107  3990 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.743161 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.759583 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.760215 32045 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:03.760921 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.761402 32045 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:19:03.762543 32113 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
W20260709 12:19:03.729602 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49176us
I20260709 12:19:03.745913  3495 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.763494 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:03.733345 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:03.794099 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.794396 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61122.4us
I20260709 12:19:03.795038 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.795842  4288 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.796032 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:03.796911  4211 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.762084 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.796523 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.798467  4288 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:03.798877 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.800352  4288 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.739782 32043 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:03.804294 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.805537 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.806356 32045 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:03.806738 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.766631 32115 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:03.768378 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:03.770215 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.806121 32043 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:03.824016 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55733.2us
I20260709 12:19:03.824712 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:03.825605 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55471.8us
I20260709 12:19:03.826715 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.832191 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.775429  4273 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:03.838013 32045 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:03.840802 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.843989 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.833014  3924 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.733803  4290 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:03.832577  4335 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.849018 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.754487  4036 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.835387  4211 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:03.843431  4288 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.855198  4297 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:03.844435  4301 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:03.859911 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:03.860601 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.769830 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.849975  4273 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:03.872656 32042 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:03.872524 32045 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:03.896018 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.854004  4335 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:03.900902 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.898710 32043 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:03.901116 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.866390  4211 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:03.905102 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:03.866535  4284 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.910056 32045 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:03.907547 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.912231 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:03.912143 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.912364 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:03.913545 32045 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.912835 31969 raft_consensus.cc:606] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:03.914072 31969 raft_consensus.cc:614] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:03.917595 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.919240 32010 leader_election.cc:400] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:03.919692 32010 leader_election.cc:403] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.921896 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.907363  4335 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.907127  4297 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.922983 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:03.906085  4211 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.928512 32042 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
W20260709 12:19:03.930974  4370 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:03.917619  4140 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.914587  4301 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:03.788398  4306 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:03.941999  4244 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.917186  4290 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.923688  4273 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:03.945701 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 330962us
I20260709 12:19:03.938736  4284 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:03.957553 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.944730 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 255293us
I20260709 12:19:03.967028 32623 leader_election.cc:400] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:19:03.967583 32623 leader_election.cc:403] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.955164  4244 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:03.952180  4330 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:19:03.981088 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:03.984988 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.956665  4329 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:19:03.990424 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:03.978365  4121 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12.
I20260709 12:19:03.997145 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:03.998530  4371 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.002508 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.003273  4321 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.003679 32041 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:04.005273 32041 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [8.6-14.8]
I20260709 12:19:04.007635  4284 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.008190  4284 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.011011  4284 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.011672 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.012368 32120 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:04.013621 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:04.014480  4284 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:04.014933  4284 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.015347  4284 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:04.017576 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.019105 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.019559 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.019488 32041 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:04.023448 32045 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:03.989851  4281 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.031041 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 866) took 1379 ms. Trace:
I20260709 12:19:03.987057  4327 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:19:04.054983  4306 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:04.060967  4321 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:04.063841 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.995932  4273 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:04.068694 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 146909us
I20260709 12:19:04.069607 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.071795 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:04.072095 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 167071us
I20260709 12:19:04.074473 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.045205 32660 rpcz_store.cc:276] 0709 12:19:02.651915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.652064 (+   149us) service_pool.cc:224] Handling call
0709 12:19:02.652370 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.652664 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.652693 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.577495 (+924802us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b1800454960. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a
0709 12:19:03.577945 (+   450us) spinlock_profiling.cc:243] Waited 899 us on lock 0x7b2c00115d88. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:03.606587 (+ 28642us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.938698 (+332111us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:03.938736 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.938814 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.030579 (+ 91765us) raft_consensus.cc:1685] finished
0709 12:19:04.030609 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.030961 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":899,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":14417,"spinlock_wait_cycles":11185664,"thread_start_us":279954,"threads_started":1,"wal-append.queue_time_us":365563}
I20260709 12:19:04.076856 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.073649  4301 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.082372  4302 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.075325  4297 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.081900  4211 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.072011  4335 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:04.086930 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 323485us
I20260709 12:19:04.087399  4335 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:04.088402  4211 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.031807  3924 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:04.088291 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:04.094796 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.096654 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.101871 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.107137 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2213) took 3753 ms. Trace:
I20260709 12:19:04.107652 32044 rpcz_store.cc:276] 0709 12:19:00.353639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.257639 (+1904000us) service_pool.cc:224] Handling call
0709 12:19:02.274690 (+ 17051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.870045 (+595355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.106833 (+1236788us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54001941d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:04.107066 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309776768}
I20260709 12:19:04.112102 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.112939 32117 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:19:04.113933 32118 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:04.115020 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.076228  4273 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:04.115849 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:04.101958  4301 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.099690  4297 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.101467  4097 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:04.100289  4302 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.126345 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:04.126713 32121 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:04.128307 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:04.128612 31968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6679c928e92e4f799b88357707650b60"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:04.170543 31968 raft_consensus.cc:606] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:04.171023 31968 raft_consensus.cc:614] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:04.130424 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.132519 32114 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:04.132372 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:04.172881 32121 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:04.137607 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:04.174880 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73103.3us
I20260709 12:19:04.138629 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:04.175676 32040 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 14. Ops: []
W20260709 12:19:04.177207 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 50950.7us
I20260709 12:19:04.177270 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:04.178175 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65069.8us
I20260709 12:19:04.179445 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.078151  4244 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.135278  4351 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:04.129925  4222 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.147960  3991 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.190861 32120 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:04.167151  4136 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:04.200379 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.172112  4296 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:04.215449 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 510) took 1148 ms. Trace:
I20260709 12:19:04.215538 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:04.215917 32038 rpcz_store.cc:276] 0709 12:19:03.066626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.066957 (+   331us) service_pool.cc:224] Handling call
0709 12:19:03.497922 (+430965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.498566 (+   644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.215246 (+716680us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54001a5ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:04.215392 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990498688}
I20260709 12:19:04.217450 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:04.187417  4232 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:04.133785  4037 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:19:04.175426  4329 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:04.192224  4363 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.218156 32038 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.232008 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:04.224018 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:04.211026  4330 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.191700  4297 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:19:04.214052  4296 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:04.214344  4351 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:04.244768 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 107248us
I20260709 12:19:04.210414  4222 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.257308 32038 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:04.246744  4330 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.259747 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:04.260231 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:04.263005  4330 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:04.215173  4136 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.265867 31968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:04.246248 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.266610 31968 raft_consensus.cc:606] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:04.213335  3991 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.240922  4232 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:04.247685  4296 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.275729  4222 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:04.281733 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:04.225539  4273 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.264142  4384 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.291715  4297 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:04.292163  4297 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:04.293124  4297 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:04.297691  4040 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 12:19:04.298276  4136 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.299362  4232 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.300020  4036 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.231194  4329 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.304522  4329 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.269920  4093 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.272392  4381 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:04.305913  4381 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:04.306813  4381 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.307976  4381 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.310390  4037 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:04.311053  4375 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:04.311769  3991 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:04.314924 32121 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:04.315907 32117 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.316865 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.317124 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 512) took 1250 ms. Trace:
I20260709 12:19:04.317466 32040 rpcz_store.cc:276] 0709 12:19:03.067145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.147973 (+ 80828us) service_pool.cc:224] Handling call
0709 12:19:04.108844 (+960871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.176340 (+ 67496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.316364 (+140024us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b54001941d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:04.316974 (+   610us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54001941d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:04.317075 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2362220032}
I20260709 12:19:04.318449 32114 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:04.320013  4351 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.6, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.321425 32116 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:04.322487  4121 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:04.323601  4121 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:04.324409  4121 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:04.325151  4121 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.326751  4302 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.327252  3924 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.327394  4302 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.327813  3924 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.329854  3924 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.264349  4390 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:04.331100  4390 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.331933  4390 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:04.335516  4249 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:04.336689  4249 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:04.337070  4249 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:04.338711  4249 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.341024  4037 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:04.342041  4037 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.340410  4249 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.346639 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:04.323352 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.349972 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:04.350975 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 166879us
W20260709 12:19:04.351536 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 205598us
I20260709 12:19:04.346056  4297 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.367724 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:04.371667 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 97143.6us
I20260709 12:19:04.368444 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.372949 32117 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:04.373216 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.380537 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.385069 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:04.389185 31938 leader_election.cc:400] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:04.330039  4302 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:04.373484  4288 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.006s
I20260709 12:19:04.421972 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:04.427704  4296 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.429275 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:04.430151 32658 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:04.398286  3924 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:04.433197 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:04.432790 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.399842  4384 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.403141  4240 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.431607  4288 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:04.406236  4222 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:19:04.438578  4296 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:04.439231  4296 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:04.439440 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:04.439915 32039 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:04.439599  4296 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.440826  4296 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.442953  4384 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.445223  4384 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.449098  4296 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.452546 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:04.453136 32651 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.371203  3990 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:19:04.459332  3990 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.460254 32651 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:04.461431 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:04.407429  4387 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.371338  4329 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:19:04.434675  4363 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.402206  4284 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.468062  4296 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:04.466676  4288 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.482199  4126 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.461994  4400 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:04.470541  4384 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.509827  4281 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:04.592808  4393 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:19:04.627040  4390 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.635660  4390 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:04.668577 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.003s
W20260709 12:19:04.672464 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 325740us
I20260709 12:19:04.734064 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:04.736331 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.735749  4126 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:04.674024 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 282039us
I20260709 12:19:04.465379  3924 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:04.740675  3924 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.471905  4097 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:04.679457 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:04.688220 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2254) took 1045 ms. Trace:
I20260709 12:19:04.747635  4097 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.692174 32041 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:04.693611 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:04.695371 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.467294 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:04.507701  4240 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:04.582372  4384 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.631961  4387 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.578456  4400 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.755717  4400 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.701617 32047 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:04.757694 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.757674 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:04.714581  3944 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.760304 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.760473 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:04.579393  4281 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:04.762467 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.763470  4384 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.764660 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.764245  4281 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.716509  4284 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.499074  3991 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:04.561059  4288 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.718446  4401 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.767632 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.768011 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.771919 32118 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.729588  4327 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.507133  4383 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 12:19:04.748155 32038 rpcz_store.cc:276] 0709 12:19:03.643010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.263506 (+620496us) service_pool.cc:224] Handling call
0709 12:19:04.264785 (+  1279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.267139 (+  2354us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:04.410304 (+143165us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.682114 (+271810us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.683435 (+  1321us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.683652 (+   217us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:04.683735 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.683878 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.687559 (+  3681us) raft_consensus.cc:1685] finished
0709 12:19:04.687602 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.687910 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1915,"prepare.queue_time_us":144367,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":6156,"raft.queue_time_us":57430,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":3803,"threads_started":1,"wal-append.queue_time_us":248390}
W20260709 12:19:04.750943 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.004s	sys 0.003s
I20260709 12:19:04.716500 31966 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:04.741911 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:04.774127 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:04.766292  4281 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.777648 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:04.778093 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.770237  4415 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:04.776919 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.792095  4415 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.792634 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.793942  4401 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.795116  4284 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.796093 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.796919  4401 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.794210 31970 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:04.790463 31964 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:04.798448 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 652128us
I20260709 12:19:04.775609  4400 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.811964 32046 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:04.812163 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.812985  4403 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.813154 32112 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:04.815711 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:04.816499 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.819754  4126 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:04.830184 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.831149 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2117) took 1277 ms. Trace:
I20260709 12:19:04.831562 32039 rpcz_store.cc:276] 0709 12:19:03.553863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.015361 (+461498us) service_pool.cc:224] Handling call
0709 12:19:04.015651 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.440348 (+424697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.830607 (+390259us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54001225d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:04.830937 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975278464}
I20260709 12:19:04.832865  4240 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:04.834342  4240 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.835716  4400 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.828744 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:04.845352 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 103679us
I20260709 12:19:04.847481 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:04.848379  4403 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:19:04.849165 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.817454  4327 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:04.851414  4288 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:04.854987  4401 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.853060  4327 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:04.883144 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:04.887887 32118 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:04.882191 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:04.883208 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:04.888993  3495 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.889230 32651 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:04.889869 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:04.891052  4415 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:04.828451  4284 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:19:04.898253 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:04.898663 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:04.899127 32039 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.902881  4403 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:04.903388  4403 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:04.904668  4403 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:04.906484 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53601.5us
I20260709 12:19:04.914441 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:04.906672 32038 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.913754 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:04.877777  4396 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:19:04.914927 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2118) took 1360 ms. Trace:
I20260709 12:19:04.928201 32041 rpcz_store.cc:276] 0709 12:19:03.554078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.020666 (+466588us) service_pool.cc:224] Handling call
0709 12:19:04.026482 (+  5816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.750638 (+724156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.914566 (+163928us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400450954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:04.914870 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529044224}
I20260709 12:19:04.906258  4403 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:04.940733 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.920897  4288 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:04.947311 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 507694us
I20260709 12:19:04.948035  4415 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:04.947795  4288 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:04.948657  4415 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.949236  4415 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:04.951539 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 148544us
I20260709 12:19:04.952642 32651 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:19:04.929591  4327 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:04.954879 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56427.7us
I20260709 12:19:04.950068 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:04.958894 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.963610 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 125878us
I20260709 12:19:04.971560  4391 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:04.972210  4391 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:04.972734  4391 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.974488 32040 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
W20260709 12:19:04.998831 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 151555us
I20260709 12:19:04.970105  4327 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.005771  4327 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.001710  4396 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:19:05.007831  4381 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.699s	user 0.013s	sys 0.007s
I20260709 12:19:05.008185  4396 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:05.008680  4396 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:05.001919 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.021524 32046 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:05.028733 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.037719 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.039995 32039 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:05.038616 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:05.058815 32038 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:05.062726 32112 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:05.050297  4288 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.061983 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:05.010241  4396 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.072333 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.075752 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.076735 32039 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:05.082392  4336 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:05.082931  3765 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:05.082446 32120 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:05.080574 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.083395  3765 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.086424  4418 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:05.086941  3765 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:05.087431  4297 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.004s
I20260709 12:19:05.087225  4418 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:05.088173 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 147784us
I20260709 12:19:05.089017 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:05.089643  4418 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:05.091485  4381 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.092782  4396 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:05.071331  3924 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:05.095302  4249 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.722s	user 0.013s	sys 0.005s
I20260709 12:19:05.095517 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2261) took 1354 ms. Trace:
I20260709 12:19:05.096609 32041 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:19:05.099416 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 149568us
W20260709 12:19:05.103168 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101483us
I20260709 12:19:05.103775 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.105413 32047 rpcz_store.cc:276] 0709 12:19:03.740848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.305164 (+564316us) service_pool.cc:224] Handling call
0709 12:19:04.305769 (+   605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.756627 (+450858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.095214 (+338587us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54000569d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:05.095449 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908588288}
I20260709 12:19:05.113101 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2268) took 1024 ms. Trace:
I20260709 12:19:05.113538 32041 rpcz_store.cc:276] 0709 12:19:04.082984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.095718 (+1012734us) service_pool.cc:224] Handling call
0709 12:19:05.096128 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.105830 (+  9702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.107135 (+  1305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.125711  3495 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:05.112483 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.127292  3495 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:05.130015  4247 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.464s	user 0.083s	sys 0.073s
I20260709 12:19:05.122730  4336 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.139236  3495 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.139999  4336 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.140903  3495 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:05.152366 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 91161us
I20260709 12:19:05.153272 32112 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:05.157374 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 198552us
I20260709 12:19:05.163831 32658 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:05.156329  4409 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.556s	user 0.000s	sys 0.004s
W20260709 12:19:05.151144  4247 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.584s	user 0.083s	sys 0.073s
I20260709 12:19:05.171094 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:05.177950 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.178579 31966 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:05.184198 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2259) took 1482 ms. Trace:
W20260709 12:19:05.188300  4426 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:19:05.161149  4381 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.190688 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78319.1us
I20260709 12:19:05.182152 32651 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:05.191915 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.197697  4321 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.198132 31972 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:05.198393  4321 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.199501 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2326) took 1023 ms. Trace:
I20260709 12:19:05.199869 32121 rpcz_store.cc:276] 0709 12:19:04.175879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.176772 (+   893us) service_pool.cc:224] Handling call
0709 12:19:04.177062 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.315289 (+138227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.199047 (+883758us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002b57d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:05.199302 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316247296}
I20260709 12:19:05.200971  4321 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.203325  4381 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.205530  4381 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.209975 32660 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:05.211537 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.212713  4405 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.213979  4428 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.193902 32043 rpcz_store.cc:276] 0709 12:19:03.702076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.284657 (+582581us) service_pool.cc:224] Handling call
0709 12:19:05.183094 (+898437us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:19:05.183438 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.183795 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.183818 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.183868 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:05.183895 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.183941 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.184046 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47829248}
I20260709 12:19:05.217404 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.220893 32120 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
W20260709 12:19:05.226940 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.227712 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:05.231161 31964 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:05.233100 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.195065  4336 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.230768 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.237769  4363 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:05.237869  4301 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.246408  4288 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:05.246850 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.247816 31964 raft_consensus.cc:2410] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:19:05.226549  4428 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.234615  4405 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:19:05.243767  4418 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:05.255146  4288 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:05.255227  4428 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.256229  4288 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.260025 32043 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:05.263034  4137 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.263685  4137 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.264257 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.265969  4137 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.254606  4392 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:05.268858  4392 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.269323  4392 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:05.274531  4405 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:05.275315  4405 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.277981  4405 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.267774 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.266992  4155 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.312776  4155 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.313207  3990 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:05.313673  3990 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:05.250998 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:05.314972 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64068.5us
I20260709 12:19:05.280141 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:05.284973  4354 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:05.285775  4399 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:05.287472  3944 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:19:05.297605  4429 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:05.297386  4391 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.298157 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2250) took 1756 ms. Trace:
I20260709 12:19:05.234627  4240 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:05.303575  4415 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.305779 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2252) took 1697 ms. Trace:
I20260709 12:19:05.307477  4384 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.308036  4201 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.253504  3765 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.282081 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.318452 32042 rpcz_store.cc:276] 0709 12:19:03.541659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.943577 (+401918us) service_pool.cc:224] Handling call
0709 12:19:03.944218 (+   641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.339204 (+394986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.339234 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.339273 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:04.339302 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.297736 (+958434us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54001a72d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:05.297795 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.297978 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907839744}
I20260709 12:19:05.300813 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46882 (request call id 18) took 1052 ms. Trace:
W20260709 12:19:05.341454 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 59461.3us
I20260709 12:19:05.340368 31968 rpcz_store.cc:276] 0709 12:19:04.248057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.248254 (+   197us) service_pool.cc:224] Handling call
0709 12:19:05.300205 (+1051951us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400288aa0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14c520 00007f228c1c9652 00007f229119760a 00007f228a71d7d4 00007f228a71d762 00007f228a71d691 00007f228a71d60c 00007f228a71c8c2 00007f228a50a48c 00007f228a5098c6
0709 12:19:05.300583 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253540608}
I20260709 12:19:05.348826 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:05.350816 32040 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:05.352192 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.315287  4155 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.314584  3990 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.335249  4301 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:05.371253 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.352155 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.335104  4201 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:05.328668  4240 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.334669  4384 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:05.387480 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.387995  4384 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.388795  4240 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:05.389230  4384 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.369812 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2255) took 1529 ms. Trace:
I20260709 12:19:05.376777 32039 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:05.377925  4321 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.379880 32658 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:05.346853  4400 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:05.381749  4201 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.332151 32044 rpcz_store.cc:276] 0709 12:19:03.608311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.117516 (+509205us) service_pool.cc:224] Handling call
0709 12:19:04.117974 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:05.064145 (+946171us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b18003f66a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1c7bfb 00007f228c1d264b
0709 12:19:05.082699 (+ 18554us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:05.083058 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.154642 (+ 71584us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.154802 (+   160us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:05.166974 (+ 12172us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b18003f66a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:05.178447 (+ 11473us) log.cc:844] Serialized 52 byte log entry
0709 12:19:05.178598 (+   151us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:05.181465 (+  2867us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.197283 (+ 15818us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.228215 (+ 30932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.236860 (+  8645us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.257055 (+ 20195us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003f66a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:05.305454 (+ 48399us) raft_consensus.cc:1685] finished
0709 12:19:05.305474 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.305622 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56186,"prepare.queue_time_us":65942,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":2028,"raft.queue_time_us":153479,"raft.run_cpu_time_us":691,"raft.run_wall_time_us":32258,"spinlock_wait_cycles":50953088,"thread_start_us":174857,"threads_started":2,"wal-append.queue_time_us":147938}
W20260709 12:19:05.383708 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:05.409185 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 195021us
I20260709 12:19:05.395481  4400 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:05.392102 32045 rpcz_store.cc:276] 0709 12:19:03.643315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.277400 (+634085us) service_pool.cc:224] Handling call
0709 12:19:04.277771 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.277986 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:04.874340 (+596354us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.085822 (+211482us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.111037 (+ 25215us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:05.151442 (+ 40405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.160489 (+  9047us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.172668 (+ 12179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133584,"prepare.queue_time_us":164346,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":4062,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":36661,"thread_start_us":522145,"threads_started":2,"wal-append.queue_time_us":212266}
I20260709 12:19:05.408733 31969 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 9.
I20260709 12:19:05.422575  4321 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:05.422564 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.408152 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.427268 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.432695  4201 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.428503 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:05.436914 32042 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: [12.11-13.12]
I20260709 12:19:05.439548 32043 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:05.426153  4400 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:05.443655 32044 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:05.398746  4405 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:19:05.518643 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:05.520226 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.400756  4354 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:05.524219 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.517143 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 540) took 1041 ms. Trace:
I20260709 12:19:05.524960 32042 rpcz_store.cc:276] 0709 12:19:04.403772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.370475 (+966703us) service_pool.cc:224] Handling call
0709 12:19:05.402694 (+ 32219us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:05.441937 (+ 39243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.445035 (+  3098us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.440505  3991 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.529351 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.516021  4327 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:05.539052 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.542428  3990 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:05.542512  4093 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:05.543090  4093 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.543473  4093 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:05.542506 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.547896 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.532847  4428 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.548790 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.536959  4327 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:05.562915  4391 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:05.563923  4321 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.567837 32115 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:05.541998  4288 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.538499  4345 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.184s
I20260709 12:19:05.578562 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:05.579702 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 223304us
I20260709 12:19:05.559839  3924 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:05.569859  4327 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:05.586439 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 140564us
I20260709 12:19:05.590610  4418 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.591557 32112 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:05.557413  4428 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.605384  4428 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.578389  4391 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.606325  4391 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.609210  4391 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.609831  4391 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.602918 32120 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:05.596549 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.615237  4418 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.595715 32651 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.578644  3765 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.619825 32118 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:05.620357 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.621335 32651 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:05.583791  4415 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.584115 32044 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:19:05.556785  3991 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.641222  4405 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:05.641744  4405 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.642133  4405 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:05.620965  4137 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.652087 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.654024 32112 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:05.654481 32112 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:05.614217  4392 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.662969 32654 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:05.616333  4288 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.665316 32118 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.665809 32118 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:05.666729 32118 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.668087 32118 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.649796  4396 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.671798 32042 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:05.674535 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.653836 32651 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.680536  4392 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.680363 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:05.682628 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:05.635628  3924 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.670668  4396 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.690699 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:05.707031 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.691658  4288 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.691810  4371 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:05.691669 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2276) took 1342 ms. Trace:
I20260709 12:19:05.663515  3944 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:19:05.693216 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72981.4us
I20260709 12:19:05.687009  4155 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:19:05.693030 32651 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.662228  4391 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.713528 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.683071 32112 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.714242 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:05.718055  4155 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:05.719645 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.719882 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.720520 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.724166  4383 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:05.661191  4137 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.710052 32043 rpcz_store.cc:276] 0709 12:19:04.349508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.261968 (+912460us) service_pool.cc:224] Handling call
0709 12:19:05.290367 (+ 28399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.440574 (+150207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.691379 (+250805us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540012fa54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:05.691598 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342535936}
I20260709 12:19:05.730154 32113 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:05.729024 32117 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:05.728868 32042 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:05.729121 32119 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:05.677896  4345 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:19:05.738555 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.710479  4371 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.742192 32044 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:05.743453 32112 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:05.746760 32047 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:05.739737  3495 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.598s	user 0.006s	sys 0.006s
I20260709 12:19:05.731695  4155 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:05.751791 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.751531 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.761507 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:05.743979  4383 consensus_queue.cc:1243] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:19:05.763399  3924 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.764261  3991 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.765938 32042 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:19:05.766849 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 542) took 1362 ms. Trace:
I20260709 12:19:05.767259 32042 rpcz_store.cc:276] 0709 12:19:04.404721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.749380 (+1344659us) service_pool.cc:224] Handling call
0709 12:19:05.765618 (+ 16238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.766495 (+   877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.766752 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.769793 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 544) took 1364 ms. Trace:
I20260709 12:19:05.774467  3986 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:19:05.745716  4371 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:05.757709 32046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2126) took 1340 ms (client timeout 1541 ms). Trace:
I20260709 12:19:05.778714  4288 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.763412  4418 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:05.781971  4434 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:19:05.788951 32042 rpcz_store.cc:276] 0709 12:19:04.405272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.768583 (+1363311us) service_pool.cc:224] Handling call
0709 12:19:05.769041 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.769338 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.769375 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.769450 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:05.769504 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.769575 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.769704 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:05.789861 32046 rpcz_store.cc:269] 0709 12:19:04.411174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.467023 (+ 55849us) service_pool.cc:224] Handling call
0709 12:19:05.742868 (+1275845us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b54001946d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:05.751196 (+  8328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104140160}
I20260709 12:19:05.793802 32118 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:05.798005  4093 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.798620  4329 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:19:05.799077  4375 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:19:05.800115 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54463.9us
I20260709 12:19:05.762444  3495 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.804986 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.805847 32046 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:05.779826  4437 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:05.809917 32043 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:19:05.817639 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2121) took 1408 ms. Trace:
I20260709 12:19:05.817993 32046 rpcz_store.cc:276] 0709 12:19:04.409614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.805145 (+1395531us) service_pool.cc:224] Handling call
0709 12:19:05.805527 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.817402 (+ 11875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.817587 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.780689  3924 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:05.818634 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71159.6us
I20260709 12:19:05.819801 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 541) took 1415 ms. Trace:
I20260709 12:19:05.820228 32043 rpcz_store.cc:276] 0709 12:19:04.404297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.751032 (+1346735us) service_pool.cc:224] Handling call
0709 12:19:05.803614 (+ 52582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.819490 (+ 15876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.819703 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.816350 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:05.824831 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.826673 32115 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:05.826748  4351 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:05.827287  4351 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.827704  4351 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:05.828473  4321 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.829042  4321 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.832480 32117 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:05.834057  4403 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:05.834477  4403 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:05.819319  4437 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:05.835398  4403 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.836115  4437 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:05.836457  4403 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:05.806979  4288 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.839304 32657 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:05.840543  4400 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.827504  4340 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:19:05.811605  4418 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.821671  3991 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.842912  4418 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.842975  3991 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.845793  3991 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.864754  4440 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.815196  4405 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.865528  4440 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.866398 32114 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:05.868268 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 43575.8us
I20260709 12:19:05.871083 32112 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:05.820667  4415 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.892824  4415 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.895624  4415 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.881896  4371 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.875685 32118 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:05.822994  3924 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.876505 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:05.877079 32113 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:05.877826 32046 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:05.878335  4301 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:05.880395 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.880612  4093 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.881044 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:05.882422  4246 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:05.884604 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73910.9us
I20260709 12:19:05.820957  3495 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.867486  4405 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.904340 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2122) took 1494 ms. Trace:
I20260709 12:19:05.918084 32046 rpcz_store.cc:276] 0709 12:19:04.409913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.877251 (+1467338us) service_pool.cc:224] Handling call
0709 12:19:05.877610 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.904171 (+ 26561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.904298 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:05.906963  4432 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.612s	user 0.011s	sys 0.000s
I20260709 12:19:05.899550  4371 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.921881  4249 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:05.934675  4249 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:05.943084 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:05.944015  4201 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:05.944140 31938 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:05.924549  4327 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.944635  4201 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.945011  4201 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:05.946506  4327 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.947541  4399 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:05.948002  4399 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.948410  4399 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:05.948704 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2278) took 1199 ms. Trace:
I20260709 12:19:05.927079  4400 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:05.949100 32045 rpcz_store.cc:276] 0709 12:19:04.749452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.947782 (+1198330us) service_pool.cc:224] Handling call
0709 12:19:05.948130 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.948340 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.948368 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.948427 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:05.948490 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.948546 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.948648 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:05.950470  4430 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.626s	user 0.000s	sys 0.002s
I20260709 12:19:05.922852  4396 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:05.952483  4249 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:05.953552  4201 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.954512 32658 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:05.955323  4240 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
W20260709 12:19:05.955487 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85033.2us
I20260709 12:19:05.955780  4240 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.956347 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.957563 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.958508 32120 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:05.959534  4384 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.005s
I20260709 12:19:05.960532 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:05.961642  4329 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:05.961711 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:05.955889  4201 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.962550  4329 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:06.004621 32659 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:06.004355 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.001309 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:05.964033 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:06.030133 32653 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 8.
I20260709 12:19:06.032246 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2279) took 1257 ms. Trace:
I20260709 12:19:06.033386 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:05.975414 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.998762 32651 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:19:06.037717 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:06.038561 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74609.6us
I20260709 12:19:06.032763 32045 rpcz_store.cc:276] 0709 12:19:04.775104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.950249 (+1175145us) service_pool.cc:224] Handling call
0709 12:19:06.016615 (+ 66366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.031884 (+ 15269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.031914 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.031986 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.032023 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.032087 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.032188 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.943921  4381 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:06.005637 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.001415 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:06.045676 32118 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:05.977566  4440 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:05.992529 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:05.980996  3924 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.050967 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2129) took 1169 ms. Trace:
I20260709 12:19:05.974502  4405 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:05.958019  4240 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:05.992779  4435 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.394s	user 0.001s	sys 0.001s
I20260709 12:19:05.996205 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.059844 32113 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:05.979869  4321 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:05.999040  4155 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:05.967425  3495 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:05.973147  4391 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.010684 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53215.8us
I20260709 12:19:06.060741  4396 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:06.061846  4396 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:05.916033  4301 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.062196  4396 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:06.062479  4415 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.063107  4415 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.063165  4396 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.064056  4301 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.006165  4351 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.064831  4441 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:06.065328  4415 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.065555  4441 consensus_queue.cc:1243] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 16.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:06.044780 32652 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:06.066362  4351 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:06.067572 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:06.046126  4297 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.068760  4297 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.069832  4201 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:06.071492  4201 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.071923  4201 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.072256  4201 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.076974  4137 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.062134 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.079747 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.081360 32044 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:06.081954 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2131) took 1199 ms. Trace:
I20260709 12:19:06.051409 32038 rpcz_store.cc:276] 0709 12:19:04.881662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.050129 (+1168467us) service_pool.cc:224] Handling call
0709 12:19:06.050465 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.050641 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.050668 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.050711 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:06.050743 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.050809 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.050908 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.028365  4400 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.029039  4384 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:06.057579 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 132063us
I20260709 12:19:06.057745  4093 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.057606  4405 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:06.060207 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:06.085449 32040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 543) took 1680 ms (client timeout 2225 ms). Trace:
I20260709 12:19:06.113854  4155 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:06.044423  4381 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:06.115401 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:06.115192  4381 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.116449  4381 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:06.116815 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:06.117426 32038 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:06.117789 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.100294  4340 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:06.082542  4321 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.101318 32652 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:19:06.096408  4433 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.534s	user 0.004s	sys 0.000s
I20260709 12:19:06.082150  4329 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:06.120967 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2272) took 1865 ms. Trace:
I20260709 12:19:06.120540  4400 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.121819  4384 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:06.113675 32040 rpcz_store.cc:269] 0709 12:19:04.404997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.263935 (+858938us) service_pool.cc:224] Handling call
0709 12:19:06.085208 (+821273us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b5400379e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:06.085351 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191011584}
W20260709 12:19:06.087369 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.086891 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.093293 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.113045 32045 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:06.086866 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2271) took 1843 ms. Trace:
I20260709 12:19:06.043582  3986 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:06.082279 32044 rpcz_store.cc:276] 0709 12:19:04.882650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.080750 (+1198100us) service_pool.cc:224] Handling call
0709 12:19:06.081062 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.081730 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.081906 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:06.147207 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.126271 32047 rpcz_store.cc:276] 0709 12:19:04.243649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.128694 (+885045us) service_pool.cc:224] Handling call
0709 12:19:05.746589 (+617895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.751499 (+  4910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.086659 (+335160us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54001946d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:06.086805 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388622720}
I20260709 12:19:06.102321  3495 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
W20260709 12:19:06.126129 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.123962 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.146852 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.084965 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2267) took 2002 ms. Trace:
I20260709 12:19:06.154546 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.154693 32039 rpcz_store.cc:276] 0709 12:19:04.082703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.077643 (+994940us) service_pool.cc:224] Handling call
0709 12:19:05.078127 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.391705 (+313578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.084725 (+693020us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54000339d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:06.084911 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686196096}
I20260709 12:19:06.055449  4345 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:06.124820  4093 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:06.085741  4336 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:06.068825  4421 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:06.185941 32117 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:06.121361 32041 rpcz_store.cc:276] 0709 12:19:04.255522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.132307 (+876785us) service_pool.cc:224] Handling call
0709 12:19:05.186957 (+ 54650us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:05.213524 (+ 26567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.213559 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:05.214376 (+   817us) log.cc:844] Serialized 52 byte log entry
0709 12:19:05.396432 (+182056us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:05.410918 (+ 14486us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.411187 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.411281 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.002079 (+590798us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b1800221aa0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:06.095641 (+ 93562us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:06.120771 (+ 25130us) raft_consensus.cc:1685] finished
0709 12:19:06.120792 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.120914 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56330,"prepare.queue_time_us":17295,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":11644,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":12312,"spinlock_wait_cycles":42720896,"thread_start_us":176251,"threads_started":1,"wal-append.queue_time_us":323557}
I20260709 12:19:06.187783 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:06.196282 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:06.214691 32118 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:06.195820 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78136.5us
W20260709 12:19:06.200259 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 72125.8us
I20260709 12:19:06.176981 32116 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:19:06.107110  4297 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.100143  4418 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.205969 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.193462 32654 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:06.212182 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:06.164402 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:06.212997 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58532.9us
I20260709 12:19:06.197624 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.107053  3935 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:06.161926 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:06.226886 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.090797  4396 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:06.221129 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.224236 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:06.224531 32044 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:19:06.153013  4375 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:06.227314 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.230268 31968 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:06.168594 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:06.178921 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.204828  4345 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:06.246680 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 91325.6us
I20260709 12:19:06.223999 32115 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:06.214007  4421 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:06.221771  4336 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:06.241425 32038 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:06.225843  4329 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:06.258502 32039 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:06.260095 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.245479  4440 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.262058 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:06.262246  4440 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.262763 31969 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:06.264451  4330 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.225832  3935 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:06.205209  3986 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.265560 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.235081  4391 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.240460  4375 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.301827 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.245832  3924 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:06.298727 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.248123  4345 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.302467 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.245293  4321 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:06.312423 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62954.8us
I20260709 12:19:06.316332 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.257128  4329 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.317034 31966 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:06.300526  4371 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.321960  3935 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.272953  4392 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.329068 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2277) took 1581 ms. Trace:
I20260709 12:19:06.322190  4439 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:19:06.325767  4436 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.326169  4446 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.341253  4446 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.326629  4450 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:06.342659 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.343362 31964 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:06.344312 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.343899  4436 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.345420 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.326849  3986 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.305289  4375 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:06.346635 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.342651  4450 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.347263 31967 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:06.348161 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.348726 31967 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:06.349377  4450 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.349713 31967 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:06.350864 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.351167 32041 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:06.330238  4371 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.352164 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.352484  4371 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.352850 31971 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:06.353111 32041 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:06.321848  4391 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.355746  4391 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.356573 32040 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:06.357059  4446 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.359176  4436 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.265079  4330 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.363960  4330 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.364732 31969 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:06.366356  4399 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.366959  4384 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.304836  4093 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:06.368381  4437 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.004s
I20260709 12:19:06.368853  4437 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.369212  4093 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.370563 32042 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:19:06.371443  3935 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.371232  4371 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.374053  4371 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.378608 31972 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:06.331815 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.329488 32046 rpcz_store.cc:276] 0709 12:19:04.748094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.968019 (+1219925us) service_pool.cc:224] Handling call
0709 12:19:05.968446 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.968747 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:06.039992 (+ 71245us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.314730 (+274738us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.328757 (+ 14027us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.328804 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.328847 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.328908 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.329011 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21664,"prepare.queue_time_us":118625,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":11871,"spinlock_wait_cycles":2623744,"thread_start_us":251782,"threads_started":2}
I20260709 12:19:06.324205  4329 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:06.391775 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:06.308975  4403 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:19:06.380162 31966 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
W20260709 12:19:06.403446 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.414820  4399 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.417564  4375 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.417881  4392 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.432821 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:06.419008 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:06.419605 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 154529us
I20260709 12:19:06.314363  3924 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.330386  4351 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:06.419982 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:06.425683 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.319811  4321 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.431720 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.439744 32657 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:06.434928 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:06.418627 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:06.436090 32046 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:06.332623  4421 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.442206 31964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:06.432973  4439 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:06.443003 31964 raft_consensus.cc:606] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:06.443332  4439 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.445386  4439 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.447559  4438 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.188s	user 0.003s	sys 0.000s
I20260709 12:19:06.450614  4439 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.451017  4439 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.451399  4439 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.340662  4336 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.455206 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:06.456040  4247 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:06.456457 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:06.457038  3990 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:06.457559  3990 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:06.458735  4351 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:06.458516  3990 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.459121  4351 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.459506  4351 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:06.456537  4247 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:06.462518  4247 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.468968  4439 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:06.469520 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.471561  3944 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:06.472507  4121 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:06.472759  3935 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:06.473157  3935 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:06.474118  3935 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.423444  4437 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.458055 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:06.477192 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 221761us
I20260709 12:19:06.478495 32118 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:19:06.483863 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 179783us
I20260709 12:19:06.484476 32656 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:19:06.488903 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:06.493548 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.496410 31936 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:06.497673 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:06.447561  4392 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.480731  4399 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.502954 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:06.489449  4441 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:19:06.497429  4155 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:19:06.499223  3990 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:06.491050  3935 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:06.508455 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.518020 32119 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:06.510591 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:06.522795 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.523958 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.515309 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.530520 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.557229  4392 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:06.559037 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.559841 32115 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:06.561851  4437 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:06.563463 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:06.566409 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51020.5us
I20260709 12:19:06.567224  4340 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:06.527932  3986 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.568980  4340 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.569267  3986 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.569732  3986 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.569671  4340 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.572427  4340 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.573405 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57691.3us
I20260709 12:19:06.511097  3990 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:06.574854  4246 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:06.575259  4246 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:06.574153  4155 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:06.557364 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.674620 32117 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:06.676695 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 102671us
W20260709 12:19:06.677026 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 178987us
I20260709 12:19:06.509737  4400 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.682044 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.504848  4463 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:06.694913 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 127253us
I20260709 12:19:06.708420 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:06.514353  4405 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.167s
I20260709 12:19:06.715968 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2127) took 2304 ms. Trace:
I20260709 12:19:06.716828 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:06.715216 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 184737us
I20260709 12:19:06.526298  4093 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.169s
I20260709 12:19:06.508183  4329 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.743464 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:06.744623 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:06.716621  4201 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
I20260709 12:19:06.575398  3990 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.697841  4288 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.528420  4399 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.733054 32043 rpcz_store.cc:276] 0709 12:19:04.411471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.871903 (+1460432us) service_pool.cc:224] Handling call
0709 12:19:05.886779 (+ 14876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.132138 (+245359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.132165 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.153562 (+ 21397us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.476947 (+323385us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.476986 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.477057 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.715746 (+238689us) raft_consensus.cc:1685] finished
0709 12:19:06.715766 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.715908 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9898,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":463206,"threads_started":2,"wal-append.queue_time_us":253066}
I20260709 12:19:06.716777  4436 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.710256  4439 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.755740  4465 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 12:19:06.756098  4396 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.004s
I20260709 12:19:06.746841  4201 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:06.779367  4457 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.440s	user 0.004s	sys 0.006s
I20260709 12:19:06.779726 32658 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:06.781868 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:06.789518  4392 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.825590  4399 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.825376  4392 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.827606  4392 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.827983  3924 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:06.825913 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2288) took 1425 ms. Trace:
I20260709 12:19:06.828725 32038 rpcz_store.cc:276] 0709 12:19:05.401022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.258475 (+857453us) service_pool.cc:224] Handling call
0709 12:19:06.258745 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.390717 (+131972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.390764 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.390823 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:06.390881 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.825691 (+434810us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001a5ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:06.825736 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.825856 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302708608}
I20260709 12:19:06.830808 32041 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:06.794629  4464 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:06.795794 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2280) took 1930 ms. Trace:
I20260709 12:19:06.836326 32045 rpcz_store.cc:276] 0709 12:19:04.868395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.039503 (+1171108us) service_pool.cc:224] Handling call
0709 12:19:06.112860 (+ 73357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.126988 (+ 14128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.795577 (+668589us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b5400379e50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:06.795727 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137055616}
I20260709 12:19:06.764160  4437 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:06.801132 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:06.802233 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2120) took 2393 ms. Trace:
I20260709 12:19:06.837669 32042 rpcz_store.cc:276] 0709 12:19:04.409275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.804121 (+1394846us) service_pool.cc:224] Handling call
0709 12:19:05.889007 (+ 84886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.370910 (+481903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.801993 (+431083us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540037a854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:06.802175 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106839936}
I20260709 12:19:06.764631  4400 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:06.846263 31963 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:06.852855 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2281) took 1888 ms. Trace:
I20260709 12:19:06.853027 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.853914 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:06.854696 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.856213 32120 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:06.841914 32046 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:06.861099 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.862001 32117 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:06.862561 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.863730 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.864845 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.756249  4246 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.866927 31970 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15.
I20260709 12:19:06.831652  4440 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.847426  4381 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.720s	user 0.013s	sys 0.001s
I20260709 12:19:06.871908  4201 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:06.878651 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.879606 31964 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:06.861187 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.883812 32119 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:19:06.885519 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:06.891019 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.892024 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.893332 32115 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:19:06.897441 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.898690 32120 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:19:06.913039 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:06.855791  4400 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:06.916785 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 168673us
I20260709 12:19:06.917863 32120 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:06.924549 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:06.924746 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:06.925123 32041 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.926136 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:06.859272  4464 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:06.926919 32045 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:06.931955 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.830236  3986 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:06.853291 32047 rpcz_store.cc:276] 0709 12:19:04.964959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.148674 (+1183715us) service_pool.cc:224] Handling call
0709 12:19:06.199065 (+ 50391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.199280 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:06.199589 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.778780 (+579191us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:06.813657 (+ 34877us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b1800463ba0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a
0709 12:19:06.852514 (+ 38857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.852575 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:06.852623 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.852691 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.852802 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5564,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":75,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"spinlock_wait_cycles":39039616,"thread_start_us":48861,"threads_started":1,"wal-append.queue_time_us":100485}
I20260709 12:19:06.910262  4375 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:06.942425 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.855697  4437 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.960945  4437 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:06.963287 32038 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:06.964687 32038 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:06.941802 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:06.944525  4392 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:06.948738  3924 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:06.966830  4321 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:06.968037  4468 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:06.927520  4400 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.922812  4415 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:06.929898  4440 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:06.947372 32660 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:06.949216 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.980399 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:06.923529  4351 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:06.987149 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:06.988518 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.982139 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:06.992311 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:06.984364 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:06.994068 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:06.995848 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2291) took 1256 ms. Trace:
I20260709 12:19:06.945906 32043 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 2 to 7, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:06.999421 32042 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:07.000635 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:07.003037 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:06.975445  4428 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.004148 32045 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:07.005264 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:07.008733 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:06.997820 32044 rpcz_store.cc:276] 0709 12:19:05.739832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.302327 (+562495us) service_pool.cc:224] Handling call
0709 12:19:06.313124 (+ 10797us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:06.313478 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.313509 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:06.328780 (+ 15271us) log.cc:844] Serialized 52 byte log entry
0709 12:19:06.487408 (+158628us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:06.588608 (+101200us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.588978 (+   370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.589097 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.995599 (+406502us) raft_consensus.cc:1685] finished
0709 12:19:06.995622 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.995786 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":766,"prepare.queue_time_us":39772,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":44106,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":46175,"thread_start_us":145012,"threads_started":1,"wal-append.queue_time_us":140519}
I20260709 12:19:06.995836  4392 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:06.983918  4415 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:06.998693  4437 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.003162  4375 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:07.003898  4321 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.008190  3924 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:06.998217  4121 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.030572  4121 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.031083  4121 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.031555  3986 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.023363  4321 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.032779  4321 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.021904 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.035353 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.036386 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.037035 32045 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 11: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:07.015980 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.016839 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:07.007268  4440 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.017593  4418 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:07.039582  4418 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:07.023211  4375 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.043206  4121 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.044291  4375 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.045701  4121 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:07.047873 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.048362 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.054247  4440 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.060410  4464 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.013018  4428 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.013363  4288 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.078918  4288 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.079429  4288 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.084403  4415 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.020771  4437 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:07.087373  4437 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.091856  4460 raft_consensus.cc:993] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:19:07.092466  4460 raft_consensus.cc:1081] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
W20260709 12:19:07.095928 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.098727 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 172629us
I20260709 12:19:07.099385 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:07.099557 31970 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.100764  4445 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:07.101261  4445 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:07.102770 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.102833 31963 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.102139  4445 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:07.108556 32660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 729) took 1828 ms (client timeout 1930 ms). Trace:
W20260709 12:19:07.109050 32660 rpcz_store.cc:269] 0709 12:19:05.279718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.279871 (+   153us) service_pool.cc:224] Handling call
0709 12:19:07.108284 (+1828413us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400154d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:07.108440 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3718086528}
I20260709 12:19:07.113641  4439 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:07.114246  4439 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.115280 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2284) took 1744 ms. Trace:
I20260709 12:19:07.115665 32039 rpcz_store.cc:276] 0709 12:19:05.372702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.198419 (+825717us) service_pool.cc:224] Handling call
0709 12:19:06.258333 (+ 59914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.258883 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.115080 (+856197us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b540032a850. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:07.115224 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136977792}
I20260709 12:19:07.119369  4354 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:07.119884  4354 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:07.120965  4354 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.124225  4421 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.137549 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:07.138303 32658 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.139567  3986 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:07.139935  4137 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.140321 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:07.140937 32659 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.140831  4137 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.145712 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:07.134080  4452 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.766s	user 0.004s	sys 0.021s
I20260709 12:19:07.146528 31972 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:07.147573 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 909) took 1146 ms. Trace:
I20260709 12:19:07.148425 31972 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:07.148826 31972 pending_rounds.cc:85] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:19:07.149037 31972 pending_rounds.cc:107] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 15.12
W20260709 12:19:07.149210  4455 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.791s	user 0.017s	sys 0.011s
I20260709 12:19:07.149302 31972 raft_consensus.cc:2889] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:07.151190 32652 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.151937  4336 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.122066  4354 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.147943 32655 rpcz_store.cc:276] 0709 12:19:06.000654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.004904 (+  4250us) service_pool.cc:224] Handling call
0709 12:19:06.005300 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.005779 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.005844 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.006374 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.124695 (+118321us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.124732 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.124792 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.133381 (+1008589us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:07.147201 (+ 13820us) raft_consensus.cc:1685] finished
0709 12:19:07.147220 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.147393 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":50642176,"thread_start_us":117689,"threads_started":1,"wal-append.queue_time_us":321333}
I20260709 12:19:07.164999 32120 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.168397  3924 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.169098  3924 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.193377  4329 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:07.195366 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:07.196113 32655 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:07.197290 32652 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.198993 32652 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:19:07.203642  3924 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.205317  4375 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:07.205523 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.205847  4375 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.209129  4137 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.211259  4375 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.220185  4351 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:07.222352  4345 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.902s	user 0.008s	sys 0.004s
I20260709 12:19:07.222973  4345 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.224373 31967 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:07.233395 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.234040 32046 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.239857 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.240856 31969 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:07.246871 32115 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:07.252717 31970 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:07.256369 32112 raft_consensus.cc:1217] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Deduplicated request from leader. Original: 14.11->[15.12-15.12]   Dedup: 15.12->[]
I20260709 12:19:07.257082 32112 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:07.258916 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:07.262732 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.263490 32040 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:07.267117 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.267972 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:07.264240 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:07.220459 32011 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.930s (SENT) [suppressed 4 similar messages]
I20260709 12:19:07.285426 32660 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:07.285543 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:07.277189 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:07.287328 32623 leader_election.cc:400] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15.
I20260709 12:19:07.287812 32623 leader_election.cc:403] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:07.288728 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55965.2us
I20260709 12:19:07.291020 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:07.295156 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.296325 31972 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:07.298750 31970 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:19:07.255563  4383 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.038s	sys 0.041s
I20260709 12:19:07.303743 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.299129 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.305101 32658 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.307343 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:07.312955 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 92525.5us
I20260709 12:19:07.265002  4460 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.311914 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:07.314137 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:07.318610 32656 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:19:07.316460 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.316632 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:07.322077 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 310120us
W20260709 12:19:07.303606  4383 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.038s	sys 0.041s
I20260709 12:19:07.330816 32046 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:07.347343 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.325525 32041 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:07.334584 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:07.317203  4436 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.008s
I20260709 12:19:07.337595 32659 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.354758 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:07.341018 32116 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:07.341056 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66218us
I20260709 12:19:07.356289 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.262449  4351 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:19:07.310773  4327 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15.
W20260709 12:19:07.343182 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.001s	sys 0.009s
W20260709 12:19:07.365271 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.001s	sys 0.009s
W20260709 12:19:07.365568 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 607183us
I20260709 12:19:07.312750  4375 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.357420 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.368938 31963 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:07.306221  3991 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:19:07.359182 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.377401 32118 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.303843  4247 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:19:07.370518 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.325237  4428 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.369750 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:07.390754 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 96287.6us
I20260709 12:19:07.332080  4421 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:19:07.364845  4436 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:07.400182 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:07.325361  4460 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.397619 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:07.398732 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 138260us
I20260709 12:19:07.400789 32039 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:07.413939 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.414568 32039 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.367308 32120 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:07.401659 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:07.402927 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.403434 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:07.403967  4403 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.405519  4439 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.416059 32045 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.341802  3924 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:19:07.415288 32044 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:07.418429 31970 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:07.387542  4405 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:07.424176 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44902.9us
I20260709 12:19:07.426745 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:07.420243 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:07.430191 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.378608  4375 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.434211 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:07.391641  4473 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:07.432838 32046 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.435305 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:07.434850 32044 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.443411 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:07.445310 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.455040 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.456014 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.456620 32118 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:07.417985  3990 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:07.499661 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.513770 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.524458 31963 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:07.453044  4436 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:07.447463  4473 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:07.454823  4428 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:07.533108 32039 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.533788 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.530560 32040 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:07.473583  4288 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.475031 31972 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:07.477080  4345 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:07.483522  4400 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
I20260709 12:19:07.393303  3944 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 12:19:07.485982 31870 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 244) took 22325 ms (client timeout 15000 ms). Trace:
I20260709 12:19:07.490383  4375 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:07.492353 31866 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 11, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 11 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.494786  4471 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.495745 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:07.431275  4403 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.406286  4321 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.463040 32046 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:07.530953 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.398518  4381 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:19:07.540340 32044 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:19:07.496027  3924 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:07.542949 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:07.578776 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.579425 32118 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:07.579802 31972 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:07.537879 32043 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:19:07.568984 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.580602 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:07.578514 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:07.539047  4458 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:07.570907 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:07.574963 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 119998us
I20260709 12:19:07.543823 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.569694 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.527377  4392 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:07.527194  4479 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:07.585428 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.591827 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.594419 32657 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:07.588376 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 150026us
I20260709 12:19:07.599592 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.600054 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:07.601326 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.536329  4327 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:19:07.578596  4246 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.519402  3990 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.568075  4436 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:07.575167  4418 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.577098  3991 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.579071  3924 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:07.608783 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:07.581784  4428 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:07.578718 32045 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:19:07.610270 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.612105 32656 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:07.610981 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:07.692768 32045 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:07.693229 32045 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:07.574721  4155 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:19:07.660205 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.612697 31866 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 7 to 10, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:07.595206  4441 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:07.613310 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.664764 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.606738  4440 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:07.576543  4400 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.607071  4399 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.691488 32659 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:07.703773 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 92903.1us
I20260709 12:19:07.705096 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:07.715667 31963 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:07.578625  4471 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.594786  4392 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.719643 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:07.723634 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:07.725062 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.729238 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:07.730624 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:07.761551 31971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:07.732640 32045 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.701376  4441 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:07.774477 32047 raft_consensus.cc:2368] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16.
I20260709 12:19:07.707804  4375 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:07.775465 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60133.5us
I20260709 12:19:07.779747 32659 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
W20260709 12:19:07.781649 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51116.3us
I20260709 12:19:07.782397 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:07.701848  4418 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.735843  4415 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.769490  4329 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.775545  3944 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:07.813341 31866 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.769261 32085 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:07.851384  4381 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:07.851950  4491 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:19:07.762866  3986 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:07.854441  4336 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.000s
I20260709 12:19:07.854514  3986 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.855029  3986 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.813721  4201 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.859436 31968 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
I20260709 12:19:07.859936 31968 pending_rounds.cc:85] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 6
I20260709 12:19:07.860203 31968 pending_rounds.cc:107] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 12:19:07.860440 31968 raft_consensus.cc:2889] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:07.862289  4502 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:19:07.862931  4482 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:19:07.770380  4440 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.863794  4440 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:07.864073  4492 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:19:07.864249  4440 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:07.864884  4490 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:19:07.814033 32045 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:07.867555  4496 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:19:07.851893  4381 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:07.868978  4381 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.869154  4441 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.870296  4441 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:07.874850  4488 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 12:19:07.875859  4428 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.845964  4472 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.846498  4329 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.879014  4329 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:07.881065 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 180285us
I20260709 12:19:07.842954  4415 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:07.882560  4415 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.884228  4392 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.885051  4392 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.878295  4472 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.612877  4473 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:07.862474  4489 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:19:07.914208 31866 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 2 to 7, term changed from 8 to 9, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to <none>, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1), VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2), VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3), VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 9 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:07.917630  4288 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.820839 32660 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:19:07.845925  4485 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
W20260709 12:19:07.845842  4480 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:19:07.816732  4460 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:07.548080 31870 rpcz_store.cc:269] 0709 12:18:45.160837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.165980 (+  5143us) service_pool.cc:224] Handling call
0709 12:19:07.485767 (+22319787us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:01.808002 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:01.808711 (+   709us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:01.808745 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:19:01.808762 (+    17us) tablet.cc:662] Decoding operations
0709 12:19:01.865129 (+ 56367us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:01.871906 (+  6777us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:01.884954 (+ 13048us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:19:02.419870 (+534916us) tablet.cc:701] Row locks acquired
0709 12:19:02.419907 (+    37us) write_op.cc:260] PREPARE: finished
0709 12:19:02.420057 (+   150us) write_op.cc:270] Start()
0709 12:19:02.420177 (+   120us) write_op.cc:276] Timestamp: P: 1783599542420009 usec, L: 0
0709 12:19:02.420201 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:19:02.420905 (+   704us) log.cc:844] Serialized 5841 byte log entry
0709 12:19:03.043436 (+622531us) op_driver.cc:464] REPLICATION: finished
0709 12:19:03.335100 (+291664us) write_op.cc:301] APPLY: starting
0709 12:19:03.339421 (+  4321us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:03.552386 (+212965us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:03.552520 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:03.638708 (+ 86188us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cc32a 00007f228c151f14 00007f228c151ec3 00007f228c151e38 00007f228c151df3 00007f228c1510dc 00007f228c17ae16 00007f228c162cbb 00007f228c14d39d 00007f228c150d75
0709 12:19:03.769051 (+130343us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:03.845216 (+ 76165us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 12:19:03.845962 (+   746us) write_op.cc:312] APPLY: finished
0709 12:19:03.851578 (+  5616us) log.cc:844] Serialized 139 byte log entry
0709 12:19:03.860122 (+  8544us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:03.871903 (+ 11781us) write_op.cc:454] Released schema lock
0709 12:19:03.902189 (+ 30286us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:03.925913 (+ 23724us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:03.925942 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":514916,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":257491,"apply.run_cpu_time_us":30733,"apply.run_wall_time_us":602372,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66767,"num_ops":15,"prepare.queue_time_us":1450506,"prepare.run_cpu_time_us":20990,"prepare.run_wall_time_us":755905,"raft.queue_time_us":239559,"raft.run_cpu_time_us":33677,"raft.run_wall_time_us":752294,"replication_time_us":590855,"spinlock_wait_cycles":36821760,"thread_start_us":1363602,"threads_started":4,"wal-append.queue_time_us":161848,"wal-append.run_cpu_time_us":8581,"wal-append.run_wall_time_us":2374819}]]}
I20260709 12:19:07.896613  4472 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.898159  4471 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.898576  4321 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.842949  4494 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:07.899828  4497 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:07.900170 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:07.935068 32115 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.884346  4240 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:19:07.901618  4400 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:07.906608  4403 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:07.906512  3944 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.819223  4436 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.907608  4473 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.907739  4499 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:07.909737  4437 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.000s
I20260709 12:19:07.890940  4093 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:07.819042  3924 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:07.915990  3990 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:07.850129 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 83155.2us
I20260709 12:19:07.818706  4375 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:07.935351 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 210420us
I20260709 12:19:07.898633  4405 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:19:07.931401  4464 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.980072 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:07.993043 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.993173 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2296) took 1615 ms. Trace:
I20260709 12:19:07.995111 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:07.990749  4464 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:07.996428 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.001515 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.003381 32115 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:07.973096  4093 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.993793 32043 rpcz_store.cc:276] 0709 12:19:06.377580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.804830 (+427250us) service_pool.cc:224] Handling call
0709 12:19:06.837449 (+ 32619us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:06.899950 (+ 62501us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:07.580488 (+680538us) log.cc:844] Serialized 14 byte log entry
0709 12:19:07.966485 (+385997us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:07.972441 (+  5956us) log.cc:844] Serialized 78 byte log entry
0709 12:19:07.972603 (+   162us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:07.972639 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.972699 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.992891 (+ 20192us) raft_consensus.cc:1685] finished
0709 12:19:07.992919 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.993100 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14021,"raft.queue_time_us":205891,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":14770,"spinlock_wait_cycles":9245440,"thread_start_us":349385,"threads_started":2,"wal-append.queue_time_us":302349}
W20260709 12:19:08.012506 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 127378us
I20260709 12:19:07.973239  4499 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.018543  4499 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.019133  4499 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.019778  4499 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:08.020583  4499 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.022360 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:07.996663  4400 raft_consensus.cc:2764] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:07.962954  4494 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.024024 32115 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.025868 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.029440 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2405) took 1131 ms. Trace:
I20260709 12:19:07.966929  4497 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.021225  4499 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:08.032259 32112 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:08.030725 32119 rpcz_store.cc:276] 0709 12:19:06.891270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.903842 (+ 12572us) service_pool.cc:224] Handling call
0709 12:19:06.904180 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.939056 (+ 34876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.939093 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.976177 (+ 37084us) log.cc:844] Serialized 26 byte log entry
0709 12:19:07.321832 (+345655us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:07.321869 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.321934 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.022746 (+700812us) raft_consensus.cc:1685] finished
0709 12:19:08.022774 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.022973 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15717,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":16002,"thread_start_us":308881,"threads_started":1,"wal-append.queue_time_us":412361}
I20260709 12:19:07.975301  4437 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.011952  4093 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.038846 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.040258 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:08.040838 32116 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 16
W20260709 12:19:08.039695 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 257378us
I20260709 12:19:08.042505 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:08.039520 32119 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:08.044184 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.045969 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.002468  4288 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:19:08.049443  4437 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.050997 32038 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:08.106770  3991 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:08.107048  4506 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.107260  3991 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:08.108346 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2137) took 1131 ms. Trace:
I20260709 12:19:08.108724 32038 rpcz_store.cc:276] 0709 12:19:06.976941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.984017 (+  7076us) service_pool.cc:224] Handling call
0709 12:19:08.108164 (+1124147us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001a72d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:08.108289 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2431334272}
I20260709 12:19:08.051453 31971 raft_consensus.cc:606] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:08.109771 31971 raft_consensus.cc:614] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:08.053375  4511 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.054682 31936 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:08.110608 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55991.4us
I20260709 12:19:08.055709 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:08.112094 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.112905 32113 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
W20260709 12:19:08.115533 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59907.8us
I20260709 12:19:08.057453  4381 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.033169  4240 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:08.058229  4472 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.046438  4466 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:08.120675 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:08.121308 32657 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:08.063239  4375 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:08.067858  4464 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.007238  4415 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.082356  4441 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.159026 32116 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:08.084595  4351 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:08.085841  4493 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:08.089972  4439 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:08.091112  4507 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:08.092432 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2327) took 1751 ms. Trace:
I20260709 12:19:08.167852 31968 rpcz_store.cc:276] 0709 12:19:06.341268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.391521 (+ 50253us) service_pool.cc:224] Handling call
0709 12:19:06.412722 (+ 21201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.861206 (+1448484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.092161 (+230955us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54000b6b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:08.092346 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3325936512}
I20260709 12:19:08.094538  4497 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.014365  3990 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:19:08.094748 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2134) took 1163 ms. Trace:
I20260709 12:19:08.171648 32047 rpcz_store.cc:276] 0709 12:19:06.939346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.947781 (+  8435us) service_pool.cc:224] Handling call
0709 12:19:08.094492 (+1146711us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54001d0954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:08.094666 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780634496}
I20260709 12:19:08.099323  3944 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.101280  4494 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:08.105014  4329 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.119266 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.116993 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:08.125675 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 310614us
W20260709 12:19:08.178143 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61226us
I20260709 12:19:08.128049 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:08.133219 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 132003us
I20260709 12:19:08.179111 32113 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.179646 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.181092 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.178588 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:08.133563 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:08.182732 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
W20260709 12:19:08.134081 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.099326  4418 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.139883 32115 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:08.166065 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:08.118043  4246 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.119028  4396 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.120715  4466 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.173199  4439 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:08.120997  4240 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:08.182186 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.161237  4441 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.188947 32042 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:08.117286  4511 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:08.190881 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 177424us
I20260709 12:19:08.163671  4351 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:08.187062  4340 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:08.211635  4351 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.212635  4351 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.212350  4340 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.214622 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:08.216796 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:08.217885 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:08.169832  4472 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:08.218971  4472 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.173362  4507 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:08.220667  4507 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.221452 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:08.221721 32659 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:08.222219 32657 raft_consensus.cc:2368] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13.
I20260709 12:19:08.226408  4469 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:08.226784  4345 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:08.227083  4469 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.227782  4469 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.229806  4469 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.187109  4464 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.231122 32113 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:08.231354  4464 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.231819  4464 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:08.201402 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 82202.8us
I20260709 12:19:08.234880  3924 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:08.159968  4383 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:19:08.191293  4497 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.208294  4246 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:08.209990 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.210705 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:08.210857  4354 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.196036 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:08.199430  4466 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.187755  4506 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:08.239459 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.284560 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56799.6us
W20260709 12:19:08.248016 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65933.3us
I20260709 12:19:08.287818 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.289482 32113 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.290964 32119 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.291810 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.292444 32112 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:08.205098  4441 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.249369 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:08.207931  4509 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:08.193871  4437 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:08.272995 32041 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:08.221822  4507 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.161415  3991 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.281538 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:08.333442 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45770.3us
I20260709 12:19:08.334605 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.227214  4345 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:08.248517 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 59860.2us
I20260709 12:19:08.217022  3986 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.274078 32011 leader_election.cc:400] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13.
I20260709 12:19:08.286912 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.313522 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.299582 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:08.330061 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 165776us
I20260709 12:19:08.274530 32658 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:19:08.300771 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.281461  4340 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.263741  4399 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:08.375442 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:08.397715 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.375212 31969 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:08.378453 32119 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:08.490219 32115 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:08.339932 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.438669 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:08.434738 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 224824us
I20260709 12:19:08.492360 32116 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:08.371081 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.493957 32117 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:08.351161 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:08.487977 32657 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:08.495570 32011 leader_election.cc:403] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:08.494082 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 207245us
I20260709 12:19:08.350946 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:08.489349 31964 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
W20260709 12:19:08.497228 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 146371us
I20260709 12:19:08.341579 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:08.489444 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.509500 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 235543us
I20260709 12:19:08.500540 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.527702 32660 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 12:19:08.527937 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 186460us
I20260709 12:19:08.534458 32121 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:08.542073 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:08.525653 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.519270 31966 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:08.281641  4486 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:08.519914 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:08.472958  4520 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:19:08.553012 31971 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:19:08.553885 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61171.8us
I20260709 12:19:08.284106  4354 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.338413  4499 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:08.434052  4471 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.338889  4493 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.565663 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.401592  4383 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.430580  4400 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:08.498728  4396 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:08.335901  4375 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.218s
I20260709 12:19:08.516777  4497 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:08.458993  4469 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:08.571821 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 832) took 1098 ms. Trace:
I20260709 12:19:08.517345  4473 raft_consensus.cc:2764] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:08.343586  4515 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:08.578111 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 174596us
I20260709 12:19:08.314252  4418 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.232s
I20260709 12:19:08.523872  4508 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.298966  4440 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.402163  4509 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:08.539597  4415 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:08.615340 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49761.4us
I20260709 12:19:08.611141  4471 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:08.615263 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.336521  4405 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.224s
I20260709 12:19:08.545401  4392 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:08.553012  4288 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:08.717904 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 198071us
I20260709 12:19:08.720206 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:08.437094  4464 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.332827  4506 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.518013  4399 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:08.666437 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.651697 32658 rpcz_store.cc:276] 0709 12:19:07.471559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.471919 (+   360us) service_pool.cc:224] Handling call
0709 12:19:07.472207 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.493459 (+1021252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.569969 (+ 76510us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54004c27d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:08.570228 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844459392}
I20260709 12:19:08.458019  4345 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.441098  4246 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.410610  4240 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.7, Last appended by leader: 7, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.743979 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:08.709347 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 208891us
W20260709 12:19:08.687989 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 139395us
I20260709 12:19:08.746721  4517 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 12:19:08.747476 32115 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:19:08.747834 32115 pending_rounds.cc:85] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:19:08.748047 32115 pending_rounds.cc:107] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:19:08.748281 32115 raft_consensus.cc:2889] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.717463  4471 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.644196  4400 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.642043  4516 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:08.651675  3990 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:08.645282  4497 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.559072  4486 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.653474  4375 consensus_queue.cc:1243] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:08.721153 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:08.723976 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:08.764756 32038 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:08.765090 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:08.687608  4479 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.685477  4329 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.687247  4396 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.619488  4524 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:19:08.620216  4383 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.683351  4469 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:08.615576  4493 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.746053 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.752170 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:08.720077  4392 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.720302  4405 consensus_queue.cc:1243] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:19:08.715054  4511 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:08.762468 32655 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:08.766422 32121 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:19:08.813639 32121 pending_rounds.cc:85] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 11
I20260709 12:19:08.814023 32121 pending_rounds.cc:107] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:19:08.814349 32121 raft_consensus.cc:2889] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.723270  4421 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:08.680696  4499 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:08.744894 32044 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:08.808377 32114 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:08.809645 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 88562.8us
I20260709 12:19:08.763227  4516 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.768187  4497 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:08.811614 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65646.2us
I20260709 12:19:08.820312 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.820837 32119 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:19:08.803447  4415 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:08.803269  4515 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:08.821242 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:08.810029  4493 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:08.806490  4512 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.351s	user 0.001s	sys 0.000s
I20260709 12:19:08.876066 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:08.774078  3990 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:08.865976 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:08.804273  3986 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:08.769256  4534 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:08.822273 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.774617  4436 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:08.809686  4392 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.899168  4381 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.871150 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:08.870572 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:08.872031 32043 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.877041 32121 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:08.879547  4530 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.880300  4535 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:08.880940  4532 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:08.881872  4523 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:19:08.883471  4399 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:08.868095  3924 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.000s
I20260709 12:19:08.884768  4400 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:08.885452 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 683) took 1060 ms. Trace:
I20260709 12:19:08.923804 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.885946  4533 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:08.888306 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:08.928694 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:08.887166  4093 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.823s	user 0.008s	sys 0.004s
I20260709 12:19:08.911994 31969 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:08.914479 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.870292  4516 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.919684 32047 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:08.923542 32118 rpcz_store.cc:276] 0709 12:19:07.825586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.825721 (+   135us) service_pool.cc:224] Handling call
0709 12:19:07.826193 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.826825 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.826860 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:07.856206 (+ 29346us) log.cc:844] Serialized 26 byte log entry
0709 12:19:08.123996 (+267790us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:08.124036 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.126364 (+  2328us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.885216 (+758852us) raft_consensus.cc:1685] finished
0709 12:19:08.885239 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.885381 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3624,"raft.queue_time_us":14244,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":40133,"thread_start_us":267394,"threads_started":1,"wal-append.queue_time_us":479266}
I20260709 12:19:08.928076 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:08.891570 32044 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:08.939693 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:08.872325  4421 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.941839 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:08.925356  4473 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:08.944841 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:08.947535 32118 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:08.900774  4415 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.963260 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:08.949693 32011 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:08.919070  4354 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.966495 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:08.967101 32047 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.894783  4515 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.961653  4473 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:08.939627  4400 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.878655  4509 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.962448 32043 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:08.921731  4436 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:08.959148  4383 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.931681  4534 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:08.934888  4329 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:08.969038 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:08.921236  4530 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.942849  4535 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:08.979388 31969 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.993757 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:08.945518  4440 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.983916 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:08.996285 31972 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:08.969190 32114 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:08.937388  4093 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:08.992722 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:09.002259 32043 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:08.970665  4288 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:08.998255 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:08.987713  4464 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.010597 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:08.992038  4329 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.003350  4545 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.014669  4473 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.014997  4545 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.016256  4329 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.017391  4545 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:08.935680  4533 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.005645  4375 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:09.019475  4375 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.019853  4533 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:09.020779  4436 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.021682  4375 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:08.980805  4400 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.024345  4400 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.024559  4535 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.026748  4534 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.027987  4421 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:09.029701  4466 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.696s	user 0.006s	sys 0.004s
I20260709 12:19:09.030093  4515 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:09.030145  4466 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.030869 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:09.031502 32657 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:09.032407 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:09.033113 32038 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:09.033310 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:09.034286  4515 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:09.034684  4515 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.035032  4515 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:09.036454  4521 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.487s	user 0.003s	sys 0.000s
W20260709 12:19:09.037608  4441 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.002s
I20260709 12:19:09.036365 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:09.038431 32657 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:09.038589 31968 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:09.041298  4507 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.000s
I20260709 12:19:09.042105  4469 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:08.974035  4536 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:09.043511  4093 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:08.985715  3990 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.044343 32120 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:09.045475 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2414) took 1018 ms. Trace:
I20260709 12:19:09.046684 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:08.999704  4506 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:09.047588 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.048171  4486 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.049351 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:09.049990 32039 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:09.050982 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:09.051959  4494 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:09.052376  4494 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:09.054121  4494 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.038102  4415 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:09.055606  4494 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:09.056107  4515 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.056985  4497 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.057682 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:09.058408  4533 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:09.058943  4533 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.059298 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:09.059914 32043 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:09.060004 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:09.061041  4533 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:09.061378 32043 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:19:09.061473  4506 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:09.062127 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:09.062013  4506 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.063140  4464 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:09.063618  4288 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:09.047631  4530 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:09.045782 32120 rpcz_store.cc:276] 0709 12:19:08.027261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.027483 (+   222us) service_pool.cc:224] Handling call
0709 12:19:08.027865 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.044727 (+1016862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.045269 (+   542us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003e53d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:09.045422 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337163136}
W20260709 12:19:09.066406 32084 connection.cc:256] RPC call timeout handler was delayed by -1.017278178999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:19:09.063633  4464 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:09.068322  4464 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:09.069604  4093 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:09.070621  4392 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.071413  4439 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.073019  4439 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:09.074815 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2109) took 1719 ms (client timeout 2085 ms). Trace:
W20260709 12:19:09.075246 31966 rpcz_store.cc:269] 0709 12:19:07.361080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.369507 (+  8427us) service_pool.cc:224] Handling call
0709 12:19:09.074608 (+1705101us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b5400079ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:09.074720 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663225088}
I20260709 12:19:09.077394  4415 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:09.077891 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:09.078339  4415 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:09.078995  4538 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.079552  4538 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.078699  4415 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:09.082170  4415 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.070948 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:09.067456 32047 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:09.085515 32046 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:09.085984 32047 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:09.086369 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:09.086534  4548 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.087106  4548 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.087786  4486 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:09.088424 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:09.088627  4093 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:09.089334  4093 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:09.089818  4246 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:09.091224  4383 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:09.091028  4093 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.093456  4354 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:09.093873  4354 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.094228  4354 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:09.091069  4246 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.094903  4246 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.095265  4246 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.095666  4399 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:09.096078  4399 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:09.088398  4486 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:09.096920  4399 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.097481  4486 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.098857  4460 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:09.099339  4460 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:09.100281  4460 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.101578  4460 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.103214 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.103334 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:09.104280 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.105410  4510 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:09.098213  4399 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:09.112995 32041 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:09.115731  4421 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:09.115729 32041 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:09.120481  4525 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.121167  4525 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.123924 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.032459  4466 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.067456  4493 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.127293 32118 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:09.127558  4515 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:09.129084  4493 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:09.129424 32013 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:09.129746 32013 leader_election.cc:403] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:09.131616  4479 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.132344 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.133611 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:09.134197 32655 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.137429 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 834) took 1137 ms. Trace:
I20260709 12:19:09.139201  4548 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.141685  4536 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.146651 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.147239 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:09.147882 32117 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.137838 32660 rpcz_store.cc:276] 0709 12:19:07.999702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.999884 (+   182us) service_pool.cc:224] Handling call
0709 12:19:08.014862 (+ 14978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.541475 (+526613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.137124 (+595649us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400403b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:09.137368 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178513664}
I20260709 12:19:08.987166 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:09.145078  3991 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.154397 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:09.143610 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54241.3us
I20260709 12:19:09.176599 32655 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:09.178449 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:09.178906 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 312228us
I20260709 12:19:09.160497 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:09.182590 32654 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.184885 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:09.191455 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:09.192327 32658 raft_consensus.cc:2410] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:09.087368 32047 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:09.193142 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:09.193773 32046 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:09.194834 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:09.208302 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:09.209709 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:09.214818 32656 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:09.228068 32084 leader_election.cc:400] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16.
I20260709 12:19:09.228703 32084 leader_election.cc:403] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:09.215137 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:09.219252 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:09.219941 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.220780 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:09.221972 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.217428 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.213799 32043 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:19:09.238904 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59266.4us
W20260709 12:19:09.240314 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.244017 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.245113 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:09.209463  4437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.000s	sys 0.015s
I20260709 12:19:09.193390  4445 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:09.190601  4400 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:08.975562  4499 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:09.194461  4466 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:09.256497 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.260717 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:09.267972 32118 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:09.038080  4441 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.212710  4375 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:09.232637  4415 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:09.207573  4515 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:09.230759  4545 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.271361  4513 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:09.242525  4533 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:09.296682 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:09.254410  4437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.000s	sys 0.016s
I20260709 12:19:09.207173  4405 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.084s
W20260709 12:19:09.299309 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47730.8us
I20260709 12:19:09.293802 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:09.230520  4549 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:19:09.232448  4421 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:19:09.154703  4536 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.310395 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:09.312047 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:09.314203 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:09.290017  4466 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.316463 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:09.318630 31970 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:09.351296 32117 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
W20260709 12:19:09.319069 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 127352us
I20260709 12:19:09.215209  4327 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:19:09.292500  4493 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16.
I20260709 12:19:09.323099 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:09.330996 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:09.295150  4400 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:09.344597 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.460264 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 137238us
I20260709 12:19:09.299196  4532 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:09.341092 32119 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:09.449579 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 96375.7us
I20260709 12:19:09.449175 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:09.328421  4405 consensus_queue.cc:1243] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:19:09.452596 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:09.453876 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.330523  4375 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:09.498428  4565 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:09.317754  4545 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.307848  4515 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.341414  4464 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:09.510248 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.353215  4466 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.828881  4093 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.353454  4201 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.144s
I20260709 12:19:09.461860  4514 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:09.487478  4499 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:09.822350 32654 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.630s	user 0.019s	sys 0.050s
I20260709 12:19:09.826593 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:09.898952 32654 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:09.824532 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:09.769016  4508 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.747s	user 0.015s	sys 0.005s
I20260709 12:19:09.829388  4375 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.455674  4525 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.458251  4497 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:09.818492 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 357570us
W20260709 12:19:09.844061  4516 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.902s	user 0.015s	sys 0.004s
W20260709 12:19:09.826599  4471 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.058s	user 0.019s	sys 0.000s
I20260709 12:19:09.821918  4288 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.872908 32040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.026s	sys 0.026s
W20260709 12:19:09.842968  4543 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.548s	user 0.000s	sys 0.009s
I20260709 12:19:09.895799 32657 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:09.840806  4437 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:09.827355  4436 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:19:09.826195  4549 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:19:09.861191  4093 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:09.928944 32040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.026s	sys 0.026s
I20260709 12:19:09.907704  4375 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.904971  4508 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13.
I20260709 12:19:09.908593  4506 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:09.908113  4421 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:09.923938  4329 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:09.978291  4566 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:19:09.979852  4354 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.885s	user 0.015s	sys 0.002s
W20260709 12:19:09.973616 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.009s	sys 0.009s
W20260709 12:19:10.007835 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.009s	sys 0.010s
W20260709 12:19:10.008272 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 687641us
I20260709 12:19:09.929828  4497 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.971565  4507 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:10.000746  4546 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.142s	user 0.001s	sys 0.001s
I20260709 12:19:09.980785  4093 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.950846  4288 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.037266 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.952080  4471 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.026615  4354 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.923163  4532 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:09.951761  4516 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.031289  4329 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:10.030095  4506 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.007575  4536 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:09.977319  4437 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.060734  4246 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.965s	user 0.010s	sys 0.016s
I20260709 12:19:10.070145  4515 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:10.071234  4494 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.015s	user 0.011s	sys 0.006s
W20260709 12:19:10.071369  4460 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.969s	user 0.006s	sys 0.005s
I20260709 12:19:10.000007  4514 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 12:19:10.060827  4562 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.398s	user 0.006s	sys 0.004s
W20260709 12:19:10.076084  4560 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:19:10.078956 31967 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:10.074594 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 621847us
I20260709 12:19:10.081590  4354 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.027482  4421 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.035692  4507 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.082661  4558 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 12:19:10.083254  4553 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:19:10.087975  4479 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.956s	user 0.007s	sys 0.002s
I20260709 12:19:10.149256  4479 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
W20260709 12:19:10.088027  4559 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 12:19:10.150501  4561 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 12:19:10.151059 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 571) took 1095 ms. Trace:
W20260709 12:19:10.089082  4567 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:19:10.151634  3991 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.996s	user 0.004s	sys 0.005s
I20260709 12:19:10.091214 32044 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:10.151511 32038 rpcz_store.cc:276] 0709 12:19:09.055336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.119184 (+ 63848us) service_pool.cc:224] Handling call
0709 12:19:09.119768 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.120189 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.120229 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.120771 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.255590 (+134819us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:09.255628 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.255693 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.150821 (+895128us) raft_consensus.cc:1685] finished
0709 12:19:10.150844 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.150994 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1312,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":134395,"threads_started":1,"wal-append.queue_time_us":734290}
I20260709 12:19:10.152707 32045 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:10.039580  4497 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:09.966753  4400 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.062876  4441 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:10.160307 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.095103  4329 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.096134  4516 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.096493  4464 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.098682 31972 raft_consensus.cc:2368] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13.
I20260709 12:19:10.100358  4471 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:10.100987  4574 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:19:10.101508  4564 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.102571 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2149) took 1162 ms. Trace:
W20260709 12:19:10.106842 31968 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.000s	sys 0.011s
I20260709 12:19:10.109232 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:10.109925  4399 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.002s	user 0.010s	sys 0.001s
I20260709 12:19:10.114142  4246 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:10.088459  4439 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.015s	user 0.013s	sys 0.006s
I20260709 12:19:10.155221 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.175292 32046 rpcz_store.cc:276] 0709 12:19:08.940653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.114012 (+173359us) service_pool.cc:224] Handling call
0709 12:19:09.193612 (+ 79600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.194211 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.102361 (+908150us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b540003a050. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.102510 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182188800}
I20260709 12:19:10.138893  4288 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.162578 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.107815  4506 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.141718  4549 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:10.152814  3990 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.115482  4466 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.168757 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 234511us
W20260709 12:19:10.176728 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 139812us
W20260709 12:19:10.177901 31968 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.000s	sys 0.013s
I20260709 12:19:10.190914 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:10.168026  4351 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.004s	sys 0.008s
I20260709 12:19:10.191017 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:10.175853  4571 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.166100  4494 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.194736 32118 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:10.194875 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.198487 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.197589 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:10.199450 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.202690 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:10.207069 31971 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:10.207480 32116 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:10.214905 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:10.187789  4464 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:10.218253 32038 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:10.215757 32039 pending_rounds.cc:85] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 11
I20260709 12:19:10.228850 32039 pending_rounds.cc:107] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:19:10.229213 32039 raft_consensus.cc:2889] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:10.226609 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:10.196218  4351 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.008s	sys 0.008s
I20260709 12:19:10.220439 32041 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:19:10.231483 32046 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:10.235544 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.194561  3990 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.241110 32042 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:10.217273  4573 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:10.181224  4564 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:10.255579 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58097.1us
I20260709 12:19:10.239255  4464 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:10.268110 32038 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:10.262593 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.270756 32041 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:10.228407  4494 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.207069  4549 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.217666  4571 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.256403  4550 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.274247 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:10.276921 32043 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:10.283330 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80551.5us
I20260709 12:19:10.261777  4405 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:10.268218  4288 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:10.272516  4513 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.955s	user 0.006s	sys 0.005s
I20260709 12:19:10.298527  4600 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.307608 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:10.309671  4594 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:10.310420 32010 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.536s (SENT)
I20260709 12:19:10.311385 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:10.286583  4499 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.287287  4437 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.306052  3991 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.313524  4581 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:19:10.314025  3991 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.314572  4579 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.315169  4579 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.316246  3991 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.307735  4405 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.322474  4494 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.323242  4466 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.324946  4564 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.325425  4533 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:10.261325  4590 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:19:10.312963 32010 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.548s (SENT)
I20260709 12:19:10.305156  4513 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.322888  4405 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.258508  4573 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.327667 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.329655 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:10.309052  4415 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.002s	user 0.007s	sys 0.004s
I20260709 12:19:10.336387 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.331710  4246 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.337284 32040 raft_consensus.cc:2410] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:19:10.337568  4580 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.338160  4580 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.307919  4351 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:10.340437  4580 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.340312  4351 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.337419  4589 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:19:10.312347  4479 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.313652  4497 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.312583  4421 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.333793 32659 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:10.303508  4288 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.318228  4579 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.328981  4533 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.327898  4437 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:10.332942  4532 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:10.291086  4550 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.311549  4093 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.336303  3991 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:10.332091  4375 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.349808  4591 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.352610  4518 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:10.330927  4499 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.366101  4499 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.352375  4533 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.359253  4568 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:19:10.369228  4588 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.118s	user 0.001s	sys 0.001s
I20260709 12:19:10.369925  4533 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.371090 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.360471  4093 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.329847  4464 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.362542  4375 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.363514  4571 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.345934  4479 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.371455  4573 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:10.381660  4592 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:19:10.370069  4549 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.371515  4518 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.383606  4479 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.356956  3991 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.384701 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 549) took 1074 ms. Trace:
I20260709 12:19:10.351366  4506 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:10.356643  4557 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.167s	user 0.001s	sys 0.001s
I20260709 12:19:10.360033  4288 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.385574 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.385138 31971 rpcz_store.cc:276] 0709 12:19:09.309880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.310043 (+   163us) service_pool.cc:224] Handling call
0709 12:19:10.384503 (+1074460us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540034d854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.384641 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965779328}
I20260709 12:19:10.384528  4518 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.391527 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.392138  4518 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:10.371441  4538 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.395144  4550 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.398813 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:10.402176 31972 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2120) took 1398 ms (client timeout 1527 ms). Trace:
W20260709 12:19:10.402765 31972 rpcz_store.cc:269] 0709 12:19:09.003738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.003894 (+   156us) service_pool.cc:224] Handling call
0709 12:19:10.401921 (+1398027us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400402ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.402101 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2500267904}
I20260709 12:19:10.420928 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.386078  4591 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.406792  4436 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:10.434619  3991 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.458920 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61673.9us
I20260709 12:19:10.462747 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.463128 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.463555 32046 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:10.463862 32043 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:10.452483  4436 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:10.469177 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 545) took 2127 ms (client timeout 1668 ms). Trace:
I20260709 12:19:10.469336  4392 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:10.469643 31970 rpcz_store.cc:269] 0709 12:19:08.341870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.350846 (+  8976us) service_pool.cc:224] Handling call
0709 12:19:10.468971 (+2118125us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54000b5754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.469084 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891824384}
I20260709 12:19:10.470783  4436 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.458379  3991 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.476485  4240 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:10.475600  4345 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.498966 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.500102 32660 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 10.
I20260709 12:19:10.502457  4591 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.504375 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.506325  4400 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:10.506811  4400 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.507225  4400 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:10.522125  4611 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:19:10.530481  4563 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.522s	user 0.002s	sys 0.000s
I20260709 12:19:10.546535 32117 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:19:10.548185 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.550447 32118 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:10.553568 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.572764  4576 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.580089  4576 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.586649 32045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2155) took 1621 ms (client timeout 1674 ms). Trace:
W20260709 12:19:10.587103 32045 rpcz_store.cc:269] 0709 12:19:08.964723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.986834 (+ 22111us) service_pool.cc:224] Handling call
0709 12:19:10.586432 (+1599598us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b5400217d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.586581 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299798784}
I20260709 12:19:10.590467 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 837) took 1431 ms. Trace:
I20260709 12:19:10.590898 32657 rpcz_store.cc:276] 0709 12:19:09.159196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.159323 (+   127us) service_pool.cc:224] Handling call
0709 12:19:09.159639 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.945313 (+785674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.590197 (+644884us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:10.590406 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679882112}
W20260709 12:19:10.593436 32044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 567) took 1674 ms (client timeout 1745 ms). Trace:
W20260709 12:19:10.594020 32044 rpcz_store.cc:269] 0709 12:19:08.919532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.939449 (+ 19917us) service_pool.cc:224] Handling call
0709 12:19:10.593224 (+1653775us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002cdb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.593360 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2507003136}
I20260709 12:19:10.594497  4518 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:10.583808  4400 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.604094  4518 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:10.604982  4518 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.605710  4400 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.605995  4518 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.607388  4518 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:10.578610  4351 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.602864  4473 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:10.602198  3990 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.607676 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:10.599617  4507 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:10.610170 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.619953 32657 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.620509  4473 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.621441  4473 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.621990 32114 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:10.627422 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 319882us
I20260709 12:19:10.628366 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:10.629204 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.631706 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.633426 32657 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:10.633656 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.633961 32044 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 13. Ops: [11.6-11.6]
I20260709 12:19:10.634851 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.635042 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.636147 32040 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:19:10.637028 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 191738us
I20260709 12:19:10.637065 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.634377 32118 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:10.637979 32116 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:10.637869 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:10.638568 32038 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:10.638656 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:10.639212 32117 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.639945 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:10.640416 32044 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.643775 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.641311 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:10.644403 32041 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:10.645741 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.646219 32045 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:10.646922 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.647500 32115 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:10.632903  3990 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:10.644187  4400 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.649139  4400 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.630393  4473 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:10.650641  4556 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.574s	user 0.003s	sys 0.000s
I20260709 12:19:10.637029  4507 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.652935  4507 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.654955  4507 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.641824  4580 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.645012 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.655709  4580 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.656066  4608 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:10.656528  4608 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.656900  4608 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.657784  4580 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.644073  4497 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:10.658885 32047 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:10.659226 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.659821 32116 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:10.660784  4576 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.661510  4464 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:10.637157  4351 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:19:10.666734 32117 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:19:10.669205 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.663707 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:10.671200  4617 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:10.673125 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.679175 32044 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:10.680527 32038 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:10.643949  4534 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.684214 31963 pending_rounds.cc:85] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 4
I20260709 12:19:10.685222 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.685989 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2410) took 1379 ms. Trace:
W20260709 12:19:10.687057 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:10.686511 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 1.31986e+06us
I20260709 12:19:10.687752 31963 pending_rounds.cc:107] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:19:10.688069 31963 raft_consensus.cc:2889] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:10.688869 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.690598 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.691267 31972 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:10.687372 32112 rpcz_store.cc:276] 0709 12:19:09.306638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.306764 (+   126us) service_pool.cc:224] Handling call
0709 12:19:09.308492 (+  1728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.452128 (+143636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.452161 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.544567 (+ 92406us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.230293 (+685726us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:10.231531 (+  1238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.671758 (+440227us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b540010efd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:10.671798 (+    40us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.671841 (+    43us) raft_consensus.cc:1685] finished
0709 12:19:10.671853 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.685916 (+ 14063us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95670,"raft.queue_time_us":99196,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":50564,"spinlock_wait_cycles":107475328,"thread_start_us":353704,"threads_started":1,"wal-append.queue_time_us":486181}
I20260709 12:19:10.693884 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.695400 31970 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:10.703840 31966 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:10.703892 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:10.674772  4575 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:10.707536 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:10.707502 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:10.708468 31966 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:10.694685 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:10.709187  4405 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:10.710103 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:10.709282  4288 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:10.710740  4288 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.710880  4506 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:10.711154  4288 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:10.711385  4506 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.711809  4506 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.711963 32084 leader_election.cc:400] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13.
I20260709 12:19:10.712291 32084 leader_election.cc:403] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:10.712982 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.714032 32112 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:10.715845 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.716470 32038 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:10.686472  4507 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:19:10.716863  4532 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.717198  4405 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.717691  4405 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.721057 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:10.721603 32657 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:19:10.680316  4497 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:10.680634  4617 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.693131  4586 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:10.723106 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:10.689007  4155 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.690253  4336 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.725926 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:10.726341 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:10.727361 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.689639  3991 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:10.741087  3991 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.674369  4579 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:19:10.735474  4460 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:10.735750  4527 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:10.729868  4550 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.726794  4581 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:10.737136 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2121) took 1690 ms. Trace:
I20260709 12:19:10.737452  4400 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.737848 31972 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:10.737161 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.780118 32120 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:10.725090 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.692152  4351 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:10.745157 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.780594 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:10.780622 31966 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 8.
I20260709 12:19:10.756731 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.722759  4533 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:10.744764  4392 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.741545  4608 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.755084  4460 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:10.744791  4549 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:10.758994  4550 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.760254 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2122) took 1713 ms. Trace:
W20260709 12:19:10.760928 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.759045 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:10.763156  4421 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.768231 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.813690 32116 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:10.775666  4545 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.781766 32115 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:10.785060 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:10.696779  4580 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:19:10.781015 31965 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:10.732255  4617 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.691475  4534 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.728773  4405 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.799139 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.822984 32112 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.800596 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.722561  3990 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:19:10.801376 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:10.825790 31972 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:10.806692 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2403) took 1533 ms. Trace:
I20260709 12:19:10.825258 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.808588 31964 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:10.722708  4576 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:19:10.811575 31970 raft_consensus.cc:2376] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13. Re-sending same reply.
I20260709 12:19:10.815997 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:10.807272  4549 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.820662 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.789641  4396 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:10.741762  4603 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.799496 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.748185  4336 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.747782  4155 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.837543 32116 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:10.777663 31968 rpcz_store.cc:276] 0709 12:19:09.046654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.065814 (+ 19160us) service_pool.cc:224] Handling call
0709 12:19:09.066138 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.066513 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.066544 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.067009 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.303406 (+236397us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:10.190360 (+886954us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.650067 (+459707us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:10.736775 (+ 86708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.736845 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.736880 (+    35us) raft_consensus.cc:1685] finished
0709 12:19:10.736967 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.737081 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":546153,"prepare.queue_time_us":183836,"prepare.run_cpu_time_us":8767,"prepare.run_wall_time_us":331744,"raft.queue_time_us":548498,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":178,"spinlock_wait_cycles":6286080,"thread_start_us":1093362,"threads_started":2,"wal-append.queue_time_us":1009038}
I20260709 12:19:10.827826 32038 raft_consensus.cc:2376] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:19:10.810262 31969 rpcz_store.cc:276] 0709 12:19:09.047051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.050206 (+  3155us) service_pool.cc:224] Handling call
0709 12:19:09.050536 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.050746 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:09.233050 (+182304us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.662636 (+1429586us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b1800462460. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:19:10.671267 (+  8631us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.759849 (+ 88582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.759917 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:10.759973 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.760044 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.760191 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173621,"prepare.queue_time_us":298563,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":1302241,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":8884,"spinlock_wait_cycles":174218112,"thread_start_us":398694,"threads_started":2,"wal-append.queue_time_us":639247}
I20260709 12:19:10.802830  4351 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.826246 32114 rpcz_store.cc:276] 0709 12:19:09.255981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.256144 (+   163us) service_pool.cc:224] Handling call
0709 12:19:10.788818 (+1532674us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400138154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:10.788934 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3062184832}
I20260709 12:19:10.754703  4586 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.794718  4527 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.816870  4486 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:10.850297 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:10.817906  4439 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.817798  4400 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.834632  4549 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:10.840734 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.841166 31969 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:19:10.828756 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:10.806983  4392 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.820261  4421 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.810721  4436 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.849668  4469 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.832176  4201 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.835801  4396 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.849784 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.834146 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:10.854998 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.872802  4531 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:10.856577 31968 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:10.834892  4533 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:10.861157 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:10.823442  4497 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.864351 31969 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:10.821377  3991 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.836223  3990 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:10.851204  4527 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.850761  4486 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:10.851432  4586 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.880482 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.873579  4155 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.888006 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:10.863097  4472 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.894982 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:10.896329  4351 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:10.905702 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:10.907985 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 219842us
I20260709 12:19:10.908711 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:10.874195  4396 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.887226  3990 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:10.915143  3990 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.916373  3990 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.874187  4469 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.919055  4486 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.874727  4405 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:10.863281  4439 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.846868  4460 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.948856 32659 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:10.902483  4531 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.888578  4533 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:10.865020  4400 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:10.931516 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.950847 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:10.955487 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46857us
I20260709 12:19:10.934835 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:10.878404  4201 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.880017  4581 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:19:10.923303 32112 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:10.960556 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:10.874257  4436 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:10.983122  3991 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.886684  4573 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:19:10.912412  4608 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:10.928489  4493 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.987866 31965 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:10.927006  4510 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:10.949625  4469 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:10.852795  4603 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:10.990137  4622 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:19:10.997335 31972 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:19:10.997431 32657 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:10.954983  4591 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:10.912323  4576 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 12:19:11.003712 31967 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2119) took 2011 ms (client timeout 1694 ms). Trace:
I20260709 12:19:10.926529  4472 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:10.959396  4533 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.993629  4581 consensus_queue.cc:1243] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:10.998291  4345 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:11.012734 31967 rpcz_store.cc:269] 0709 12:19:08.992817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.993443 (+   626us) service_pool.cc:224] Handling call
0709 12:19:11.003485 (+2010042us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000f7654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:11.003644 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482191104}
I20260709 12:19:10.989019  4421 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.011694 31971 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:11.021035 32658 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:11.023162 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.044612 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.045362 31966 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
W20260709 12:19:11.055254 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.004300  4531 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:19:11.002780  4493 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:11.062700  4623 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 12:19:11.071698 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.043105  4550 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.058974 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:11.043241  4620 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:11.076934 31966 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:11.060601  4518 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.022634  4591 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.064010  4336 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:11.076547  4554 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.998s	user 0.001s	sys 0.004s
I20260709 12:19:11.059564  4345 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.067994  4392 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.081208  4534 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:11.114269 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 219259us
I20260709 12:19:11.115746 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:11.085734  4288 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.085275  4518 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.120199 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:11.130416 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.130803  4247 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:11.131381  4247 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:11.131835  4247 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.084446 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.135273 31966 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:11.136412 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.132073  4345 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.140863 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.117295  4540 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:11.120396  4439 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.122433  4550 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:11.151621  4247 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.146808 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:11.154736  4472 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.175918 32046 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.160267  4532 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13.
I20260709 12:19:11.160907  4436 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.163141  4400 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.178139 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.204644 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.205474 32045 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:11.145884  4573 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.146904  4201 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:11.170403  4550 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:11.198457  4581 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:11.219947 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.215081  4625 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:19:11.199620  4540 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:11.211386  4439 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.214519  4603 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.211086  4405 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.220929  4514 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.248428 32046 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:11.218437  4550 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:11.246369  4581 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:11.272607  4603 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.275560  4493 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.267524  4527 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.261509  4439 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.267823  4549 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.266870  4405 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.341046 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.323398  4577 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.319650  4351 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.344538  4288 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:11.346177  4632 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:19:11.352717  4473 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.003s
W20260709 12:19:11.354871  4631 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:19:11.359548 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.380270 32043 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:11.362390 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:11.364923 31964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2112) took 2454 ms (client timeout 1692 ms). Trace:
I20260709 12:19:11.381433 32039 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:19:11.277702  4608 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 12:19:11.381335 31964 rpcz_store.cc:269] 0709 12:19:08.910798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.910998 (+   200us) service_pool.cc:224] Handling call
0709 12:19:11.364763 (+2453765us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540042fed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:11.364866 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3967658368}
I20260709 12:19:11.367108 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.272130  4514 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.371006  3991 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:11.377988 32115 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:11.382587 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.385249 32113 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:11.370494  4577 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:11.408871  4635 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:11.410264  4577 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.407631 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.392002  3991 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:11.416177  3991 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.416452 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:11.417088 32043 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.417366  4493 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:11.417939  4493 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:11.391450 32039 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:11.418885  4493 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.333784  4603 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.419731 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.420660 32039 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:11.421712  4603 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:11.422032 32039 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:11.414567 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.373303  4518 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:11.390391  4506 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.002s
I20260709 12:19:11.373595  4591 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:11.430275 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.435379  4439 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:11.438042 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:11.439195 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.439141 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.374418  4351 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.444099 31963 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.338256  4540 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.451032 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.452651 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:11.424408  4642 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:11.455281 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.459578 32039 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:19:11.441839  4514 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:11.460846 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.463539 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.466913 31964 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:11.449370  4247 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:11.450474  4576 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:11.473888 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.441885  4506 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.478842  4576 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:11.459167  4642 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.465278  4518 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:11.484843 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.486019 31964 raft_consensus.cc:2376] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13. Re-sending same reply.
I20260709 12:19:11.487165  3990 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.487831  3990 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.491025  4240 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:11.491549  4240 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:11.491140  4576 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.492429  4240 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.430375  4581 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.503888 31963 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:11.479677  4514 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:11.512986 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.516729 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:11.495414  4518 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:11.502301  4638 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:19:11.524173 31963 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:11.496392  4642 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.500032  3990 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.479862  4473 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 12:19:11.514997  4636 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:19:11.535794 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.514483  4247 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.536648 31963 raft_consensus.cc:2368] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:11.507659  4514 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.524843  4633 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:11.542868 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.544935 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.526007  4518 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.560432 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.561290 31963 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.542457  4405 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.547317  4514 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.559616  4549 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:11.559156  4617 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.734s	user 0.005s	sys 0.007s
I20260709 12:19:11.555706  4633 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:11.596623 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.597707 31969 raft_consensus.cc:2368] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
W20260709 12:19:11.587116 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.595115 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.573513  4573 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.551628  4247 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.614327  4633 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.618494 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.573748  3991 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.602542  4617 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.640623 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.648801 31963 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:11.649674 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.653275 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.653658 32047 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:11.654863 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.655735 32042 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:11.661849 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.662485 31963 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:11.634897  4351 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.660907 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.664112 32047 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:11.665606 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.623086  4573 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.669241 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.670037 32039 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:11.677443 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.678301 31969 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:11.681634 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.688176 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.689145 32042 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:11.694327 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.661460  4473 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:11.663174  4481 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.691545 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.700620 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.700661 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.697690 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.703153 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.706271 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.701754 32039 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:11.672250  4549 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.716199 32626 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:11.680008  4506 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.717530 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:11.718729 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.720654 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.726687 31972 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:11.700241  4473 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.728446 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.729780 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.713182  4493 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:11.716547  4405 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:11.725576  4533 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.003s
I20260709 12:19:11.699623  4351 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:11.751333 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2446) took 1025 ms. Trace:
I20260709 12:19:11.759477 32112 rpcz_store.cc:276] 0709 12:19:10.725599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.726095 (+   496us) service_pool.cc:224] Handling call
0709 12:19:11.751184 (+1025089us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b54003e53d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:11.751279 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189540224}
I20260709 12:19:11.760012 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.752418 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:11.752157 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:11.760784 32117 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.760838 32119 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:11.765594 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.766206 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.766847 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:11.767678 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.727645  3990 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:11.769148 32113 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:11.757445 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:11.780774 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:11.771024 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:11.722508  4608 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:11.809854 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.728138  4637 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:11.742990  4493 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.718782  4549 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.733992  4473 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.782900 32658 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:11.813417 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:11.813438 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:11.822923 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.776997 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 707) took 1009 ms. Trace:
I20260709 12:19:11.824591 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:11.825173 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.826670 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.827286 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.710402  4481 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.774658  4137 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:19:11.778805  3990 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.826227  4637 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.876071  3990 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.876235  4637 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.879276  4573 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.880106  4573 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:11.880259 31964 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.880633  4573 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:11.881201  4627 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.837s	user 0.004s	sys 0.000s
I20260709 12:19:11.881250  4573 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.882696 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2178) took 1166 ms. Trace:
I20260709 12:19:11.774976  4497 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:11.883049 32044 rpcz_store.cc:276] 0709 12:19:10.716715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.716865 (+   150us) service_pool.cc:224] Handling call
0709 12:19:10.717184 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.717510 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.717546 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.718025 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.899918 (+181893us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:10.899959 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.900038 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.880656 (+980618us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:11.882408 (+  1752us) raft_consensus.cc:1685] finished
0709 12:19:11.882431 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.882639 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":474,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":6523648,"thread_start_us":181487,"threads_started":1,"wal-append.queue_time_us":280059}
I20260709 12:19:11.884454  4497 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.863327 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:11.885515 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.865381 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:11.874969 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:11.886811 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:11.862841  4396 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.917s	user 0.012s	sys 0.000s
I20260709 12:19:11.887362 31967 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
W20260709 12:19:11.887684  4586 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.967s	user 0.011s	sys 0.000s
I20260709 12:19:11.888734 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2140) took 1089 ms. Trace:
I20260709 12:19:11.889106 31968 rpcz_store.cc:276] 0709 12:19:10.799458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.840312 (+ 40854us) service_pool.cc:224] Handling call
0709 12:19:10.852995 (+ 12683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.874237 (+ 21242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.888536 (+1014299us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b54001ceb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:11.888683 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7532800}
I20260709 12:19:11.890328 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.890829  4617 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.891094  4403 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:11.746471  4405 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:11.895850  4405 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.731367  4550 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.899637 32085 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:11.901599  4617 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:11.887455  4396 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.903836  4396 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.727604  4421 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:19:11.906848  4137 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:11.907229  4137 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:11.907992  4137 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.908656  4460 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:11.909186  4137 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.910682 31963 raft_consensus.cc:2368] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:11.871235  4288 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.911755  4288 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.913925 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.914631 31972 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:11.868901  4642 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:11.918551  4247 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.919276 31970 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:11.920744 32118 rpcz_store.cc:276] 0709 12:19:10.768061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.768979 (+   918us) service_pool.cc:224] Handling call
0709 12:19:10.769324 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.810638 (+ 41314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.810668 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.811088 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 12:19:11.133761 (+322673us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:11.133820 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.747885 (+614065us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54002fa654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:11.747935 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.747983 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:11.747999 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.776911 (+ 28912us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6755,"raft.queue_time_us":4174,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6998,"spinlock_wait_cycles":87629056,"thread_start_us":322352,"threads_started":1,"wal-append.queue_time_us":392030}
I20260709 12:19:11.922698 31969 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:11.924562 32654 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:11.890838 31968 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:11.918545 31971 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:11.893591 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.931511 31967 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:19:11.888254  4586 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:11.935876 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.835191 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:11.948346 32656 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:11.948741 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.923648 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:11.939458 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.915621 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:11.950210 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.952648 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.833861 32113 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:11.944741 32115 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:11.950328 31972 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:11.955077 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:11.836105  4643 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 12:19:11.964545 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:11.951630 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 231083us
W20260709 12:19:11.865167  4644 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 12:19:11.907222  4641 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.448s	user 0.000s	sys 0.005s
I20260709 12:19:11.955706 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:11.953761 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 560) took 1032 ms. Trace:
I20260709 12:19:11.880131  4481 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:11.969815 31970 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:11.941082 31964 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:11.770390  4533 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:11.875077  3991 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:11.967633 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.968168 32652 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:11.981881 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:11.981979 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 10.
I20260709 12:19:11.982259 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.982658 32044 raft_consensus.cc:2368] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
I20260709 12:19:11.982883 31971 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:11.966739 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.983865 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:11.825402 32117 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:11.968879 31968 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:11.984867 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.972847 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:11.985723 32046 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.985913 31969 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:11.986678 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:11.987468 31965 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:19:11.984647 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:11.955231  4550 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:11.956094 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 230007us
I20260709 12:19:11.976321 31967 rpcz_store.cc:276] 0709 12:19:10.921519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.952876 (+1031357us) service_pool.cc:224] Handling call
0709 12:19:11.953170 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.953343 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:11.953367 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:11.953411 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:11.953446 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.953607 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.953710 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:11.978932 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.979671 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:11.995546 32038 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.954865  4575 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.980636 32039 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:12.000947 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:11.834725  4493 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.008327 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 587) took 1154 ms. Trace:
I20260709 12:19:12.008787 32040 rpcz_store.cc:276] 0709 12:19:10.854288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.958744 (+104456us) service_pool.cc:224] Handling call
0709 12:19:10.959138 (+   394us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:19:10.990889 (+ 31751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.990922 (+    33us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:19:10.997422 (+  6500us) log.cc:844] Serialized 104 byte log entry
0709 12:19:11.550020 (+552598us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b18004a6080. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:19:11.685427 (+135407us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:11.740217 (+ 54790us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.740587 (+   370us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.744609 (+  4022us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.744828 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.765675 (+ 20847us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.892977 (+127302us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b48000fe358. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:11.895524 (+  2547us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b18004a6080. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:11.915135 (+ 19611us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b18004a6080. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:12.008029 (+ 92894us) raft_consensus.cc:1685] finished
0709 12:19:12.008060 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.008258 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":282943,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":11622,"raft.queue_time_us":39907,"raft.run_cpu_time_us":2187,"raft.run_wall_time_us":205787,"spinlock_wait_cycles":141771776,"thread_start_us":82773,"threads_started":1,"wal-append.queue_time_us":819247}
I20260709 12:19:11.965546  4642 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:11.980271  4571 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.003976 32116 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:12.014986 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.016785  4571 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.017508  4571 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:12.026144 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 952) took 1092 ms. Trace:
W20260709 12:19:12.036345 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.026686 32652 rpcz_store.cc:276] 0709 12:19:10.933947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.934083 (+   136us) service_pool.cc:224] Handling call
0709 12:19:10.941229 (+  7146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.988566 (+1047337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.025918 (+ 37352us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54004e4b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:12.026087 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361284992}
I20260709 12:19:12.040454 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.041285 31969 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:12.041643 32038 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:12.041311 32046 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:12.045656 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:12.088899 32654 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:12.089293 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:12.024772 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:12.094909 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.096156 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71727.6us
I20260709 12:19:12.105792 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.108722 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:12.109638 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 389123us
I20260709 12:19:12.110685 31972 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:12.038702  3991 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.039407  4497 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:12.112790 31968 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:12.110390 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.011972  4575 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.038851  4550 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:12.115084 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.112596 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.111089  4642 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:12.148108 31972 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:19:12.113448  4536 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:19:12.119220 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.108610  4436 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:12.106899  4288 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:12.142761  4514 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.009s
I20260709 12:19:12.165098 31971 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:12.117578 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.111793  4421 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:19:12.168642 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 561) took 1220 ms. Trace:
I20260709 12:19:12.168771  4421 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:12.169066 31971 rpcz_store.cc:276] 0709 12:19:10.948463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.148538 (+1200075us) service_pool.cc:224] Handling call
0709 12:19:12.164199 (+ 15661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.168381 (+  4182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.168584 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:12.155014 32652 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:12.157492  4436 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.170259  4436 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:12.170665 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:12.171146  4247 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.171623  4247 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.173641  4247 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.167266 32115 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:12.174603  4405 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.175369 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:12.176605 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:12.176759  4405 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.177300 32115 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:12.178553  4586 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.157526  4642 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:12.149075  4497 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:12.179729  4497 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.180114  4497 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.180001  4642 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.182826  4436 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.183589 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:12.178068 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:12.184798  4436 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
W20260709 12:19:12.186868 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67741.1us
I20260709 12:19:12.187103  4436 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:12.187601 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.160914 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2348) took 1204 ms. Trace:
I20260709 12:19:12.169363  4421 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.187533  4436 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.142378  4654 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.167800  4549 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:12.190398 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72919.2us
I20260709 12:19:12.174101  4247 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:12.163547  4518 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.003s
I20260709 12:19:12.191655 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:12.195200 31968 rpcz_store.cc:276] 0709 12:19:10.956278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.148746 (+1192468us) service_pool.cc:224] Handling call
0709 12:19:12.160300 (+ 11554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.160513 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.160554 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:12.160607 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:12.160651 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.160722 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.160849 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:12.142470  3991 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.198329 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:12.191067 32113 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:12.145936  4550 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.188694  4651 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.208076 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:12.204741 32010 leader_election.cc:336] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.649s (SENT)
I20260709 12:19:12.219723 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.207644 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:12.194013  4617 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:12.218200 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:12.196935  4436 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.221290 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.215935 32085 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:12.199481  4654 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.224793  4579 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.224864  4549 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:12.206876  4608 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.218128  4652 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:12.223312  4651 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.226279  4656 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:12.229490 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.189524  4514 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.236093 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2359) took 1090 ms. Trace:
I20260709 12:19:12.238925 32118 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:12.239691 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:12.264274 32010 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.683s (SENT)
W20260709 12:19:12.332195 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67968.9us
W20260709 12:19:12.333146 32010 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.721s (SENT)
I20260709 12:19:12.333678 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:12.328146 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 98728.3us
I20260709 12:19:12.222895  4586 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.328408 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:12.330991 32114 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.330809 31963 rpcz_store.cc:276] 0709 12:19:11.170405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.701693 (+531288us) service_pool.cc:224] Handling call
0709 12:19:12.235883 (+534190us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000b6654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:12.236023 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474947968}
I20260709 12:19:12.335785 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:12.343468 32084 leader_election.cc:400] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
I20260709 12:19:12.343873 32084 leader_election.cc:403] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:12.325408  4530 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:12.351305 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.840s (SENT) [suppressed 29 similar messages]
I20260709 12:19:12.210654  4518 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.234653  4654 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.206705  4247 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.241055  4549 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.241240  4576 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:12.361791 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.393852 31963 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:12.362026 32115 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.241637  4608 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.376544 32039 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:12.388329 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:12.334172  4351 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:12.327677  4533 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.391476 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:12.222339  4655 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.139s
I20260709 12:19:12.399821 32046 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:12.352367  4532 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14.
I20260709 12:19:12.340695  4586 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.407951 32038 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.409719 31966 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:12.361523  4375 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.406638  4608 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:12.415673  4637 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.539s	user 0.005s	sys 0.005s
I20260709 12:19:12.347935  4591 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.325924  4579 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.337600  4652 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.417593 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:12.228755  4531 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.128s
W20260709 12:19:12.426689 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 442057us
W20260709 12:19:12.458433 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.462301 32038 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:12.352690  4649 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:12.392305  4663 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.361137  4405 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 12:19:12.480255 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:12.398276  4549 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.407473  4351 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:12.487959 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 271160us
I20260709 12:19:12.440399  4460 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.471485  4533 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:12.490839 32085 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.450948  4586 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.362821  4530 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.475780  4649 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.426726 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.502148 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.492962 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:12.489583 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.492609  4351 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.472711  4375 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.491909  4591 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:12.570498 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 80987us
I20260709 12:19:12.543838 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:12.571357 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:12.509207  4573 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.627s	user 0.005s	sys 0.009s
W20260709 12:19:12.555169 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.533619 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2355) took 1575 ms. Trace:
W20260709 12:19:12.542395 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.574875 31964 rpcz_store.cc:276] 0709 12:19:10.958092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.702207 (+744115us) service_pool.cc:224] Handling call
0709 12:19:12.533412 (+831205us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540034e254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:12.533546 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341981056}
W20260709 12:19:12.541213  4473 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.001s
I20260709 12:19:12.576043 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.576714 31964 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:12.576114  4473 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.577760 32121 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.577675 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.578964 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.604422 32119 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:12.605430 31938 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.013s	sys 0.043s
I20260709 12:19:12.480080  4637 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.504123  4575 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.604296 32040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:36592
W20260709 12:19:12.605839 31938 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.013s	sys 0.043s
W20260709 12:19:12.673666 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 720426us
I20260709 12:19:12.674170 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:12.675138 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:12.611828 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:12.612192 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:12.545351  4571 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.527s	user 0.002s	sys 0.008s
I20260709 12:19:12.534560  4663 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:12.671026 31964 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:12.541965  4655 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:12.509434  4649 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.544835  3991 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.498175  4460 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.694258 32658 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.509807  4533 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.544363  4421 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.545957  4530 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.679158 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:12.509660  4576 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.565068  4506 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.706095  4506 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.708446  4506 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:12.680421 32010 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.819s (SENT)
I20260709 12:19:12.495146  4662 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 12:19:12.605801  4656 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:12.712212  4656 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:12.712564 32038 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:12.720388  4579 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.608688  4573 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:12.723040 32113 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:12.726279  4472 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:12.726656  4472 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:12.726802  4473 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.683143  4594 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:12.728140 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2350) took 1771 ms (client timeout 1683 ms). Trace:
W20260709 12:19:12.728518 31966 rpcz_store.cc:269] 0709 12:19:10.956937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.136077 (+179140us) service_pool.cc:224] Handling call
0709 12:19:12.727904 (+1591827us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001e9bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:12.728004 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2472368384}
I20260709 12:19:12.729259  4460 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.729534 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.729947  4460 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.680469  4617 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.712653  4656 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.692844 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.616745  4650 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.738474 32038 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:12.739424 32040 raft_consensus.cc:606] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:12.739775 32040 raft_consensus.cc:614] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.730104 31966 raft_consensus.cc:2376] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16. Re-sending same reply.
I20260709 12:19:12.740970 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.741966 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.742614 31966 raft_consensus.cc:2368] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:12.743274 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:12.734232  4493 consensus_meta.cc:287] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.001s
I20260709 12:19:12.749727 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.736088 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.697908  4550 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:12.691184  4571 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.751467 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.792927 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:13.188870 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 715) took 1021 ms. Trace:
I20260709 12:19:13.192823 32119 rpcz_store.cc:276] 0709 12:19:12.167483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.167617 (+   134us) service_pool.cc:224] Handling call
0709 12:19:12.167906 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.605262 (+437356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.188610 (+583348us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400147dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:13.188807 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003357952}
I20260709 12:19:12.989251 32653 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:13.195214 31964 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
W20260709 12:19:13.197263 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.198468 31968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.008s	sys 0.008s
W20260709 12:19:13.198792 31968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.008s	sys 0.008s
W20260709 12:19:13.139087 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 508004us
I20260709 12:19:12.814194 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2148) took 1631 ms. Trace:
I20260709 12:19:13.037905 32040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:36592
I20260709 12:19:13.202270 32040 raft_consensus.cc:606] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:13.203132 32040 raft_consensus.cc:614] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.863826 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:12.822516  4659 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:13.188982 31966 raft_consensus.cc:2368] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
W20260709 12:19:13.196337 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.002389 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:12.813256 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 339252us
I20260709 12:19:13.205753 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:13.206882 32047 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:13.214079 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:12.737852  4530 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.194796 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:13.216423 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2360) took 2021 ms. Trace:
I20260709 12:19:12.731869  4240 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.142544  4405 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.225661  4678 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:12.734587  4594 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:13.200423 31967 rpcz_store.cc:276] 0709 12:19:11.182300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.148691 (+966391us) service_pool.cc:224] Handling call
0709 12:19:12.149006 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.149422 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.149460 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.154051 (+  4591us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.484589 (+330538us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:12.484656 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.484766 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.813963 (+329197us) raft_consensus.cc:1685] finished
0709 12:19:12.813986 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.814133 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4166,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4466,"thread_start_us":299214,"threads_started":1,"wal-append.queue_time_us":571332}
I20260709 12:19:13.240039 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:13.241340 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:13.242352 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:13.248013 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53340.6us
W20260709 12:19:12.822286  4667 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:12.745754  4617 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:12.758127  4493 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.340526 32120 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:13.422780 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.234563 31971 rpcz_store.cc:276] 0709 12:19:11.209872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.178633 (+968761us) service_pool.cc:224] Handling call
0709 12:19:12.178926 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.191772 (+ 12846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.191819 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.192534 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.494060 (+301526us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:13.189438 (+695378us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.189782 (+   344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.189876 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.189927 (+    51us) raft_consensus.cc:1685] finished
0709 12:19:13.189957 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.216369 (+ 26412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12357,"prepare.queue_time_us":327891,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":39163,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5435,"thread_start_us":677652,"threads_started":2,"wal-append.queue_time_us":484857}
I20260709 12:19:13.421972 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:13.318686 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.309620 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.863832  4642 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.891538  3991 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:13.016685  4637 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:13.021566  4137 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.112s	user 0.009s	sys 0.004s
I20260709 12:19:12.747501  4514 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.875080  4473 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:13.431744 31967 raft_consensus.cc:2376] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12. Re-sending same reply.
I20260709 12:19:13.460234 31964 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:13.335645  4497 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.155s	user 0.012s	sys 0.000s
I20260709 12:19:12.800439  4518 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:12.816991  4586 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.460971 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:13.449615 32044 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.018s	sys 0.014s
W20260709 12:19:13.327731  4649 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.001s
I20260709 12:19:13.464501  4691 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:13.464850 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2415) took 1296 ms. Trace:
I20260709 12:19:13.465166 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:12.791245  4421 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.431202  4679 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.446620 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:12.932343  4652 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.820101  4527 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:12.875777  4506 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:13.472591 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.014s	sys 0.025s
W20260709 12:19:13.579977 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.016s	sys 0.028s
W20260709 12:19:13.580741 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 905579us
I20260709 12:19:13.317354  4571 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:13.277808  4655 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.485822  4240 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.484828  3991 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.524870 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.535451 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.593971 31963 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:13.594233 31964 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:13.315186  4594 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.465407  4642 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:13.456782  4591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.015s	sys 0.019s
W20260709 12:19:13.540095 32044 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.020s	sys 0.014s
W20260709 12:19:13.485953 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.544512  4682 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:13.460791 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.612874 31970 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:13.542290 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.216391  4532 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:13.463871  4493 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.628783 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.630290 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.631035 31963 raft_consensus.cc:2376] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:19:13.631615 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2401) took 1081 ms. Trace:
I20260709 12:19:13.632001 31963 rpcz_store.cc:276] 0709 12:19:12.550207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.594763 (+1044556us) service_pool.cc:224] Handling call
0709 12:19:13.631550 (+ 36787us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:13.401072  4608 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.913s	user 0.008s	sys 0.001s
I20260709 12:19:13.544951 32114 rpcz_store.cc:276] 0709 12:19:12.168065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.186137 (+ 18072us) service_pool.cc:224] Handling call
0709 12:19:12.186436 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.336680 (+150244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.464591 (+1127911us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540043b2d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:13.464778 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331935616}
I20260709 12:19:13.546571 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2389) took 1223 ms. Trace:
I20260709 12:19:13.641609 31966 rpcz_store.cc:276] 0709 12:19:12.323060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.240497 (+917437us) service_pool.cc:224] Handling call
0709 12:19:13.546494 (+305997us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:13.642655 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:13.642836 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.461113 32118 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:13.643992 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2462) took 1135 ms. Trace:
I20260709 12:19:13.644390 32118 rpcz_store.cc:276] 0709 12:19:12.508104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.241067 (+732963us) service_pool.cc:224] Handling call
0709 12:19:13.643922 (+402855us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:13.224567  4436 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.901s	user 0.013s	sys 0.001s
I20260709 12:19:13.268692  4530 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.646382 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:13.264382  4659 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.551736 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:13.216846  4375 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:12.970566  4676 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:13.657029 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:13.386286  4654 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.940s	user 0.000s	sys 0.009s
W20260709 12:19:13.657407  4680 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.687s	user 0.004s	sys 0.005s
I20260709 12:19:13.657804  4654 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.658232 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:13.658835 32114 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:13.659662 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:13.660152  4654 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:13.199127  4663 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:13.661366  4663 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.661770  4530 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:13.554145 31971 raft_consensus.cc:2368] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:13.651301  4436 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.663478 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2399) took 1161 ms. Trace:
I20260709 12:19:13.663869 31971 rpcz_store.cc:276] 0709 12:19:12.502059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.432038 (+929979us) service_pool.cc:224] Handling call
0709 12:19:13.663417 (+231379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:13.664763 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.665239  4436 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.665853 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:13.429219  3990 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.553s	user 0.017s	sys 0.005s
I20260709 12:19:13.556931 32046 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.666680  3990 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.668723  3990 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.668877 32657 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:13.558014 32656 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:13.670907  4486 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.671908 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 961) took 1280 ms. Trace:
I20260709 12:19:13.672418 32656 rpcz_store.cc:276] 0709 12:19:12.391016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.391144 (+   128us) service_pool.cc:224] Handling call
0709 12:19:12.417637 (+ 26493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.670586 (+1252949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.671591 (+  1005us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001ebc54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:13.671823 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622007552}
I20260709 12:19:13.674647 32119 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:13.582026 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:13.675412 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.610762  4685 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:13.314777  4677 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:19:13.392860  4656 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.656s	user 0.007s	sys 0.002s
W20260709 12:19:13.688756 32010 leader_election.cc:336] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.553s (SENT)
I20260709 12:19:13.585469 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:13.287263  4660 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:13.240669  4550 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:13.589910 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 716) took 1422 ms. Trace:
I20260709 12:19:13.637182  4532 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.602198  4240 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.634152  4682 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.597823  4655 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.681608  4693 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:13.699100 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.734356  4550 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.734793  4693 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:13.735031 31971 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:13.735764  4240 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.735963 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.735707  4693 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.689194 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:12.821291  4549 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.724222 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:13.550645  4691 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:13.738005  4691 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:13.738534  4549 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.312613  4678 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.274s
I20260709 12:19:13.485414 32652 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:13.613559 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:13.614141 31967 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.616540 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2388) took 1379 ms. Trace:
I20260709 12:19:13.616703 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:13.551137  4649 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.300138  4405 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.541548  4497 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:13.609375  4591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.019s	sys 0.019s
I20260709 12:19:13.550693 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:13.651788 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:13.598067  4571 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:13.648450 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.655841 32040 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.666150 32659 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:13.676029 31965 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:13.670066 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:13.677201 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.678333 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.326941  4573 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.584290  4421 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:13.685612 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 103663us
I20260709 12:19:13.684057 32046 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:13.710342  4375 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:13.737198 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.739382 32042 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:13.707577 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2434) took 1344 ms. Trace:
I20260709 12:19:13.706133 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2453) took 2863 ms. Trace:
I20260709 12:19:13.739336 32112 rpcz_store.cc:276] 0709 12:19:12.167765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.184538 (+ 16773us) service_pool.cc:224] Handling call
0709 12:19:12.184838 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.185181 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.185214 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.185685 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.487602 (+301917us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:12.487658 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.487736 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.589662 (+1101926us) raft_consensus.cc:1685] finished
0709 12:19:13.589681 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.589851 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2140,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":296613,"threads_started":1,"wal-append.queue_time_us":498016}
I20260709 12:19:13.589809  4506 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.029968  4351 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:13.739657 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 960) took 1405 ms. Trace:
I20260709 12:19:13.774524 31970 rpcz_store.cc:276] 0709 12:19:12.236751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.422100 (+1185349us) service_pool.cc:224] Handling call
0709 12:19:13.616474 (+194374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:13.776073 32043 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:13.770852 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.647912  4688 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.648322  4608 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.553567  4473 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.657298  4676 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:13.722047 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:14.134582 32119 rpcz_store.cc:276] 0709 12:19:12.362896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.239765 (+876869us) service_pool.cc:224] Handling call
0709 12:19:13.707393 (+467628us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400149bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:13.707520 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487451392}
I20260709 12:19:14.135057 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.135705 32042 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:13.957321 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 313212us
W20260709 12:19:14.109200 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.136979 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:13.702394 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.709633  3991 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.140014 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.140272 31966 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:14.153620 32116 rpcz_store.cc:276] 0709 12:19:10.842677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.842858 (+   181us) service_pool.cc:224] Handling call
0709 12:19:10.844186 (+  1328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.014803 (+1170617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.705759 (+1690956us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000807d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:13.706036 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666654464}
I20260709 12:19:14.155221 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.736466  4550 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.140656  4571 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.156118 31966 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.156993 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.157709  4571 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:13.724778  4603 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.416s
W20260709 12:19:13.687103  4650 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.937s	user 0.012s	sys 0.000s
I20260709 12:19:14.158155 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.158778  4650 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:13.728072 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.159376 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2397) took 1681 ms. Trace:
I20260709 12:19:14.159920 31966 rpcz_store.cc:276] 0709 12:19:12.478201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.157909 (+1679708us) service_pool.cc:224] Handling call
0709 12:19:14.159302 (+  1393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.160645 32654 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:14.160952 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.142270 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.161916  4603 consensus_queue.cc:1243] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:19:13.711021  4533 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.163879 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.165025 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:13.700139  4530 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.172472 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.136992 31963 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:13.894168 32115 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:14.177551 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2407) took 1425 ms. Trace:
I20260709 12:19:14.178021 31963 rpcz_store.cc:276] 0709 12:19:12.752197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.632814 (+880617us) service_pool.cc:224] Handling call
0709 12:19:14.177483 (+544669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.179118 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.179737 31963 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17. Re-sending same reply.
I20260709 12:19:14.180380 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2409) took 1246 ms. Trace:
I20260709 12:19:14.180783 31963 rpcz_store.cc:276] 0709 12:19:12.934221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.178847 (+1244626us) service_pool.cc:224] Handling call
0709 12:19:14.180272 (+  1425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.181474 32047 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:14.181907 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.182605 31963 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17. Re-sending same reply.
I20260709 12:19:14.183164 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2410) took 1185 ms. Trace:
I20260709 12:19:14.183562 31963 rpcz_store.cc:276] 0709 12:19:12.997397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.181655 (+1184258us) service_pool.cc:224] Handling call
0709 12:19:14.183109 (+  1454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.184315 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.190474 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.150841 31964 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:19:14.151930 31965 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:14.194271 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.195134 31965 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:14.195950 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.196529 31965 raft_consensus.cc:2376] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12. Re-sending same reply.
I20260709 12:19:14.197422 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:13.689579  4685 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.152172 32660 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.198778  3991 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:14.199043  4682 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.199311  3991 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.199822  3991 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.710399  4642 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.202551 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2403) took 1527 ms. Trace:
I20260709 12:19:14.191376  4375 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.202960 31964 rpcz_store.cc:276] 0709 12:19:12.675309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.629986 (+954677us) service_pool.cc:224] Handling call
0709 12:19:14.202489 (+572503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.204034 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.204752 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2396) took 1739 ms. Trace:
I20260709 12:19:14.205097 31964 rpcz_store.cc:276] 0709 12:19:12.465610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.203744 (+1738134us) service_pool.cc:224] Handling call
0709 12:19:14.204696 (+   952us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.204761  4375 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:13.683730  4514 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.459s
I20260709 12:19:14.161981 32046 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:14.210738  4617 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.211825 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:14.214581 32045 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:14.216295 31967 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:14.217089 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2404) took 1529 ms. Trace:
I20260709 12:19:14.217545 31967 rpcz_store.cc:276] 0709 12:19:12.687611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.542049 (+854438us) service_pool.cc:224] Handling call
0709 12:19:14.217020 (+674971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:13.803320  4660 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.232409 32043 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:19:14.147472  4677 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:19:14.243069 31967 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:19:14.148344  4497 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:14.245486 32044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.004s	sys 0.048s
W20260709 12:19:14.245957 32044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.004s	sys 0.049s
I20260709 12:19:14.246496 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2164) took 2220 ms. Trace:
I20260709 12:19:14.246903 31967 rpcz_store.cc:276] 0709 12:19:12.026575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.242153 (+2215578us) service_pool.cc:224] Handling call
0709 12:19:14.242607 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.246214 (+  3607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.246442 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.248744 31967 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:14.249430 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2383) took 2109 ms. Trace:
I20260709 12:19:14.137791  4591 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:14.175498  4532 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.176658 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.282382 31970 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:14.283843 31970 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:14.284344 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2384) took 2124 ms. Trace:
I20260709 12:19:14.284643 31970 rpcz_store.cc:276] 0709 12:19:12.159542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.283230 (+2123688us) service_pool.cc:224] Handling call
0709 12:19:14.283511 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.284129 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.284296 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.291066 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2166) took 1944 ms. Trace:
I20260709 12:19:14.180783 32658 rpcz_store.cc:276] 0709 12:19:12.334569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.337047 (+  2478us) service_pool.cc:224] Handling call
0709 12:19:12.337438 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.694843 (+357405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.739447 (+1044604us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400603bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:13.739604 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819968896}
I20260709 12:19:14.291618 31970 rpcz_store.cc:276] 0709 12:19:12.346218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.289782 (+1943564us) service_pool.cc:224] Handling call
0709 12:19:14.290482 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.290746 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.290787 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.290829 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:14.290864 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.290916 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.291016 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.293732 31970 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:14.150028  4676 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.188347 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:14.189059 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52180.4us
I20260709 12:19:14.302736 32119 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.189742 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.190091 32121 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.192081 32112 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:13.587432  4679 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.215075 31969 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:19:14.215396 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2370) took 2640 ms. Trace:
I20260709 12:19:14.304872 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.305197 31964 rpcz_store.cc:276] 0709 12:19:11.574923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.205770 (+2630847us) service_pool.cc:224] Handling call
0709 12:19:14.206128 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.214909 (+  8781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.214947 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.215016 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:14.215073 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.215172 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.215331 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.198235 31965 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:14.165434 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.306579 32121 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.307297 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2431) took 1956 ms. Trace:
I20260709 12:19:14.308228 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2438) took 1889 ms. Trace:
I20260709 12:19:14.308655 32112 rpcz_store.cc:276] 0709 12:19:12.418533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.184060 (+1765527us) service_pool.cc:224] Handling call
0709 12:19:14.308157 (+124097us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.310673 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2436) took 1910 ms. Trace:
I20260709 12:19:14.311139 32112 rpcz_store.cc:276] 0709 12:19:12.400592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.309553 (+1908961us) service_pool.cc:224] Handling call
0709 12:19:14.309944 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.310181 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.310226 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.310293 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:14.310377 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.310458 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.310603 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.313499 32118 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.232273 32042 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:14.315924 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.316771 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:14.318882 32626 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:14.235899 32046 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:14.233780 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.320597 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.320851 32043 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.321233 32046 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:13.789237 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.166116  4688 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.239478 32040 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:14.155221 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.331853 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.243129 32114 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.854910  4686 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:14.173775 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.249854 31967 rpcz_store.cc:276] 0709 12:19:12.139921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.247935 (+2108014us) service_pool.cc:224] Handling call
0709 12:19:14.248326 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.249154 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.249368 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.184597 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.294623 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2167) took 1948 ms. Trace:
I20260709 12:19:14.340001 31970 rpcz_store.cc:276] 0709 12:19:12.346603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.292984 (+1946381us) service_pool.cc:224] Handling call
0709 12:19:14.293306 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.294236 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.294556 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.341468 31970 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:14.341990 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2176) took 1041 ms. Trace:
I20260709 12:19:14.334926  4637 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:14.301378  4532 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.342289 31970 rpcz_store.cc:276] 0709 12:19:13.301051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.340842 (+1039791us) service_pool.cc:224] Handling call
0709 12:19:14.341182 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.341786 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.341941 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25728}
I20260709 12:19:14.371994  4591 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.372895 31970 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:14.343147  4688 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:14.343565 32656 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:14.345817 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.346141 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.377868 32113 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:14.379279 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:14.379777 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 717) took 1887 ms. Trace:
I20260709 12:19:14.349016 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 603) took 1144 ms. Trace:
I20260709 12:19:14.349251 32046 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:14.383298 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.380285 32113 rpcz_store.cc:276] 0709 12:19:12.492230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.378776 (+1886546us) service_pool.cc:224] Handling call
0709 12:19:14.379129 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.379343 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.379383 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.379446 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:14.379492 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.379576 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.379703 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.384502 32046 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:19:14.385946 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2460) took 1878 ms. Trace:
I20260709 12:19:14.344645 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 858) took 2112 ms. Trace:
W20260709 12:19:14.354405 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 665611us
W20260709 12:19:14.354550 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 505348us
I20260709 12:19:14.354526 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 711) took 2188 ms. Trace:
W20260709 12:19:14.352993 32038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 598) took 1743 ms (client timeout 1810 ms). Trace:
I20260709 12:19:14.365427 31969 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:14.371101 31963 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:14.364177 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2170) took 1987 ms. Trace:
I20260709 12:19:14.394397 31964 rpcz_store.cc:276] 0709 12:19:12.376625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.306271 (+1929646us) service_pool.cc:224] Handling call
0709 12:19:14.363342 (+ 57071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.363635 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.363683 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.363753 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.363809 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.363913 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.364090 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:14.390868 32038 rpcz_store.cc:269] 0709 12:19:12.610169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.673878 (+ 63709us) service_pool.cc:224] Handling call
0709 12:19:14.352828 (+1678950us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54001a54d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:14.352937 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86649600}
I20260709 12:19:14.398759 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.392112 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2392) took 2036 ms. Trace:
I20260709 12:19:14.400058 31969 rpcz_store.cc:276] 0709 12:19:12.355292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.305473 (+1950181us) service_pool.cc:224] Handling call
0709 12:19:14.364907 (+ 59434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.391762 (+ 26855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.392033 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.386683 32653 rpcz_store.cc:276] 0709 12:19:12.231788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.231969 (+   181us) service_pool.cc:224] Handling call
0709 12:19:12.974959 (+742990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.194727 (+219768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.344393 (+1149666us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b540063e550. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:14.344563 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481194752}
I20260709 12:19:14.300006 32045 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:14.240517  4506 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:14.303265 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2433) took 1940 ms. Trace:
I20260709 12:19:14.303247 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:14.242465  4460 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:14.331858 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 718) took 1087 ms. Trace:
I20260709 12:19:14.405846 32119 rpcz_store.cc:276] 0709 12:19:12.362604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.188052 (+1825448us) service_pool.cc:224] Handling call
0709 12:19:14.303191 (+115139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.406246 32116 rpcz_store.cc:276] 0709 12:19:13.244620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.155002 (+910382us) service_pool.cc:224] Handling call
0709 12:19:14.331786 (+176784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.407728 32119 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:13.730415  4586 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.333148 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.307765 32121 rpcz_store.cc:276] 0709 12:19:12.351056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.306043 (+1954987us) service_pool.cc:224] Handling call
0709 12:19:14.306378 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.306992 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.307221 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.246626  4662 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:13.777999  4656 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.317879 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.421833 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 963) took 1932 ms. Trace:
W20260709 12:19:14.370396  4668 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.395s	user 0.009s	sys 0.007s
I20260709 12:19:14.403676 32114 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:14.300750  4351 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.434418 32659 rpcz_store.cc:276] 0709 12:19:12.489004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.489160 (+   156us) service_pool.cc:224] Handling call
0709 12:19:12.499649 (+ 10489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.137443 (+1637794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.421506 (+284063us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540053edd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:14.421754 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682053376}
I20260709 12:19:14.409603 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:14.408449 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2461) took 1900 ms. Trace:
W20260709 12:19:14.410390 32010 leader_election.cc:336] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.784s (SENT)
I20260709 12:19:14.442931  4405 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.434209  4460 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.445865  4656 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.413340 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.381762 32041 rpcz_store.cc:276] 0709 12:19:13.204291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.204463 (+   172us) service_pool.cc:224] Handling call
0709 12:19:13.204819 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.205016 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:13.430886 (+225870us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.619120 (+188234us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.348329 (+729209us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b18003cb3e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:19:14.348544 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.348619 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:14.348689 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.348783 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.348943 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20803,"prepare.queue_time_us":225826,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":1472525,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":174,"spinlock_wait_cycles":14652672,"thread_start_us":184328,"threads_started":1,"wal-append.queue_time_us":257665}
I20260709 12:19:14.416314 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 862) took 1413 ms. Trace:
I20260709 12:19:14.446964 32047 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:14.413341 31970 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:14.415055 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 962) took 1966 ms. Trace:
I20260709 12:19:14.448729 32660 rpcz_store.cc:276] 0709 12:19:12.448501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.448732 (+   231us) service_pool.cc:224] Handling call
0709 12:19:12.449127 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.198526 (+1749399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.414709 (+216183us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400544054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:14.414971 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3916117888}
I20260709 12:19:14.388288  4608 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.390130 32117 rpcz_store.cc:276] 0709 12:19:12.166242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.166438 (+   196us) service_pool.cc:224] Handling call
0709 12:19:12.166709 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.238452 (+ 71743us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:12.549332 (+310880us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.917760 (+368428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:12.917840 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:12.917910 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.354192 (+1436282us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b5400439ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:14.354258 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.354437 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53558,"prepare.queue_time_us":506315,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":6889,"raft.queue_time_us":57819,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3545,"spinlock_wait_cycles":164318976,"thread_start_us":532858,"threads_started":2,"wal-append.queue_time_us":763557,"wal-append.run_cpu_time_us":1049,"wal-append.run_wall_time_us":1001063}
W20260709 12:19:14.366117  4677 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:19:14.433063 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.454845 32043 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 10: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:14.433072 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 129922us
I20260709 12:19:14.434981 32042 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.457301 32117 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:19:14.458086 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2466) took 1595 ms. Trace:
I20260709 12:19:14.458459 32117 rpcz_store.cc:276] 0709 12:19:12.862798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.456631 (+1593833us) service_pool.cc:224] Handling call
0709 12:19:14.457017 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.457803 (+   786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.458023 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.392199 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.459821 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 719) took 1214 ms. Trace:
I20260709 12:19:14.339318  4686 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:14.371264  4698 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:19:14.354772  4571 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.436637  4659 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.438530 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.388254  4688 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:14.345263  4472 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.484649 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2430) took 2143 ms. Trace:
I20260709 12:19:14.484989 32118 rpcz_store.cc:276] 0709 12:19:12.341188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.200363 (+1859175us) service_pool.cc:224] Handling call
0709 12:19:14.313192 (+112829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.335253 (+ 22061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.484466 (+149213us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540012bbd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:14.484593 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258733952}
I20260709 12:19:14.410960 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:14.486779 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75891.8us
I20260709 12:19:14.487417 32042 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:14.443037 32657 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:14.488109 32657 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:14.353883  4654 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:14.389756  3991 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.490783 32040 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:14.447127 32651 rpcz_store.cc:276] 0709 12:19:13.002599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.004200 (+  1601us) service_pool.cc:224] Handling call
0709 12:19:14.416098 (+1411898us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:14.416240 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236951296}
I20260709 12:19:14.367868  4247 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.370337  4685 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:14.448135 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.493112 32047 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.469493  4655 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:14.494948  4701 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:19:14.498382  4575 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:14.499075  4575 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:14.440661  4662 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.500126  4575 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.468813  4686 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.510373 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 955) took 2755 ms. Trace:
I20260709 12:19:14.510914 32655 rpcz_store.cc:276] 0709 12:19:11.755051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.755962 (+   911us) service_pool.cc:224] Handling call
0709 12:19:11.812641 (+ 56679us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:12.121079 (+308438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.136563 (+ 15484us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:12.145534 (+  8971us) log.cc:844] Serialized 52 byte log entry
0709 12:19:12.598630 (+453096us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:12.598694 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.598810 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.202613 (+1603803us) spinlock_profiling.cc:243] Waited 9.94 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:14.510011 (+307398us) raft_consensus.cc:1685] finished
0709 12:19:14.510042 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.510263 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57478,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":22866304,"thread_start_us":703410,"threads_started":2,"wal-append.queue_time_us":800489}
I20260709 12:19:14.421499  4649 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.351882  4651 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.513754 32042 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.501680  4654 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:14.514727  4654 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:14.515762 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.517542 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.461457 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:14.518496 32120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2417) took 2333 ms (client timeout 2069 ms). Trace:
I20260709 12:19:14.386507 32113 rpcz_store.cc:276] 0709 12:19:12.507388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.384903 (+1877515us) service_pool.cc:224] Handling call
0709 12:19:14.385201 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.385395 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.385424 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.385467 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.385498 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.385689 (+   191us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.385866 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.362406  4532 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.467631 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.521716  3991 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:14.452200  4586 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:14.450681  4702 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:14.354962  4288 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:19:14.441210 32119 rpcz_store.cc:276] 0709 12:19:12.507771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.406954 (+1899183us) service_pool.cc:224] Handling call
0709 12:19:14.407319 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.408150 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.408387 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.525867 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.527079 32113 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:14.482961 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 758) took 1289 ms. Trace:
W20260709 12:19:14.501381  4486 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.830s	user 0.009s	sys 0.002s
I20260709 12:19:14.482173 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.505374  4472 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.506295  4660 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.466198  4473 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.509289 32118 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:14.534355 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 722) took 1074 ms. Trace:
I20260709 12:19:14.534650 32654 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:19:14.534845 32118 rpcz_store.cc:276] 0709 12:19:13.459692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.508808 (+1049116us) service_pool.cc:224] Handling call
0709 12:19:14.509109 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.534071 (+ 24962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.534267 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.486370  4688 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:14.518896 32120 rpcz_store.cc:269] 0709 12:19:12.168650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.170409 (+  1759us) service_pool.cc:224] Handling call
0709 12:19:14.485813 (+2315404us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002b57d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:14.502032 (+ 16219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2688120064}
I20260709 12:19:14.445978  4460 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:14.520465 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.460144 32117 rpcz_store.cc:276] 0709 12:19:13.244987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.459157 (+1214170us) service_pool.cc:224] Handling call
0709 12:19:14.459393 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.459555 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.459578 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.459616 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:14.459646 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.459692 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.459776 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:14.489075 32657 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.542860 32657 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:14.522836 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61439us
I20260709 12:19:14.523860 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 568) took 3128 ms. Trace:
W20260709 12:19:14.501403  4691 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.763s	user 0.004s	sys 0.006s
I20260709 12:19:14.527153 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.553500 32046 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:14.554586 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.508203  4421 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:14.527753 32656 rpcz_store.cc:276] 0709 12:19:13.194269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.687353 (+493084us) service_pool.cc:224] Handling call
0709 12:19:13.811367 (+124014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.375216 (+563849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.482677 (+107461us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54004a2c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:14.482892 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222893312}
W20260709 12:19:14.521426  3990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.012s	sys 0.020s
I20260709 12:19:14.530031 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.531877 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:14.563391 32046 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.563544 32041 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 12:19:14.537223 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:14.537675 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 399831us
W20260709 12:19:14.536504  4675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.037s	sys 0.075s
I20260709 12:19:14.567147 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.567355 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.568089 31970 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:14.568379 32046 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:14.569329 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.539057 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.570446 32042 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.571343 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.572180 31969 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:14.573406 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.574738 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.575546 31966 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:14.578564 32112 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:14.526713  4649 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:14.557133 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.574218 31963 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:14.546525 31968 rpcz_store.cc:276] 0709 12:19:11.390287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.224922 (+834635us) service_pool.cc:224] Handling call
0709 12:19:12.225242 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.236312 (+ 11070us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:12.338698 (+102386us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.821742 (+483044us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b18002a0e20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:13.215691 (+393949us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.216560 (+   869us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.216695 (+   135us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.308649 (+ 91954us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.308847 (+   198us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:13.308897 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.309083 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.716694 (+407611us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b18002a0e20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f
0709 12:19:13.718187 (+  1493us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b480036c0d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:14.518737 (+800550us) raft_consensus.cc:1685] finished
0709 12:19:14.518757 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.518878 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11171,"prepare.queue_time_us":210115,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":152,"raft.queue_time_us":171967,"raft.run_cpu_time_us":10240,"raft.run_wall_time_us":578235,"spinlock_wait_cycles":376095616,"thread_start_us":780377,"threads_started":2,"wal-append.queue_time_us":581614}
I20260709 12:19:14.581555 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.592799 32046 raft_consensus.cc:2368] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13.
I20260709 12:19:14.593751 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:14.594676 32046 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:14.583554 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.596920 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.597877 31963 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:14.600498 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.601579 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.537540  4702 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:14.615573 32046 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:14.542536  4688 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.610657 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 13 index: 5 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.598767 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:14.567500  4675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.037s	sys 0.075s
I20260709 12:19:14.613445 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.612385 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.541428  4579 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.616500 32041 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.617862  4702 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.583036  4421 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:14.615957  4486 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:19:14.619153 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:14.619626  4532 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:14.566502  3990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.012s	sys 0.020s
I20260709 12:19:14.624437 32046 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:14.650635 31970 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:14.625607  4654 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.562938  4691 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.631313 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.669651 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 602) took 2058 ms. Trace:
I20260709 12:19:14.670159 32044 rpcz_store.cc:276] 0709 12:19:12.611718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.614385 (+  2667us) service_pool.cc:224] Handling call
0709 12:19:12.614795 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.601455 (+986660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.601489 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.601984 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.246452 (+644468us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:14.246489 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.246569 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.669379 (+422810us) raft_consensus.cc:1685] finished
0709 12:19:14.669408 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.669581 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53351,"raft.queue_time_us":94461,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":1503796,"threads_started":2,"wal-append.queue_time_us":943107}
I20260709 12:19:14.672665 32044 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:14.632706 32047 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:14.633304 31969 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:14.634804 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.638010 32657 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:14.641609 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.641841 32041 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:14.646477 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:14.650828 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.659394 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:14.612387 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2172) took 1637 ms. Trace:
I20260709 12:19:14.668537  4711 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:14.632718 31966 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:14.849409 32042 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:14.849902 31969 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:14.850330 31969 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:14.756350 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:14.755486 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:14.852543 31970 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:14.852609 31964 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:19:14.762140 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 319406us
W20260709 12:19:14.761842 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 312540us
I20260709 12:19:14.813937 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2205) took 1268 ms. Trace:
W20260709 12:19:14.853783 32010 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.817s (SENT)
I20260709 12:19:14.868337 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:14.867895 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.021s	sys 0.013s
I20260709 12:19:14.867444 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 863) took 1328 ms. Trace:
W20260709 12:19:14.870039 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.021s	sys 0.014s
W20260709 12:19:14.870685 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.22431e+06us
I20260709 12:19:14.870290 32657 rpcz_store.cc:276] 0709 12:19:13.539504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.138672 (+599168us) service_pool.cc:224] Handling call
0709 12:19:14.139011 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.442929 (+303918us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b54005eb8a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c146dc0 00007f228c1d11c0 00007f228c1d2231 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c
0709 12:19:14.867205 (+424276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.867392 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53733632}
I20260709 12:19:14.853663 31965 rpcz_store.cc:276] 0709 12:19:12.974670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.383377 (+1408707us) service_pool.cc:224] Handling call
0709 12:19:14.420115 (+ 36738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.610058 (+189943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.610092 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.610143 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.610181 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.610261 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.612325 (+  2064us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":232992,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":66,"thread_start_us":174573,"threads_started":1}
I20260709 12:19:14.868993 31968 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.868084 32039 rpcz_store.cc:276] 0709 12:19:13.545636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.545823 (+   187us) service_pool.cc:224] Handling call
0709 12:19:13.588083 (+ 42260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.763009 (+174926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.763057 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.131091 (+368034us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.535375 (+404284us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.535422 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.535503 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.813665 (+278162us) raft_consensus.cc:1685] finished
0709 12:19:14.813694 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.813864 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":642491,"raft.queue_time_us":51679,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":37729,"thread_start_us":578282,"threads_started":2,"wal-append.queue_time_us":673892}
I20260709 12:19:14.860495 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2464) took 2348 ms. Trace:
I20260709 12:19:14.648228  4421 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.876338  4685 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.851055 31969 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:14.880321 31969 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:14.877856 32010 leader_election.cc:336] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.982s (SENT)
I20260709 12:19:14.867005 31963 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.882055 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:14.886857 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:14.830825  4704 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:19:14.885480 32044 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:14.905556 32044 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.875456 32116 rpcz_store.cc:276] 0709 12:19:12.511890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.407348 (+1895458us) service_pool.cc:224] Handling call
0709 12:19:14.407737 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.408021 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:14.438433 (+ 30412us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.860039 (+421606us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.860113 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.860177 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.860268 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.860415 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35413,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":44,"raft.queue_time_us":100451,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":10724,"thread_start_us":323574,"threads_started":1}
I20260709 12:19:14.906920 32044 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:19:14.907644 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 420042us
W20260709 12:19:14.889962 32010 leader_election.cc:336] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.988s (SENT)
I20260709 12:19:14.909739 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:14.911480 32010 leader_election.cc:336] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.071s (SENT)
I20260709 12:19:14.911901 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:14.915273 32010 leader_election.cc:336] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.566s (SENT)
I20260709 12:19:14.915773 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:14.882452  4536 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:19:14.917488 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.926826  4654 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:14.927125 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.927731  4654 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:14.928632 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.928593  4654 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:14.929579 32120 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:14.934360 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.882432  4421 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:14.789175  4460 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 12:19:14.650190  4579 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.937577 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:14.908957  4288 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:14.888461  4710 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.940733 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:14.945767 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 316497us
I20260709 12:19:14.945868 32120 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:14.947190 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.917750  4506 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:14.949565 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:14.847713  4718 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:19:14.951278 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:14.937088  4709 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:14.740713  4686 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.938383  4711 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:14.954988  3990 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.951864  4532 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.962469  4405 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:14.966265  4709 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:15.000777  4591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.013s	sys 0.034s
I20260709 12:19:15.004746 32652 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:14.973361  4718 consensus_queue.cc:1243] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:14.972031  4506 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:15.010536  4686 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:15.010771  4506 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:14.951650  4710 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.001415  3990 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.004007  4532 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:14.970501  4716 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:15.000998  4722 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:15.020730  4571 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 12:19:15.027657  4711 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:15.044199  4711 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.044363  4579 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:15.031250  4656 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:19:15.034741  4671 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 2.063s	user 0.009s	sys 0.016s
I20260709 12:19:15.037978  4532 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.006139  4691 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:15.018534  4591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.013s	sys 0.034s
W20260709 12:19:15.143536 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.008s	sys 0.008s
W20260709 12:19:15.162492 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.009s	sys 0.008s
W20260709 12:19:15.143903 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.144352 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.000s	sys 0.011s
I20260709 12:19:15.145860 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2174) took 1905 ms. Trace:
I20260709 12:19:15.146131 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2211) took 1544 ms. Trace:
I20260709 12:19:14.957042 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:15.148337 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 725) took 1488 ms. Trace:
I20260709 12:19:14.962479 32120 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:14.960206 32113 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:15.049580  4656 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:15.054901  4660 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:15.265055  4660 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:15.267205  4660 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.269207  4660 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:15.076977  4662 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.073865  4679 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.097221  4688 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.312021  4688 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:15.060413  4710 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.045372  4711 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.041168  4722 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.177023 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2439) took 2706 ms. Trace:
W20260709 12:19:15.177448 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.000s	sys 0.012s
W20260709 12:19:15.314760 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.170675 31967 rpcz_store.cc:276] 0709 12:19:13.239757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.340784 (+1101027us) service_pool.cc:224] Handling call
0709 12:19:14.341172 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.341501 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.341529 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.341973 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.745187 (+403214us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.745248 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.745346 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.144977 (+399631us) raft_consensus.cc:1685] finished
0709 12:19:15.145068 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.145527 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":326678,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"spinlock_wait_cycles":32768,"thread_start_us":402842,"threads_started":1,"wal-append.queue_time_us":743201}
W20260709 12:19:15.315109 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 698969us
I20260709 12:19:15.182767  4506 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.048434  4571 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:15.062227  4709 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.216899 31965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
W20260709 12:19:15.246155 32115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2432) took 2889 ms (client timeout 2067 ms). Trace:
I20260709 12:19:15.261703 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2149) took 4004 ms. Trace:
W20260709 12:19:15.276459 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 319305us
I20260709 12:19:15.277220  3990 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.277560  4532 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.322019 32043 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:15.322152  4532 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:15.322904  4532 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.320156 31972 rpcz_store.cc:276] 0709 12:19:11.182650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.148895 (+966245us) service_pool.cc:224] Handling call
0709 12:19:12.166796 (+ 17901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.191568 (+ 24772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.191597 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.196793 (+  5196us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.701436 (+504643us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:12.701795 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.702190 (+   395us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.033964 (+2331774us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:15.169951 (+135987us) raft_consensus.cc:1685] finished
0709 12:19:15.169995 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.186787 (+ 16792us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27192,"raft.queue_time_us":24605,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":16655,"spinlock_wait_cycles":24661632,"thread_start_us":480416,"threads_started":1,"wal-append.queue_time_us":769629}
W20260709 12:19:15.278918 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.314512 32112 rpcz_store.cc:276] 0709 12:19:12.470356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.312575 (+1842219us) service_pool.cc:224] Handling call
0709 12:19:14.312942 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.579131 (+266189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.176501 (+597370us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540012acd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:15.176841 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610110208}
I20260709 12:19:15.280910 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 965) took 1091 ms. Trace:
I20260709 12:19:15.170748 32038 rpcz_store.cc:276] 0709 12:19:13.601199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.398904 (+797705us) service_pool.cc:224] Handling call
0709 12:19:14.399206 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.399436 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:14.726562 (+327126us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.145062 (+418500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:15.145194 (+   132us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:15.145364 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.145536 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.145802 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":636120,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":12967,"thread_start_us":708906,"threads_started":2}
W20260709 12:19:15.295011 32045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2210) took 1703 ms (client timeout 1959 ms). Trace:
I20260709 12:19:15.199326 32119 rpcz_store.cc:276] 0709 12:19:13.660138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.526946 (+866808us) service_pool.cc:224] Handling call
0709 12:19:14.527204 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.527532 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.527563 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.528003 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.904074 (+376071us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:14.904210 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.904321 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.147573 (+243252us) raft_consensus.cc:1685] finished
0709 12:19:15.147618 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.148003 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1113,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6908,"thread_start_us":346927,"threads_started":1,"wal-append.queue_time_us":534742}
W20260709 12:19:15.319871 32115 rpcz_store.cc:269] 0709 12:19:12.356259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.240425 (+884166us) service_pool.cc:224] Handling call
0709 12:19:15.245584 (+2005159us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b540043add4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.245808 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061724416}
I20260709 12:19:15.278098  4656 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.326560  4591 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:15.325721 32654 rpcz_store.cc:276] 0709 12:19:14.189231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.189493 (+   262us) service_pool.cc:224] Handling call
0709 12:19:15.280318 (+1090825us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54003efb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.280620 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529332480}
I20260709 12:19:15.332413 31965 raft_consensus.cc:606] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:15.332679  4405 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:15.333735  4709 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.335721  4460 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:15.336243  4460 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.338092  4506 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.341361  4718 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:19:15.325985 32045 rpcz_store.cc:269] 0709 12:19:13.591334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.613200 (+ 21866us) service_pool.cc:224] Handling call
0709 12:19:15.294519 (+1681319us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400217854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.294744 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010122112}
I20260709 12:19:15.344358  4530 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:15.345260  4530 consensus_queue.cc:1243] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:19:15.345813 32652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 864) took 1634 ms (client timeout 2178 ms). Trace:
I20260709 12:19:15.355572  4591 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.362716  4729 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:15.352849 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:15.364130  4741 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:19:15.365214 32114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2437) took 2946 ms (client timeout 3208 ms). Trace:
I20260709 12:19:15.365228  4676 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:15.365367  4741 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.366036  4676 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:15.365836 32114 rpcz_store.cc:269] 0709 12:19:12.418225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.659461 (+1241236us) service_pool.cc:224] Handling call
0709 12:19:15.364750 (+1705289us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54003e4ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.364997 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1340640640}
I20260709 12:19:15.366887  4676 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.363193  4591 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.368235 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2213) took 1749 ms. Trace:
W20260709 12:19:15.369273 32040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2212) took 1752 ms (client timeout 2212 ms). Trace:
I20260709 12:19:15.368665 32043 rpcz_store.cc:276] 0709 12:19:13.619095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.462976 (+843881us) service_pool.cc:224] Handling call
0709 12:19:14.481183 (+ 18207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.322551 (+841368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.367931 (+ 45380us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54001950d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:15.368173 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931740416}
W20260709 12:19:15.347369 32652 rpcz_store.cc:269] 0709 12:19:13.710855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.837764 (+126909us) service_pool.cc:224] Handling call
0709 12:19:15.345310 (+1507546us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400592254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.345615 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529151872}
W20260709 12:19:15.369851 32040 rpcz_store.cc:269] 0709 12:19:13.616619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.332891 (+716272us) service_pool.cc:224] Handling call
0709 12:19:15.368923 (+1036032us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b54000578d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.369098 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156258560}
I20260709 12:19:15.374575 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.377300  4656 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:15.378866 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:15.381089 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.382472 31968 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
W20260709 12:19:15.384521 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.348367 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:15.386857 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57683.7us
I20260709 12:19:15.388633  4679 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:15.389106  4679 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.389523  4679 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:15.392086 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.392136  4654 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.394261  4571 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.395181  4405 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.395107  4571 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.396057  4405 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.397644  4571 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.393102  4654 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.398263  4571 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
W20260709 12:19:15.390194 31964 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.004s
I20260709 12:19:15.402772 31964 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:15.407050 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.401305  4654 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.337972  4460 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.409096  4460 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:15.410817 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44488.4us
I20260709 12:19:15.412213 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:15.413506 32046 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:15.406018  4709 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:15.414656  4709 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.415766  4709 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.418414  4405 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:15.410390 32121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.027s	sys 0.022s
I20260709 12:19:15.370848 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:15.430441 32121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.029s	sys 0.022s
I20260709 12:19:15.431644 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2465) took 2895 ms. Trace:
I20260709 12:19:15.432219 32121 rpcz_store.cc:276] 0709 12:19:12.536300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.423401 (+1887101us) service_pool.cc:224] Handling call
0709 12:19:14.423832 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.424120 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:14.761431 (+337311us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.431086 (+669655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:15.431197 (+   111us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:15.431331 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.431420 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.431586 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14671,"prepare.queue_time_us":19586,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":987141,"threads_started":2,"wal-append.queue_time_us":585561}
W20260709 12:19:15.368655 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 348390us
I20260709 12:19:15.435268 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.454137  4743 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.455240  4743 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:15.462015 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 121225us
I20260709 12:19:15.367218  4741 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.460829  4743 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:15.483777 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 72421.2us
I20260709 12:19:15.485492 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:15.485394  4743 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:15.487665  4743 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.488922  4747 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:15.490584  4743 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.490988  4743 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.492069  4743 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.493194  4743 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.479867 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.499506 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.503367 32038 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:15.507401 31963 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.004s
I20260709 12:19:15.507946 31963 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:15.513118 31867 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 12, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 12 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.515249 31867 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 13, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 13 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.516985 31867 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.526821  4530 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:15.528528  4530 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:15.529019  4530 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:15.530809 31969 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.006s
I20260709 12:19:15.531375  4709 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.533573 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:15.538672 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61729us
I20260709 12:19:15.539564 32654 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:19:15.539564 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:15.541486 32653 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.542996 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.543988 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.543016 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.545675 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.546366 32655 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:15.553697  4741 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:15.555718  4722 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.557205  4722 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.560082 32654 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.560321 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.561057 32656 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:15.559496 32655 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:19:15.562810 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.561544 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.564287 32660 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:15.562757 32657 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:15.567445 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.568048 32660 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:15.569084 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:15.571211 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 585825us
I20260709 12:19:15.573544 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:15.569896 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.562933 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.582015 32653 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:15.497742  4405 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:15.591284 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84393.6us
I20260709 12:19:15.592247 32657 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:19:15.597445  4685 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:15.597891 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.598858 32654 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 9.
I20260709 12:19:15.565920 31867 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 11, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 11 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.479046  4679 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.606777  4734 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.607373  4734 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.607802  4734 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.608140  4532 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:15.614671 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.621081  4747 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:19:15.574404  4729 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:19:15.620251  4532 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.650362  4711 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.655164 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.656453 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:15.656715 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:15.659297 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:15.659898 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:15.660130 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:15.660974 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 175688us
W20260709 12:19:15.671159 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.677500 32043 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:15.617424  4654 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:15.670704 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:15.592263  4530 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.678535 32047 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:15.697365  4530 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:15.697533  4421 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.002s
I20260709 12:19:15.681602 32038 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:15.681634 32039 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:15.682382 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:15.699507 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56004us
I20260709 12:19:15.701556  4506 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:15.701965  4506 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:15.683246  4729 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:15.703171  4729 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:15.703020  4506 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.703719  4729 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.704198  4506 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.704250  4729 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:15.707149  4716 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.004s
I20260709 12:19:15.707675  4716 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:15.690704 32041 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:15.672465  4685 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:15.708907  4685 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:15.709273  4685 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:15.710352  4685 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.711751  4685 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.675227 31867 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 12, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 12 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.714589 32045 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:15.714936  4532 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:19:15.716470  4591 consensus_queue.cc:1175] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "38c927f0ee104b57972899f5b27aa99b" responder_term: 12 status { last_received { term: 9 index: 8 } last_committed_idx: 8 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:15.717545  4532 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:15.717971  4532 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.695627  4654 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.684873 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:15.698145  4421 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.718434 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:15.731815 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 319704us
I20260709 12:19:15.726615 32652 raft_consensus.cc:2368] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20.
I20260709 12:19:15.720062 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:15.736066 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51409.7us
I20260709 12:19:15.740084 32047 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
W20260709 12:19:15.742470 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.726596 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:15.722362  4709 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.759186 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:15.718995  4532 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.723160  4710 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.723384  4747 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:15.769753 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45980.4us
I20260709 12:19:15.775552 32039 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.780601 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:15.781929  4749 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:15.784036  4421 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:15.760296  4709 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.767494  4710 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.792318  4710 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:15.797194 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.799256 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.801054  3990 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:15.804133 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:15.806028 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:15.753685  4405 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:15.810201 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.825155 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 66141.4us
I20260709 12:19:15.721783  4676 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.794116  4710 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:15.832872 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 132466us
W20260709 12:19:15.836477 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56087.8us
I20260709 12:19:15.812163  4660 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.839500 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:15.846377 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 120042us
I20260709 12:19:15.803228  3990 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 12:19:15.848749 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.849889  4472 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.293s	user 0.013s	sys 0.008s
I20260709 12:19:15.850283  4737 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:15.851004 31971 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:15.849501 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:15.852286 32011 leader_election.cc:400] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20.
W20260709 12:19:15.852963 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.853353 32011 leader_election.cc:403] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:19:15.851788  4742 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.856148  4679 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.837730  4716 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:15.857089 31971 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2417) took 2127 ms (client timeout 1691 ms). Trace:
I20260709 12:19:15.857326  4745 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 12:19:15.857564 31971 rpcz_store.cc:269] 0709 12:19:13.729414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.736969 (+  7555us) service_pool.cc:224] Handling call
0709 12:19:15.856850 (+2119881us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400312c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:15.856966 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3934577408}
I20260709 12:19:15.857120  4716 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:15.860757 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.861361  4716 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:15.854250  4405 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.7, Last appended by leader: 7, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:15.856186  4742 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.864972  4751 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:15.869828 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.873373 32114 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:15.865693 31938 connection.cc:256] RPC call timeout handler was delayed by -1.2396350900000925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32080000000000003s for a response
I20260709 12:19:15.875196  4747 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:15.875826  4747 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.878623 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:15.879200 31969 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:15.879424 31964 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:15.879848 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.880486 32652 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:19:15.880497 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:15.872459  4472 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:15.886848 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:15.887040  4472 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.887553 31964 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:15.887887 31963 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:15.844285  4460 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:15.892951 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 58717.9us
I20260709 12:19:15.898173  4460 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:15.898846  4749 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:15.899405  4749 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:15.899827  4749 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:15.899894  4460 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.901248  4460 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:15.898517 31867 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 13 to 14, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 14 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:15.903664  4654 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.904450 31867 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.907975 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.907871  4679 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:15.893096  4716 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:15.893399  4472 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.895244 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:15.909096 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.910058 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.917743 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.913609 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:15.913865 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:15.918612  4745 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:15.919726  4745 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:15.914618 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:15.920147  4745 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:15.920460 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:15.911655  4679 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.921322  4679 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:15.921166  4745 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.906186 31867 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 9, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 9 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.924817 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:15.925416 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 826cffce8bb540c8beb56cc00a702782 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:42276
I20260709 12:19:15.925475 31867 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:15.927616  4654 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.925657 32119 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:15.928061 31867 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 11, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 11 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.925629 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:15.827629  4709 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.931017  4654 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.931396 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:15.926988 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71718us
I20260709 12:19:15.927069  4745 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.916633 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:15.919899 32112 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.929735 31867 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 9 to 12, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 12 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:15.935909 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:15.936717 32658 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:15.936002 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:15.941021  4684 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:15.941768  4684 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:15.942072  4684 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:15.943159  4684 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.944819 32010 leader_election.cc:400] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:15.945183 32010 leader_election.cc:403] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:15.946458 32039 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:15.944744  4684 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.910005  4747 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:15.950435  4747 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:15.950618 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77477.6us
I20260709 12:19:15.952176 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:15.955137  4743 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:15.957178  4532 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:15.958851  4751 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:15.959321  4751 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.959697  4751 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.954348  4709 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.960052 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:15.961474 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89850.9us
I20260709 12:19:15.962098  4654 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:15.959712  4506 raft_consensus.cc:2764] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:15.969976  4676 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:15.971159 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:15.971688 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:15.972590  4654 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:15.972784 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:15.973634 31969 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:15.974084 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:15.975220 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:15.975807 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:15.976446 32119 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:15.977306 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:15.977913 32120 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:15.979075 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:15.979473 32113 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:19:15.982589 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:15.998587 31969 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:15.980496 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.003914 31867 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 11 to 12, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 12 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:16.009516 31965 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:16.010223 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.012467 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:16.013080 32657 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.011726 32112 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:15.866300  4734 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.018816 31867 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:16.017032 32113 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:16.015268 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.015268 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:15.998483  4532 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.021492 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.021930 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:16.010221  4506 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:16.025635 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.028206 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.028702 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.028993 32117 raft_consensus.cc:2368] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 14.
I20260709 12:19:16.029544 32120 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:19:16.038964 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67500.1us
I20260709 12:19:16.039892 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:16.040534 31866 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 244) took 31576 ms (client timeout 15000 ms). Trace:
I20260709 12:19:16.024844 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:16.041548 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.042038 32113 raft_consensus.cc:2368] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17.
W20260709 12:19:16.041929 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89898.3us
I20260709 12:19:16.042764 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.043314 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.043656  4591 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.044108 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.045290  4591 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.045622 32121 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:16.046435 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.046509  4530 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:16.048677 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.049661 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.051831 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.053660  4660 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:16.054916 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:16.055413 32115 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:16.055414 32119 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:16.041208 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.063139 31968 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:16.065613 31967 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:16.023507 31867 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 8 to 13, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 13 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:16.066993 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.012876  4737 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:16.070276 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.006896  4679 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.071416 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:16.078855 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.081786 32113 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.027076  3990 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.084410 32120 raft_consensus.cc:1217] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-15.7]   Dedup: 10.6->[15.7-15.7]
I20260709 12:19:16.084614 32658 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:16.086360  4591 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:16.086732  4571 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.087211  4702 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.093536  4716 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20.
I20260709 12:19:16.098964 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.064008  4660 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:19:16.119488 32121 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:16.099803 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.100860 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.101464 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.102124 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.102365 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 826cffce8bb540c8beb56cc00a702782 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:19:16.102964  4722 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:16.124644 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.074370  4685 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.104496  4734 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.107411 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:16.108014 32120 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [10.6-15.7]
I20260709 12:19:16.108167 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.075562 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:16.116621 31867 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 9 to 10, leader changed from <none> to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 10 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:16.130788  4758 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:16.108706 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62451.5us
I20260709 12:19:16.109771 31936 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.136857 32119 raft_consensus.cc:2368] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10.
I20260709 12:19:16.109412 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.071920  4742 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.122591 31972 raft_consensus.cc:2376] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15. Re-sending same reply.
I20260709 12:19:16.129426 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.130121 31970 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:16.121652 31966 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:16.027693  4749 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.105846  3990 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:16.131899 32657 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:16.130796  4405 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:16.133848 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.141021 31969 raft_consensus.cc:2376] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14. Re-sending same reply.
I20260709 12:19:16.150938 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:16.137714  4685 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:16.144201 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68764.7us
I20260709 12:19:16.146171 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.146971 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.147109  4532 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.147795 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.177742 31964 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:16.149336 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.145370 31867 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:16.151924 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.152757  4579 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:16.177321 31966 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18. Re-sending same reply.
I20260709 12:19:16.153342  4679 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.158803  4745 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:16.162726 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 86174.6us
I20260709 12:19:16.145742  4710 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:16.151006  3990 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.164844  4749 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.165921 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:16.109604  4654 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:19:16.187384  4579 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:16.199401  4579 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.188051  4737 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.189186 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:16.173659 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:16.176440 32114 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:16.208792 32654 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 9.
I20260709 12:19:16.212903 31970 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:16.185490 31963 raft_consensus.cc:2376] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:19:16.186446 31971 raft_consensus.cc:2393] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:16.186767 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 145631us
I20260709 12:19:16.179124  4685 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.190428 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.186416  4405 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:16.215576 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.214802 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.226966  4460 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:16.227831  4729 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.007s
I20260709 12:19:16.228295  4729 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.216550  4749 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:16.230943  4749 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.231294 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.233371  4679 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:16.233786  4679 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.234119  4679 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:16.237890 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.238682  4761 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.239162  4761 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:16.239646  4761 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.251519 31967 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
W20260709 12:19:16.040984 31866 rpcz_store.cc:269] 0709 12:18:44.464306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.465349 (+  1043us) service_pool.cc:224] Handling call
0709 12:19:16.040303 (+31574954us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:08.731557 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:08.904556 (+172999us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:08.919063 (+ 14507us) write_op.cc:435] Acquired schema lock
0709 12:19:08.924772 (+  5709us) tablet.cc:662] Decoding operations
0709 12:19:08.954928 (+ 30156us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:08.955044 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:08.955073 (+    29us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:19:08.955599 (+   526us) tablet.cc:701] Row locks acquired
0709 12:19:08.955651 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:19:08.955835 (+   184us) write_op.cc:270] Start()
0709 12:19:08.955984 (+   149us) write_op.cc:276] Timestamp: P: 1783599548955756 usec, L: 0
0709 12:19:08.956022 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:19:08.956754 (+   732us) log.cc:844] Serialized 1699 byte log entry
0709 12:19:10.436208 (+1479454us) op_driver.cc:464] REPLICATION: finished
0709 12:19:11.163411 (+727203us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18001ec1c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:11.163623 (+   212us) write_op.cc:301] APPLY: starting
0709 12:19:11.163725 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:11.262830 (+ 99105us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:11.268277 (+  5447us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:11.485653 (+217376us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:11.541938 (+ 56285us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:19:11.542008 (+    70us) write_op.cc:312] APPLY: finished
0709 12:19:11.578862 (+ 36854us) log.cc:844] Serialized 49 byte log entry
0709 12:19:11.741359 (+162497us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:11.747696 (+  6337us) write_op.cc:454] Released schema lock
0709 12:19:11.806382 (+ 58686us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:11.917312 (+110930us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:11.917358 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":147145,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":92168,"apply.run_cpu_time_us":83746,"apply.run_wall_time_us":753618,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20831,"num_ops":4,"prepare.queue_time_us":442018,"prepare.run_cpu_time_us":20142,"prepare.run_wall_time_us":768432,"raft.queue_time_us":265212,"raft.run_cpu_time_us":12753,"raft.run_wall_time_us":143892,"replication_time_us":1440644,"spinlock_wait_cycles":32460544,"thread_start_us":944006,"threads_started":5,"wal-append.queue_time_us":1520139,"wal-append.run_cpu_time_us":14894,"wal-append.run_wall_time_us":2211889}]]}
I20260709 12:19:16.255903 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.215916  4745 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:16.249785  4709 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:16.268579  4660 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:16.274703  4710 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.275187  4710 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:16.265708 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 121822us
I20260709 12:19:16.233392  4749 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.262827  4688 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.265552  4758 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:16.290939 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58046.6us
W20260709 12:19:16.296067 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 114935us
I20260709 12:19:16.268541  4729 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.278920  4709 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:16.308172 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.310573 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 87791.7us
W20260709 12:19:16.312913 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.315048 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.293042  4688 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.315861 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.317225 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.317699  4688 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.318636  4749 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:16.319828 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.314034 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:16.331195 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.319041  4749 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:16.334353 31938 leader_election.cc:400] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 14.
I20260709 12:19:16.336596 31938 leader_election.cc:403] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:16.340086 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.340911 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:16.341792  4767 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
I20260709 12:19:16.337878  4460 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:19:16.348855  4742 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 14.
W20260709 12:19:16.349912 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.351401  4532 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.351673 31938 leader_election.cc:400] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17.
I20260709 12:19:16.352305 31938 leader_election.cc:403] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:16.332633  4745 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.354482 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.330298  4729 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:16.355748 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.356678  4734 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:16.357429  4734 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17.
I20260709 12:19:16.357481 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.307271  4405 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.358099  4734 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:16.367681 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.370513  4660 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.371349  4660 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:16.363303 32119 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:16.365906  4750 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:16.376103 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.376741  4750 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:16.378825  4722 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:16.377846  4750 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:16.389235 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.339376  4749 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.394899 31938 leader_election.cc:400] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10.
I20260709 12:19:16.417201 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.418011 32118 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
W20260709 12:19:16.430071 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.398914 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:16.439034 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48522.4us
I20260709 12:19:16.440053 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.391978  4729 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:16.442529  4729 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:16.443608  4729 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.444306  4751 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.445178  4729 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.446414  3990 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.447486 32115 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:16.448465  3990 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:16.449255  4591 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.453326  4591 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:16.454464  4591 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:16.454921  4591 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:16.457482  4751 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:16.456771  4591 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.464327  4591 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:16.467450 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68873.8us
I20260709 12:19:16.468482 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.469506  4722 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:16.476593 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.374197  4710 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.484541  4710 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.485337  4685 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.486013 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:16.486860 32657 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.489074  4685 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.491432 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:16.492322 32654 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.392910  4702 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:16.498771  4702 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.498890 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:16.499750 32118 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.394683  4654 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:16.383256  4679 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.505067  4679 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:16.433785  4460 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:16.432435  4709 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.437795 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:16.469980 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:16.514228 32657 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
W20260709 12:19:16.511051 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 76102.9us
I20260709 12:19:16.523167 32654 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
W20260709 12:19:16.525245 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55372.6us
I20260709 12:19:16.522693 31936 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.526528 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.531674  4421 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:16.533828  4421 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.534279  4421 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:16.536374 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.538978 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 143210us
I20260709 12:19:16.540159 31935 leader_election.cc:400] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13.
I20260709 12:19:16.541265 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.542521  4771 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.523288  4460 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:16.545305 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.547428  4688 raft_consensus.cc:2764] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:19:16.548628 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.555761  4755 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.556352  4755 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:16.556804  4755 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.559224 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.543956 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:16.566097  4710 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.574676 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:16.576422  4722 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.577405  4734 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:16.578063  4734 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:16.579195  4734 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.582952  4722 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:16.572311 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56360.5us
W20260709 12:19:16.576472 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 134156us
W20260709 12:19:16.598764 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57620.3us
W20260709 12:19:16.602689 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.601868 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.605055 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:16.605232 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 296103us
W20260709 12:19:16.616945 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.653386 32115 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:16.626541 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67414.3us
I20260709 12:19:16.616710 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:16.639640 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.636049 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.600807  4591 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:16.605614  4771 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.601292  4722 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:16.647565 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.654237 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.655409 32118 raft_consensus.cc:2368] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10.
I20260709 12:19:16.657369 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.656019  4771 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.669435 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:16.673182 32114 raft_consensus.cc:2376] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16. Re-sending same reply.
I20260709 12:19:16.675336  4749 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.676657 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.677644 31969 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 10.
I20260709 12:19:16.671804  4771 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.678485  4749 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.669651  4761 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.686762  4676 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:16.686932 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:16.687759  4532 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:16.688354  4532 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:16.689594 32119 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:16.692600 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.692905 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.693307 31971 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:16.693502  4654 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:16.694453 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.695091 31963 raft_consensus.cc:2376] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11. Re-sending same reply.
W20260709 12:19:16.694978 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59082.6us
I20260709 12:19:16.696105 32121 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.695891 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.698259 31970 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:16.671177 31967 pending_rounds.cc:85] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 12
I20260709 12:19:16.689323  4532 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.702759 31938 leader_election.cc:400] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 9 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10.
I20260709 12:19:16.705681 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.706681 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:16.707636 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:16.707707 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:16.710587 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:16.710692 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.713085  4688 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:16.713260 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.713768  4688 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:16.714715 32046 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:16.714843  4688 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.716702  4460 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.717413  4460 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:16.718775 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 94757.9us
I20260709 12:19:16.719331  4702 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.719753 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.720590 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.696460  4676 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:16.721256 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.722491  4676 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.722668  4755 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.658263  4591 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:16.710522 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.725338 31968 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.724933  4591 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.726565  4709 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.724728  4722 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.698906 31967 pending_rounds.cc:107] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:19:16.729993 31967 raft_consensus.cc:2889] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:16.730002 32116 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:16.735065 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.736194 31969 raft_consensus.cc:2410] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:19:16.731194 32047 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:16.735219 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:16.737078 32045 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.739773 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:16.742111 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.686226  4761 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:16.742686 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:16.746766 31968 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:16.708323  4710 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.757059 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:16.758728 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:16.759213 31965 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:16.759594 32047 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.761229 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.761346 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:16.763837 32084 connection.cc:256] RPC call timeout handler was delayed by -2.0289621329998226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 12:19:16.763163 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:16.764238 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9983621329997732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260709 12:19:16.788007 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:16.788203 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.788676 32040 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.789309 31964 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 13: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:16.789806 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44876.5us
I20260709 12:19:16.709371  4654 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:19:16.792508 31968 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.795338 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:16.799710 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 145504us
I20260709 12:19:16.801235 32039 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:16.756018  4685 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.804409 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:16.762688  4722 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:16.810632 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.784190  4737 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.811280 31964 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.811553 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:16.810652 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68090.7us
I20260709 12:19:16.815126 31965 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.816258 32047 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:16.818248  4722 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:16.819453  4722 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:16.820748  4783 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:16.822592  4660 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:16.814112  4737 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:16.824856  4737 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.825810  4685 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:16.826365  4685 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.829134  4685 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.829881 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:16.813426 32046 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 6. Preceding OpId from leader: term: 19 index: 8. (index mismatch)
I20260709 12:19:16.842178 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:16.847273 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.848176 31970 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.850412  4749 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:16.854413  4729 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.855070  4729 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.858474  4755 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.849953 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:16.857839  4729 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:16.846577 31968 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:16.866266 31964 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:16.867551 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:16.867903 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.857766  4737 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.868211 32654 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:16.871894 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.873286  4710 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.873723  4710 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:16.874122  4710 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.879599 32040 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:16.881688 31936 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:16.882419 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.891150  4685 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:16.892196  4685 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:16.892647  4685 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:16.842346  4421 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.893611  4685 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:16.895756  4421 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:16.899175 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 111315us
I20260709 12:19:16.909107 31969 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:16.926221 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:16.926959 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 118363us
I20260709 12:19:16.927440 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:16.932474 32114 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:16.936744 31968 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.927716 31965 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:16.928798 32654 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.942662 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:16.944465 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:16.945436 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.946529 31964 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 11: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:16.927572  4792 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 12:19:16.883245  4729 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.963508  4676 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:16.964488 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:16.965198 32040 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:16.965260 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.967764 31965 raft_consensus.cc:2368] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:16.972421 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:16.972957 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.973201 32040 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:16.975112 32039 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:16.976063  4506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.024s	sys 0.014s
W20260709 12:19:16.976426  4506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.024s	sys 0.014s
I20260709 12:19:16.976657 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:16.977366 32042 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:16.965302  4729 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.980901  4729 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.980966 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:16.981781 32046 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
W20260709 12:19:16.982602 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.985462 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:16.986162 32039 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.990206  4775 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:16.990705  4775 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:16.991739  4775 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:16.993054  4775 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.995402 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:16.995831  4737 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:16.996121 32040 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
W20260709 12:19:16.998523 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 206308us
W20260709 12:19:16.999100 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9533621329999278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 12:19:16.999395 32084 connection.cc:256] RPC call timeout handler was delayed by -1.8741621329997997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 12:19:16.999655 32084 connection.cc:256] RPC call timeout handler was delayed by -1.725662132999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
W20260709 12:19:16.999907 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6646609909998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 12:19:17.000238 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3550609909998457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
W20260709 12:19:17.000535 32084 connection.cc:256] RPC call timeout handler was delayed by -1.3073609909999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
I20260709 12:19:17.000787 31970 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
W20260709 12:19:17.000829 32084 connection.cc:256] RPC call timeout handler was delayed by -1.2551609909999115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
I20260709 12:19:17.004273 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:17.002571 32039 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
W20260709 12:19:17.014539 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 81112.9us
I20260709 12:19:17.009002 32114 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:17.015791 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:17.013289 31969 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:17.007162 31968 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:17.018669 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.020933 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:17.021512 32042 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.022511 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:16.966938  4685 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.025988 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:17.049393 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.050784 31964 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:17.049252 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:17.051692 32046 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:16.953387  4771 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:17.066388 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64446us
I20260709 12:19:17.059890 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.050256  4460 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:17.093039 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43720.2us
I20260709 12:19:17.069453 31970 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 5. Preceding OpId from leader: term: 19 index: 8. (index mismatch)
I20260709 12:19:17.023550  4742 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.013708  4749 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.058941  4737 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:17.112703 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 96980.4us
I20260709 12:19:17.088140  4791 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.111845  4460 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:17.127349 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.097172  4722 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.103842  4421 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:17.128589  4460 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.125588  4709 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:17.132550  4702 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:17.128454 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:36592
I20260709 12:19:17.111418  4710 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.133080 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.123984  4734 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.123701  4791 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:17.137730 32039 raft_consensus.cc:606] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:17.152283 32039 raft_consensus.cc:614] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:17.121490  4742 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.125295  4737 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.141618  4421 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:17.129108  4676 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:19:17.127060  4749 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:17.164450 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 104662us
I20260709 12:19:17.154745  4791 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.166061 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:17.168272  4796 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:19:17.168152  4810 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.136581  4722 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:17.180106 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47615.7us
W20260709 12:19:17.189574 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65858.7us
I20260709 12:19:17.166268  4532 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:17.191246  4816 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.191989  4816 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.193960  4813 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.191308  4532 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:17.178120  4737 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.198546 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.155094  4734 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.194562  4810 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.207227 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:17.195353  4813 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.209429  4421 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:17.210258  4807 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:19:17.233659  4702 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.234814  4702 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:17.235245  4702 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.210299  4749 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:17.211055  4804 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:17.209045 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:17.211897  4805 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:19:17.241415  4806 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
I20260709 12:19:17.242075 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:17.222066  4801 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:19:17.194172  4816 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:17.243357  4802 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 12:19:17.224365 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 103233us
I20260709 12:19:17.244283 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.223446 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:17.246608 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.247210  4684 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.248431 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.249053 32121 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:17.248867  4816 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:17.248486  4734 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.250991 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:17.251410  4816 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:17.251888  4816 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:17.252897  4816 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.256557  4742 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.247212 32113 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:17.228119 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:17.257992 32115 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.258862 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:17.246169 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:17.267282 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.267995 32114 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:17.268847 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.268949  4709 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:17.269703  4709 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:17.270985  4709 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.277375  4685 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.277891  4685 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.278231  4685 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:17.279136  4797 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.284s	user 0.001s	sys 0.003s
W20260709 12:19:17.280313  4808 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:17.281692  4532 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.277911  4688 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:17.282863  4688 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:17.283789  4688 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:17.285043  4783 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:19:17.307765 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 108205us
I20260709 12:19:17.228363  4813 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.319331 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:17.320068  4815 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:17.332180  4815 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:17.332662  4815 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.333069  4815 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.334071  4734 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:17.310354  4749 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.334651  4734 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:17.335160  4749 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.335927  4734 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.225400  4771 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:17.339592 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 93535.3us
I20260709 12:19:17.328303  4722 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.342283 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.343446 32119 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:17.257045  4810 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:17.344944 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68242.2us
I20260709 12:19:17.347805 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.228720  4819 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:17.338027  4749 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.357060  4710 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:17.359789  4800 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:19:17.362869 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:17.372140 32654 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:17.377223 32119 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:17.354713  4755 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:17.426599 32657 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:17.357559  4737 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:17.427361 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 228888us
I20260709 12:19:17.357560  4819 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.353991  4734 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.441470 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:17.443746 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:17.445998 31970 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:17.443857 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:17.447696 31969 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.362870  4810 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.451406  4818 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:17.451670 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.452024  4818 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.453305  4810 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.365172  4579 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:19:17.458725 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:17.459728  4755 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:17.460711  4755 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.466531  4818 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.447613 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.434873  4816 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.483542 32042 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:17.365101  4722 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:19:17.495111 32115 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
W20260709 12:19:17.496865 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 287928us
I20260709 12:19:17.496903 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:17.500892 31970 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:17.501812 31969 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:17.445403  4819 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:17.498384  4775 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.003s
I20260709 12:19:17.433421  4729 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.440920  4460 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.515663 32119 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:17.498750  4821 raft_consensus.cc:2764] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:17.520767 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:17.538103 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:17.522167 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:17.538306 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:17.538864 32047 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:17.507006  4676 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:17.470530  4737 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.532840 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.538364 32040 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
W20260709 12:19:17.541592 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 94048.8us
I20260709 12:19:17.553889 32046 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:17.509694  4771 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:17.533156 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.560279 32115 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:17.553090 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:17.561565 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:17.559031 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:17.553267 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:17.498515  4818 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 12:19:17.562716 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 133731us
I20260709 12:19:17.495957  4810 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:19:17.561057 32041 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:17.565652 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.568437 32042 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:17.571642 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.560195  4676 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:17.575027  4676 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.576396 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.577780  4460 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:17.580389  4702 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.587661 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:17.594520 31969 raft_consensus.cc:606] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.563791  4688 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.601747 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.559904  4710 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:17.566206 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.568s (SENT) [suppressed 29 similar messages]
I20260709 12:19:17.603873  4821 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:19:17.596475  4771 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:17.558467  4685 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.594969  4742 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:17.636610 32038 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:17.608232 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.611342 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50702.8us
I20260709 12:19:17.627735 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.629560 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:17.605901  4777 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:17.599994  4460 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.595319  4830 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.664894 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:17.667948 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50821.3us
I20260709 12:19:17.678090 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.626791  4702 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:17.678983 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 112725us
I20260709 12:19:17.680202  4742 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:17.674248  4791 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.680640 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:17.686823 32623 leader_election.cc:336] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.810s (SENT)
I20260709 12:19:17.614358  4688 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.676256  4749 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.686023  4834 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.731467 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.673065  4654 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:17.713244  4709 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:17.707773 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68656.3us
I20260709 12:19:17.728237 32118 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:17.675558  4460 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.761601  4460 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.761871 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:17.762830  4654 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.764392 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:17.765457 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:17.711547  4830 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.772941  4830 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.773440  4830 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:17.699647  4710 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.775282 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:17.734218 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:17.775866 32113 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.776098 32039 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.759164 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:17.684247  4506 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:17.706066  4749 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.793980 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.685482  4742 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.707753 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:17.739205  4831 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:19:17.740095  4827 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:19:17.740417  4833 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:17.741384  4829 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:19:17.741950  4771 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.747470  4685 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:17.751129 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 154178us
I20260709 12:19:17.686630  4777 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:17.761418  4826 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:19:17.735265 32041 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:17.780438 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.791473 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:17.814985 32038 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.755441  4684 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.820773 32116 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:17.758332  4839 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:17.829501 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:17.776924  4709 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:17.814361  4749 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.773880  4830 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.821043  4742 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:17.821910  4729 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:17.846072 32652 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.853024 32010 leader_election.cc:400] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:17.855597 32039 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:17.832648  4825 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:17.861516  4836 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:19:17.861631  4688 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.760933  4834 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.908639  4834 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.913741  4709 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.879590 32041 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:17.873531  4729 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:19:17.920130  4654 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:17.920655  4654 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:17.921149  4654 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.922214 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:17.924238  4813 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.594s	user 0.001s	sys 0.009s
W20260709 12:19:17.926467 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 126619us
I20260709 12:19:17.926714 32117 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:17.929215  4838 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:17.930486 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.931219 32112 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:17.932248  4814 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:17.932665  4814 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:17.932858  4688 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:17.933543  4814 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.893277  4579 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:17.896314  4777 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.880734 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:17.930101  4775 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:17.882177 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:17.887351  4460 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:17.953531  4814 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.936740 32657 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:17.939172  4825 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:17.948776 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.981516 32038 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
W20260709 12:19:17.982738 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54151us
I20260709 12:19:17.983299 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:17.910127  4815 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.003s
I20260709 12:19:17.851974  4839 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:17.940634 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.935277  4737 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:17.879151  4676 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:17.934168  4688 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:17.940240 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:17.936847  4684 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:17.936187 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:17.959197 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78538.1us
I20260709 12:19:17.962965 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:18.021070 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 138961us
I20260709 12:19:17.974328 32113 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:17.924787  4813 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.025580 32120 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:18.026753 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.014812 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:17.976238  4838 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:17.975481  4579 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.027418 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.021981 32652 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:17.984495  4825 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.005416  4834 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.078054 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:18.076117 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 140033us
I20260709 12:19:18.101498 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:18.103222 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76553.2us
I20260709 12:19:18.105010 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:18.016194  4818 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:18.106455 31971 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:18.105693 31967 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:18.107791 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.024106  4684 consensus_queue.cc:1243] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:18.017565  4688 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:18.105284 32118 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:18.115353 32118 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:18.020576  4791 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:18.089393  4816 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.003s
I20260709 12:19:18.030295  4737 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:18.008922  4815 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.030061  4685 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:18.030512  4676 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:18.121007 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 106277us
I20260709 12:19:18.104816  4421 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:18.124673 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.108608  4654 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.114468  4834 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:18.138479  4818 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.122473  4710 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:18.141301 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63344.7us
I20260709 12:19:18.118353  4709 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.123319 32118 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.116288  4821 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:18.127863  4851 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 12:19:18.130601  4591 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:18.156699 31965 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:18.209872 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.127728  4688 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.076025  4722 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:19:18.113569  4838 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:18.116662  4734 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.113821  4729 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:18.221653 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 97065.8us
I20260709 12:19:18.231282 32118 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.211091  4826 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.217773  4421 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.226094  4737 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:18.245247 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.225555  4654 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:18.226758  4710 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.246763 32040 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.231633  4821 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:18.258903 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.259581 32042 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17.
I20260709 12:19:18.243528  4421 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.227962  4749 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:18.252627 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 136576us
I20260709 12:19:18.244987  4688 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:18.309002 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 60741us
I20260709 12:19:18.231040  4685 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.230813  4742 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.316648 32118 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:19:18.244308  4660 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.317668  4660 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.319849  4660 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.317281 32118 pending_rounds.cc:85] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
W20260709 12:19:18.320686 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 74939.5us
I20260709 12:19:18.320935 32118 pending_rounds.cc:107] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:19:18.229945  4709 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.321859 32118 raft_consensus.cc:2889] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:18.330632 32040 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:18.242825  4734 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:18.333994 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.253981  4710 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.342461  4830 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.328016  4676 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.336920  4722 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.248611  4826 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.257553  4654 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.351645 32040 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.370690 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.371677 32039 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:18.341188  4460 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:18.352569  4856 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:19:18.349472  4858 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.117s	user 0.005s	sys 0.005s
W20260709 12:19:18.357137  4849 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.116s	user 0.002s	sys 0.006s
I20260709 12:19:18.376921  4722 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.396018  4813 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.376439 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.376554 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.338614  4684 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:19:18.357860  4852 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:19:18.379582  4795 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 12:19:18.380539  4855 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:19:18.349854  4579 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.351887  4821 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.382746  4839 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.384196  4742 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.385040  4826 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.385488  4860 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:18.359534  4851 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:18.388684  4654 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:18.389125 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.403911 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.357287  4863 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.423699  4865 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.356283  4749 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.355732  4709 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.425673 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.377547  4862 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.467865  4862 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:18.432264  4819 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.911s	user 0.009s	sys 0.004s
W20260709 12:19:18.430009  4506 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.003s
I20260709 12:19:18.331821  4834 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.440487  4860 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.444128  4851 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.332473  4838 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.451661  4421 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.453244 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.453471  4865 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.453994 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.454700 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.455982 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.457520 32043 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:18.458052  4859 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:19:18.458518 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.458694  4863 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.459609  4857 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.460054 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.439459  4815 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.470292  4862 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.489830 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.490190 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.490409  4654 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:18.524662  4654 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.493767  4819 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.528782  4819 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.496070  4860 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.498926  4851 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.507336 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.531216  4654 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.507963  4865 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.508136 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:18.505093 31867 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 151) took 20984 ms (client timeout 15000 ms). Trace:
I20260709 12:19:18.533509 32047 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:19:18.533946 32047 pending_rounds.cc:85] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 10
I20260709 12:19:18.356604  4825 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.534283 32047 pending_rounds.cc:107] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:19:18.541577 32047 raft_consensus.cc:2889] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:18.511425  4421 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.514199  4857 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.512223  4818 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.519680  4853 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.544798  4853 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.513537  4710 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.547278 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.547457  4853 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.519977 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.548807  4775 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:18.494107  4506 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:18.552009 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:18.552793  4775 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:18.502979 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.555634 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.556383  4775 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:18.556783  4775 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:18.557142  4775 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.518728  4722 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.508539 32045 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:18.509289 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.564479 32043 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:18.564510  4775 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.567095 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:18.568060  4861 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:18.569900 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:18.570680  4861 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:18.511310 32040 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:18.577682 32623 leader_election.cc:336] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.260s (SENT)
I20260709 12:19:18.514883 32044 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17.
I20260709 12:19:18.578162 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:18.547931 32654 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:18.520447 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.583717 32042 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:18.585140 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:18.586015 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.589895 32119 pending_rounds.cc:85] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
I20260709 12:19:18.590353 32119 pending_rounds.cc:107] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:19:18.590665 32119 raft_consensus.cc:2889] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:18.599951 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.599725 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.601667 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.602497 32047 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 12:19:18.603312 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.603588 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.599841 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.606016 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:18.606199 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.606639 32121 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.606868 32113 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 10: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:18.609938 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.610607 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.611671 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.517400  4734 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.513638  4863 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.545506  4857 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:18.614138 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.514385  4685 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:18.617687 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47868.1us
I20260709 12:19:18.608873  4832 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:18.621851 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:18.645448 32116 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.536648  4825 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.605952  4853 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:18.606643  4861 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.617101  4830 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:18.633842 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:18.651787 32042 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:18.654510 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:18.613648  4795 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:18.658427 32117 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:18.641026 31865 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 9 to 12, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 12 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:18.642689  4775 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:18.762328 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 98439.8us
W20260709 12:19:18.532805 31867 rpcz_store.cc:269] 0709 12:18:57.520570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.521154 (+   584us) service_pool.cc:224] Handling call
0709 12:18:57.659429 (+138275us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1d7f3a 00007f2291f8de9c 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:19:18.505023 (+20845594us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:16.372533 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:16.373425 (+   892us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:16.373486 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:19:16.373512 (+    26us) tablet.cc:662] Decoding operations
0709 12:19:16.447454 (+ 73942us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:16.447549 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:16.447580 (+    31us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:19:16.448847 (+  1267us) tablet.cc:701] Row locks acquired
0709 12:19:16.448916 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:19:16.449109 (+   193us) write_op.cc:270] Start()
0709 12:19:16.449278 (+   169us) write_op.cc:276] Timestamp: P: 1783599556449048 usec, L: 0
0709 12:19:16.449327 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:19:16.570672 (+121345us) log.cc:844] Serialized 6247 byte log entry
0709 12:19:17.170451 (+599779us) op_driver.cc:464] REPLICATION: finished
0709 12:19:17.321439 (+150988us) write_op.cc:301] APPLY: starting
0709 12:19:17.321565 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:17.465750 (+144185us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:17.465885 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:17.924690 (+458805us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:17.925119 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:19:17.925181 (+    62us) write_op.cc:312] APPLY: finished
0709 12:19:17.925648 (+   467us) log.cc:844] Serialized 148 byte log entry
0709 12:19:17.925839 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:17.926030 (+   191us) write_op.cc:454] Released schema lock
0709 12:19:17.926104 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:17.952036 (+ 25932us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:17.952068 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1045688,"reported_tablets":24,"spinlock_wait_cycles":63514240,"child_traces":[["op",{"apply.queue_time_us":150618,"apply.run_cpu_time_us":27717,"apply.run_wall_time_us":629303,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70943,"num_ops":16,"prepare.queue_time_us":132588,"prepare.run_cpu_time_us":34576,"prepare.run_wall_time_us":427850,"raft.queue_time_us":173932,"raft.run_cpu_time_us":16957,"raft.run_wall_time_us":29941,"replication_time_us":720886,"thread_start_us":458708,"threads_started":4,"wal-append.queue_time_us":281352}]]}
I20260709 12:19:18.613857  4591 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:18.629701 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.768366 32047 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:18.656018  4863 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.657552  4734 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:18.660655 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.660768 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.769299  4863 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.769378  4734 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.771991  4863 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.772061  4734 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:18.661216  4710 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:18.662405 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.779773 32655 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:18.662643  4857 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.663136 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.661253 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:18.648057  4861 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.653990  4832 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:18.753489 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:18.766127 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.789825 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:18.790354 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.790951 32038 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.791090 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.672302  4722 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:18.791675 32040 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.762866  4830 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.760809  4865 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.794607 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.794754 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.796267 32047 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.796118 32044 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:18.801982 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.802635 32042 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:18.786219  4861 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.805155 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:18.838909 32043 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.835606 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.840029 32042 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:18.835610 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.840799 32040 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:18.841248 32045 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:18.783847  4771 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:19:18.851213 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:18.852289 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.785513  4734 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.864365 32047 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:18.866523 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:18.869230 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:18.870018 31969 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:18.870334 32046 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:18.873558 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:18.874673 31966 raft_consensus.cc:2410] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 12:19:18.878623 32038 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:18.880493 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:18.881531 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.882325 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.884042 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:18.868805 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:18.885871 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.886476 32045 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:18.858150  4579 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.789611  4857 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.889173 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:18.891158 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:18.891856 31964 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.893529 31971 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:18.894848 31965 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 12:19:18.834578  4866 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.803530  4830 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:18.902391 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.797797  4722 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.915235 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:18.916060 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.791992  4710 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.916687 32045 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:18.918049 31963 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:18.805749  4865 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:18.922221 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 85296.9us
I20260709 12:19:18.912001  4777 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:18.940969 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.906520  4866 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.883462  4734 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:18.946956 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:18.949002 31964 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:18.951376 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:18.936002  4729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.003s	sys 0.022s
I20260709 12:19:18.950209 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.952630 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:18.902691  4838 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:19:18.898936  4579 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:18.938225  4868 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.995769 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.017158 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:18.956099  4688 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.001s
I20260709 12:19:19.022472 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.021929 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:19.031847  4834 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:19.035508  4654 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:18.947974  4866 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:19.047986 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.991803  4729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.004s	sys 0.022s
I20260709 12:19:19.054828 32121 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:19.080500 32117 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.041251 32038 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:19.040139 32047 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.076372 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:19.078711 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.041326 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.069682 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.085597 32120 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:19.055429 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 196409us
I20260709 12:19:19.080083  4863 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.069036  4862 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:19.038247  4877 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:19.021190  4868 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.051033  4834 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.090977 32046 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.113921 32660 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:19.094921  4654 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.115092  4857 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.116235  4654 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.116654  4722 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.078698  4729 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:19.117352  4722 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:19.043246 32045 raft_consensus.cc:2410] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:19.119481 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:19.112056 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.124732 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:19.128470 32043 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:19.119247 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.147315 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.148023 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.106575  4862 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:19.159524  4579 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.160002  4869 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:19.160651  4579 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:19.108881  4868 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.106911  4877 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.163328 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.166873 32118 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.159463  4862 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.167964  4862 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:19.110579  4849 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:19:19.100428  4878 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:19.170506 32655 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:19.086210  4591 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.090828  4870 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:19.147624  4795 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.174891 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.118065  4866 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:19.182111 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 242881us
W20260709 12:19:19.183709 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 64548.4us
W20260709 12:19:19.099737  4867 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:19:19.182832 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:19.175390 32116 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.002s
I20260709 12:19:19.218918 32116 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:19.215306 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.147871  4835 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.112674  4838 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:19.099537  4863 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.235090 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:19.118177  4729 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.110852  4834 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.189783  4868 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.154592  4421 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 12:19:19.257139 31969 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:19.111539  4710 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.184322  4865 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.190101  4873 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.264268  4821 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:19.266775  4873 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.266976  4821 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.269176  4873 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.269212  4821 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.270248 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.234300  4654 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:19.234416  4853 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:19.271102 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.223922 32047 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:19.242872  4722 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.190404  4755 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:19:19.275795 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.259277  4854 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.279398 32121 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:19.191552  4591 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.282775 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:19.283504 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:19.285377 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.276758  4830 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.289980 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.286756  4791 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:19.286222 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:19.294988 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2403) took 1216 ms. Trace:
I20260709 12:19:19.276580  4877 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.282366  4868 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:19.297927 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:19.289101  4871 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:19:19.281961  4654 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:19.297684 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:19.258028  4870 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.304724  4888 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:19.305408  4888 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.306448  4888 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.281378  4722 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.297075 32041 rpcz_store.cc:276] 0709 12:19:18.076115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.078797 (+  2682us) service_pool.cc:224] Handling call
0709 12:19:18.110782 (+ 31985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.112826 (+  2044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.112874 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.120626 (+  7752us) log.cc:844] Serialized 26 byte log entry
0709 12:19:18.598020 (+477394us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:18.598057 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.598117 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:19.291965 (+693848us) raft_consensus.cc:1685] finished
0709 12:19:19.291997 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.292162 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8586,"raft.queue_time_us":7519,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":3444,"thread_start_us":358115,"threads_started":1,"wal-append.queue_time_us":653889}
I20260709 12:19:19.286573  4832 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:19.311311  4832 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.311697  4832 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.288553  4885 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:19:19.110486 31865 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 13, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 13 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:19.316021  4885 consensus_queue.cc:1243] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:19.303958  4791 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.316635 31865 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:19.318146  4791 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.293167  4854 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:19.321674  4854 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.324234  4825 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.324739  4825 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.295244  4834 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:19.326663 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:19.326799  4834 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.327903 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:19.299559  4532 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:19.330396 31969 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:19.332239 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:19.332789 31969 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.293869  4734 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:19.332993 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 256694us
I20260709 12:19:19.335776 32118 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:19.279919  4853 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:19.364643 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 211244us
I20260709 12:19:19.352387 32657 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:19.368974 31964 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:19.370471 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:19.370687 32039 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:19.371644 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:19.371963 31971 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:19.372233 32047 raft_consensus.cc:2376] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14. Re-sending same reply.
I20260709 12:19:19.393620 31966 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:19.394275 31963 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:19.396025 31969 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:19.368979 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.301306  4868 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.401998 32043 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:19.344691  4885 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:19.402662 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:19.404798 32039 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:19.402175 32045 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:19.405766 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.407384 31965 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:19.411675 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:19.411594 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:19.412700 31964 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:19.399354  4688 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:19.343415  4775 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:19:19.421367 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:19.421844 32045 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.396548  4821 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:19.410976  4838 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.412065  4877 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:19.417680 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.423833 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:19.433841 31965 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
W20260709 12:19:19.435043 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 171211us
I20260709 12:19:19.417528  4688 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:19.436852 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.427929 32039 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:19.458663  4676 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.460423 32038 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:19.439687 31965 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:19.420372  4654 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.462514 31965 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:19.463692 31965 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.420835  4895 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:19.465180  4895 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.466154  4834 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.446187 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.466744  4688 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.447224 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:19.467919 32652 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.468086  4688 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.459173  4676 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.469058  4591 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.318152 31865 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:19.470592  4676 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:19.471228 31865 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:19.472874  4832 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.472725 31865 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 13, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 13 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:19.435894  4866 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.455128 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:19.457608 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.477548 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.443890  4873 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.484634 32045 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:19.490945 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.491597 32119 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:19.493396 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48014.4us
W20260709 12:19:19.481631 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.499419 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.500026 32118 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:19:19.500102 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:19.499810 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.500916 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.505122 31967 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:19.484634 32652 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:19.485752  4889 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:19:19.485946  4863 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.487514  4868 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.523300 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.523967 32116 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:19.487771  4722 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.488363  4853 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.489398  4825 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.437018  4877 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:19.457976  4532 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.459157 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:19.526476  4861 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.006s
I20260709 12:19:19.466216  4895 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.521215 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.499548 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.521425 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.432797  4838 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.538806 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.539031 32045 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.543852 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:19.546521 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87428.4us
I20260709 12:19:19.577210 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.512461 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:19.549377  4863 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:19.558872  4832 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.568917 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:19.490084  4873 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.579672 32046 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:19.580245 32041 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:19.580791 32118 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:19.455927  4734 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:19:19.586989 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.597172 32658 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:19.578877  4825 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:19:19.611610  4825 consensus_queue.cc:1243] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:19.593899 32041 raft_consensus.cc:1217] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-13.9]   Dedup: 10.7->[12.8-13.9]
I20260709 12:19:19.603232 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:19.612813 32041 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [10.7-13.9]
I20260709 12:19:19.607244 31970 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:19.614017 32084 connection.cc:256] RPC call timeout handler was delayed by -1.2465428350001275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:19:19.540845  4771 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:19:19.587858  4863 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:19.600342  4591 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:19.614697 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1898428350000358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 12:19:19.596839  4873 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.615439 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:19.623315 32084 connection.cc:256] RPC call timeout handler was delayed by -1.1574428349999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260709 12:19:19.597931  4900 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:19.634054  4887 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.364s	user 0.000s	sys 0.004s
I20260709 12:19:19.589262  4894 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:19:19.610344  4835 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:19.584831  4861 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.626111  4885 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.623199  4591 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:19.656268 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.615959  4729 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.622758  4863 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.493930 31865 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:19.648514 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:19.664819 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77912.1us
I20260709 12:19:19.680775 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.681823 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.666570  4734 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:19.648844  4918 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.610817  4854 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:19:19.637408  4771 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:19.636302  4900 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:19.659276  4893 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.658080  4825 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:19:19.683826 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44811.5us
I20260709 12:19:19.685941 32655 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 10.
I20260709 12:19:19.686141 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.686302 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:19.687564 32113 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:19.697301 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81942.6us
I20260709 12:19:19.676571  4591 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.713605 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:19.685241  4729 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.727147 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:19.668227  4755 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:19.729911 32658 raft_consensus.cc:1217] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[10.9-14.11]   Dedup: 10.9->[12.10-14.11]
I20260709 12:19:19.730646 32658 pending_rounds.cc:85] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 9
I20260709 12:19:19.730948 32658 pending_rounds.cc:107] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:19:19.731293 32658 raft_consensus.cc:2889] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:19.700126  4832 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.696502  4771 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:19.698890  4853 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:19.679576  4722 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.710634  4795 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.663879  4862 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:19:19.686849  4734 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:19.723783 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.723696 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:19.698827  4918 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:19.740643 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 92176.1us
I20260709 12:19:19.702067  4865 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:19.740422 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.706066  4900 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.662375  4877 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.739228 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.718559  4825 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.746542 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.751363 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 608) took 1715 ms. Trace:
I20260709 12:19:19.734218  4791 raft_consensus.cc:2764] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:19.736418  4710 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:19.757987 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:19.674738  4532 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.737150  4920 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:19.764972 32652 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.747617  4722 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.736335  4832 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:19.782606 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59040.8us
I20260709 12:19:19.727094  4893 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.767112 31970 rpcz_store.cc:276] 0709 12:19:18.028523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.029246 (+   723us) service_pool.cc:224] Handling call
0709 12:19:18.106778 (+ 77532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.613375 (+1506597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.744070 (+130695us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001e96d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:19.744239 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3442106112}
I20260709 12:19:19.841457 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.820804 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:19.819947  4830 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:19.777119  4885 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.765853  4907 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.854384 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:19.754739  4921 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:19.776369  4825 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.861670  4921 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.862275  4921 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:19.863190  4777 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.863641  4777 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:19.864006  4777 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.864176 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:19.865411  4908 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:19.756498  4870 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.866827  4870 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:19.867200  4870 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:19.867358  4906 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:19.867640  4870 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:19.871423  4777 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.873579  4777 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:19.781759  4868 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:19.874457  4868 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.874965  4868 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:19.879390 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:19.880262  4777 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.880347  4921 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.881076  4777 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.882737  4921 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:19.883117  4825 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:19.857213 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 118055us
I20260709 12:19:19.885535  4777 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.773286  4838 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.848271  4722 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.841244  4710 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:19.892656  4910 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:19:19.787106  4795 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.787884  4923 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.931697  4923 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.932096  4923 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.884609  4908 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:19:19.891916  4916 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:19:19.900063 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:19.856508  4927 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.857697  4901 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:19:19.763628  4865 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:19.896114  4899 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:19.895964  4928 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.926973  4795 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:19.924674  4896 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.438s	user 0.002s	sys 0.003s
W20260709 12:19:19.942946 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48174us
W20260709 12:19:19.944294 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 191805us
I20260709 12:19:19.860164  4830 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.940689  4921 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.852537  4734 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:19.993068 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:19.846426  4771 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.989979 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:20.001612 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.945902  4928 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:19.955792  4861 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:20.023981 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 174655us
I20260709 12:19:19.806143 31865 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 14, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 14 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:20.009593  4771 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:20.025141 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.024340 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:20.010849  4734 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:20.009235  4928 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.998796  4870 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.039682  4935 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:19:20.040352  4930 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:20.041025  4935 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:20.040979  4930 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.041617  4933 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:19:20.042038  4931 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:19.924964  4918 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.042500  4832 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.043190  4934 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:19:20.043669  4893 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.044711  4870 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.044835  4830 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:19.943660  4927 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:19.998606  4911 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:19:20.017155  4777 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:20.046367 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:20.041569  4935 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:19.998919  4901 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:19:20.030494  4729 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:20.057597 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 311122us
I20260709 12:19:20.048115 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:20.060930 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:20.062920 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:20.003577  4710 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.058794  4911 consensus_queue.cc:1243] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:20.087752 32121 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:20.066051  4920 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:20.109232 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.059244  4931 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:20.085868  4777 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.132619  4944 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.133055 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.134642  4944 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:20.135644  4944 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:20.137220 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.136852  4944 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.112295  4930 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.138867  4825 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.142056  4920 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.153026 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.153730 32047 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:20.155046 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:20.160979 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.159394 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.165518 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.166190 32042 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
W20260709 12:19:20.167769 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.163106 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.167323 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.170177 32042 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:20.172781 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.159168 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.176611 31966 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:19:20.177242 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.176533 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.178040 32039 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:20.192500 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:20.193763 32042 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:20.195348 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:20.195744 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.196595 31969 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:20.152691  4877 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:20.198391 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.198725 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.198846 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.203042 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.203866 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.204131 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.227700 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.155810  4870 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.204979 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.233688 32045 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:20.234665 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:20.238727 31966 raft_consensus.cc:2376] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:19:20.192957  4795 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.241204 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.198294  4865 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.245240 31963 raft_consensus.cc:2368] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:20.245325 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.255950 32045 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:20.257211 32042 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:20.265628 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:20.252048  4688 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.784s	user 0.007s	sys 0.007s
I20260709 12:19:20.266607 31966 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:20.115188 31865 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 14, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 14 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:20.267561 31963 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:20.271975 32044 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:20.271405 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.283061 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.285959 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.279147 32041 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:20.286796 31963 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.285562 31971 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:20.240918  4870 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:20.294628  4890 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.766s	user 0.004s	sys 0.008s
I20260709 12:19:20.286803 31966 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:20.144289  4935 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.307804 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.308326 32045 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:19:20.265831  4950 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.276293  4877 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:20.239537  4777 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.319003 32044 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: [13.12-16.13]
I20260709 12:19:20.319132 31967 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:20.282815  4939 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.325284 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.326229  4950 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.326699 31865 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 3 to 11, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 11 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:20.305728  4927 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.330050 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.328171 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:20.319389  4948 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:19:20.331163 32045 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:20.226959  4931 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.316824  4684 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:20.357215 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.358212 31968 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:20.353385  4940 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 12:19:20.398586 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 405546us
I20260709 12:19:20.329130  4579 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:19:20.399349 31963 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
W20260709 12:19:20.433393 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103370us
I20260709 12:19:20.404804 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:20.406119 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 617) took 1535 ms. Trace:
I20260709 12:19:20.436064 31968 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:20.435381 31972 rpcz_store.cc:276] 0709 12:19:18.871170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.871361 (+   191us) service_pool.cc:224] Handling call
0709 12:19:18.871760 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.887021 (+ 15261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.887081 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.911173 (+ 24092us) log.cc:844] Serialized 26 byte log entry
0709 12:19:19.331868 (+420695us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:19.331905 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.331970 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:20.064865 (+732895us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:20.403659 (+338794us) raft_consensus.cc:1685] finished
0709 12:19:20.403692 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.406039 (+  2347us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10479,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":13497,"spinlock_wait_cycles":25096448,"thread_start_us":385376,"threads_started":1,"wal-append.queue_time_us":548779}
I20260709 12:19:20.442502 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.332186  4939 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.449476 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:20.451488 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46763.3us
I20260709 12:19:20.454748 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.456986 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:20.457670 32652 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:20.405620  4888 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:19:20.470968  4868 consensus_meta.cc:287] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.595s	user 0.000s	sys 0.011s
I20260709 12:19:20.471478  4868 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.458693 32039 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:20.452441  4676 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:20.461241  4942 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:19:20.452621  4579 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:20.474740  4888 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:19:20.455916  4946 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:19:20.475178  4579 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.475759  4579 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.473469  4775 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:19:20.476919  4775 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:20.477290  4775 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:20.462934  4591 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.001s
I20260709 12:19:20.478149  4775 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.478673  4591 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.479383  4775 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.479882 32043 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:20.464556  4873 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.816s	user 0.009s	sys 0.000s
I20260709 12:19:20.481732  4873 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.484186  4873 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:20.465121  4953 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 12:19:20.466992  4863 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.003s
I20260709 12:19:20.485437  4863 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.486498 31965 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:20.468274 32657 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:20.475196  4888 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.447842  4877 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.488309  4888 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.488804  4591 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.489446  4888 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:20.469000  4923 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.004s
I20260709 12:19:20.440653 31865 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 9 to 10, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:20.473922  4676 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.500975 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:20.504558  4863 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.506676 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34592 (request call id 776) took 1893 ms. Trace:
I20260709 12:19:20.507452 31865 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 6 to 8, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 8 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:20.509565 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.509738  4885 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:20.510003  4676 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:20.505604  4857 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:19:20.520027 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:20.520442 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.520759 32653 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:20.491376  4722 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.564s	user 0.003s	sys 0.007s
I20260709 12:19:20.507121 32113 rpcz_store.cc:276] 0709 12:19:18.613773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.613903 (+   130us) service_pool.cc:224] Handling call
0709 12:19:20.506481 (+1892578us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:20.506616 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441363200}
I20260709 12:19:20.520381 32658 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:20.523430 32121 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:19:20.524650 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.536931 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:20.537439 32010 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.811s (SENT)
I20260709 12:19:20.458954  4950 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.546540 32652 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:20.548904 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:20.553112 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:20.554453 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60623.6us
I20260709 12:19:20.554242 32044 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:20.501559  4777 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:19:20.541822  4885 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.549552  4870 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.566356 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:20.557533  4900 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.001s
I20260709 12:19:20.570120 31963 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:20.500034  4923 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.509591  4579 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.565037 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:20.577708 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.577876 32653 raft_consensus.cc:2410] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:20.565253  4777 consensus_queue.cc:1243] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:20.579579  4777 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:20.580068  4777 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.549690  4722 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.582381  4926 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 12:19:20.582818 32043 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 12:19:20.583179 32043 pending_rounds.cc:85] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 7
I20260709 12:19:20.583395 32043 pending_rounds.cc:107] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:19:20.583636 32043 raft_consensus.cc:2889] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:20.584023  4722 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.584760 31964 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: [14.15-14.15]
I20260709 12:19:20.584903 32043 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:20.576308  4933 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:19:20.587002 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:20.587193 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:20.588369 32038 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:20.570541  4791 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:19:20.589604 32038 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:20.509780 31865 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 14, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 14 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:20.591758 31865 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 12, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:20.559576  4857 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:19:20.582175  4777 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:20.578289 31963 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:20.598912 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.599606 31963 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.599875 32043 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:20.593662  4676 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.607070  4885 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.609251  4676 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.609462 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:20.610121 32659 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:20.615828 31963 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
W20260709 12:19:20.618191 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.621523 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:20.624631 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:20.593436 31865 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 14, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 14 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:20.622901 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:20.635241 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.636418 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.637008 31963 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:20.637753 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.638823 31966 raft_consensus.cc:2368] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 17.
I20260709 12:19:20.639928 31963 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:20.644048  4906 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:20.644614  4906 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.648409 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.653334 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.657394 32118 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:20.659113 31963 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:20.666464  4924 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:19:20.679605  4579 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.619764  4722 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:20.681559  4722 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.683782  4722 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.684767 31865 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 14, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 14 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:20.686789 31865 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:19:20.666095  4965 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:19:20.689289  4971 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 12:19:20.693799  4734 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.003s
I20260709 12:19:20.694756 31972 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:20.695665 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.696286 31969 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:20.701236  4923 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.701632 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2238) took 1324 ms. Trace:
I20260709 12:19:20.611243  4911 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:19:20.575781  4900 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.704741  4900 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.706238 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.708493 31969 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:20.707343 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.702137 31965 rpcz_store.cc:276] 0709 12:19:19.377589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.464358 (+ 86769us) service_pool.cc:224] Handling call
0709 12:19:19.464615 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.486982 (+1022367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.701330 (+214348us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400111054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:20.701572 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2349611776}
I20260709 12:19:20.721019 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.721925 31965 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:20.723361 32659 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:20.725003 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.725225 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.725756 31964 raft_consensus.cc:2376] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:19:20.725970 31969 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:20.729156 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.641541  4868 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:20.734925 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.737884 31963 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:20.732610 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.738970 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.736634 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.688546 31865 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 9 to 17, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:20.741034 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 176601us
I20260709 12:19:20.744752 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.747984 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:20.645695  4906 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.747671 31965 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:20.749091 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.727744  4911 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:20.751899 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.752874 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:20.783243 31963 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:20.753132 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.754348 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.784462 31969 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:20.751188  4923 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:20.785218 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.787348 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:20.787551 31963 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:20.785672 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.751524  4918 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:20.791603 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.789307 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:20.791096 31963 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:20.794569 31963 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:20.795647 31963 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:19:20.797616  4969 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:20.792675 31964 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
W20260709 12:19:20.802727  4970 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:19:20.808573  4532 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:20.809017  4532 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.792001  4918 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.811422  4918 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.790962  4885 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.814599 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.815922  4795 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:20.816521  4795 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.809865  4532 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:20.819123 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:20.819288  4532 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.819662  4795 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:20.819566 31865 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 14, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 14 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:20.821429 31865 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:19:20.822564  4928 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.004s
I20260709 12:19:20.822918  4885 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:20.799861  4868 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:20.825052  4885 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:20.825628  4868 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.826113  4868 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.827893 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:20.828593 32652 raft_consensus.cc:2368] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:20.818624 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:20.829766 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:20.830842 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:20.832015  4921 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:19:20.832676  4947 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.591s	user 0.000s	sys 0.002s
I20260709 12:19:20.834658  4933 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:20.834932  4924 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 12:19:20.835284  4933 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.835783 31969 raft_consensus.cc:606] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.836308  4933 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:20.837062  4968 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.334s	user 0.003s	sys 0.000s
I20260709 12:19:20.838482 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:20.839519 31969 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:20.791575 31965 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:20.850661 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46126.1us
I20260709 12:19:20.851615 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.849640 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:20.859874 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.826723  4972 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:20.853324  4921 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:20.871668 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.872112  4921 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.872892  4921 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.873090 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.874053  4921 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:20.874466  4921 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.874830  4921 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.823005  4928 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:20.879284 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.930874  4900 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:20.931516  4900 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.931950  4900 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.863096  4734 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.939003 31971 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:20.858201  4908 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.863631  4972 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:20.791890  4911 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:20.923540 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44768.8us
I20260709 12:19:20.947753 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.948706 31965 pending_rounds.cc:85] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 9
I20260709 12:19:20.949193 31965 pending_rounds.cc:107] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:19:20.862063  4857 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:19:20.949544 31965 raft_consensus.cc:2889] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:20.950465 32655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 990) took 1679 ms (client timeout 2075 ms). Trace:
I20260709 12:19:20.951915 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:20.923576  4791 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:19:20.955889 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:20.951017 32655 rpcz_store.cc:269] 0709 12:19:19.270611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.270776 (+   165us) service_pool.cc:224] Handling call
0709 12:19:20.950273 (+1679497us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:20.950395 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945747328}
I20260709 12:19:20.928957  4861 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:19:20.947109  4777 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:19:20.958138  4857 consensus_queue.cc:1243] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:20.962383  4710 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.966450  4923 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 12:19:21.013387  4891 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.527s	user 0.001s	sys 0.004s
I20260709 12:19:21.018014 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.019441 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.019687 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.020032 31972 raft_consensus.cc:2376] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14. Re-sending same reply.
I20260709 12:19:21.020819 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.020977 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.021770 31970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:21.022945  4775 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:21.023325  4775 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:21.024196  4775 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.024827 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.025074 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:21.025401  4775 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.948467  4972 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.026927  4972 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:21.029594  4974 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:19:21.030658 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.031010 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.031746 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.031951  4930 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.032198 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.032410  4930 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.032747 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.033255 32120 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.033352 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.034773 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.035374 32115 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:21.035625 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.036090  4952 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:21.036391 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.036674  4952 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.037127 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.036120 32121 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.039824 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.041070  4926 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:21.041961  4926 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:21.042358  4926 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:21.042507  4883 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:21.042881  4926 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:20.862970 31865 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:21.043599  4771 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:21.045120 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:21.022298 31970 raft_consensus.cc:606] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:20.964939  4919 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:21.047596 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.017875  4918 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.062090 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:21.047344 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 122530us
I20260709 12:19:21.068995 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:21.071521 32040 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:21.046077 32011 leader_election.cc:400] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:21.072094 32011 leader_election.cc:403] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:21.072218 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:21.054272 32114 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:21.056603 31970 raft_consensus.cc:614] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:21.049533 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.067281 32121 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:21.078372 32043 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:21.051049  4888 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.003s
I20260709 12:19:21.079811 32043 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:21.080952 32043 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:19:21.081561 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.082394 32043 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:21.048022  4591 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.100853 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.117686 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.068135  4734 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:21.055846  4771 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:21.127305  4866 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:21.127511  4734 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:21.128492  4734 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.129194  4866 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:21.059785  4927 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:21.130332 32113 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:21.130522  4927 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:21.131517  4927 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.132477  4927 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:21.087378  4934 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 12:19:21.078173 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.134934  4737 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.137725 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:21.138906 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.139160 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:21.142443 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.150583 31969 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:19:21.154383 32120 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:21.116880  4722 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.117036  4870 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:21.179391  4870 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:21.179852  4870 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:21.053133  4921 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.117128  4834 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:21.182498  4654 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:21.182660  4834 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.184777  4834 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:21.172462 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.100033  4930 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:21.135980  4591 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.128001  4771 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.15, Last appended by leader: 15, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:21.192800 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 103648us
W20260709 12:19:21.167819  4973 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:19:21.224547 31971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6679c928e92e4f799b88357707650b60"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
W20260709 12:19:21.232865 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.168164  4934 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:21.099331  4710 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:19:21.117640  4869 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
W20260709 12:19:21.282434 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.284332 31971 raft_consensus.cc:606] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.312335 31971 raft_consensus.cc:614] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:21.283555 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 115230us
I20260709 12:19:21.286275 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2433) took 1114 ms. Trace:
W20260709 12:19:21.286917 32010 leader_election.cc:336] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 2.125s (SENT)
I20260709 12:19:21.314412 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.315026 32039 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.315249 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:21.313032 32047 rpcz_store.cc:276] 0709 12:19:20.172031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.174604 (+  2573us) service_pool.cc:224] Handling call
0709 12:19:20.174886 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.175135 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:20.229144 (+ 54009us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.953361 (+724217us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b18001d5900. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:19:21.285902 (+332541us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.285954 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:21.286003 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.286066 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.286219 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142303,"prepare.queue_time_us":54063,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"spinlock_wait_cycles":10041728,"thread_start_us":189928,"threads_started":1}
I20260709 12:19:21.283604 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:21.321488 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:21.324393 32010 leader_election.cc:400] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:21.333758 32010 leader_election.cc:403] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:19:21.335132  4977 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:19:21.337208 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53665.4us
I20260709 12:19:21.182518  4921 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.305776  4894 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:21.333262 32039 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:21.326117 32043 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:19:21.309583  4830 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:21.339709  4929 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:21.283977  4900 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.339838  4771 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:21.343319 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 297359us
I20260709 12:19:21.168987  4737 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.338241 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:21.351809 31971 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:21.396261 31971 pending_rounds.cc:85] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 9
I20260709 12:19:21.396559 31971 pending_rounds.cc:107] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:21.396862 31971 raft_consensus.cc:2889] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:21.284416  4928 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:21.337745  4934 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:21.284197  4919 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:21.394348 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:21.383790 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68653.4us
I20260709 12:19:21.343466  4952 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:21.352949  4532 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.001s
I20260709 12:19:21.307371  4972 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:19:21.349591  4930 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:21.456555  4939 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.385227  4830 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.443995 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:21.484503 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.484627 32047 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.446210  4919 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.446718  4963 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:21.563087 32118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.040s
W20260709 12:19:21.565661 32118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.008s	sys 0.040s
I20260709 12:19:21.453548 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:21.457273  4737 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:21.452775  4982 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.282s	user 0.000s	sys 0.000s
I20260709 12:19:21.396631  4961 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:21.479946 32652 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:21.396476  4861 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:21.480912 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 156656us
I20260709 12:19:21.481065 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:21.493660 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51027us
W20260709 12:19:21.599205 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.598016 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:21.597975 32654 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:21.497414 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:21.602253 31968 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:21.497820  4983 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:21.496552  4972 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:19:21.498714  4980 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:19:21.499539 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.499325  4934 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.500011 31971 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:21.649298 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:21.500622 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.510471  4950 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:21.515237  4985 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:19:21.515735  4981 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:19:21.509722 32047 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:21.659103 32046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.009s	sys 0.027s
I20260709 12:19:21.425133  4929 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:21.522914 31865 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 13, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 13 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:21.529606  4775 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.001s
I20260709 12:19:21.350611  4894 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.439033  4966 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:21.440236  4676 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:19:21.545437  4722 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:21.459407  4857 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:19:21.339152  4868 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.459686  4931 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.547601  4906 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.446183  4935 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.472213  4952 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.554759 31969 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:21.710834 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.003s	sys 0.016s
W20260709 12:19:21.711311 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.003s	sys 0.016s
I20260709 12:19:21.555011  4900 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.567831  4963 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.570350  4907 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:21.724331 31970 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:21.622602 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:21.726150 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 103651us
W20260709 12:19:21.580824  4908 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.032s	sys 0.043s
I20260709 12:19:21.632345  4951 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:21.632647 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.633291  4737 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:21.633980  4861 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:21.635520 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2251) took 1075 ms. Trace:
I20260709 12:19:21.704850  4654 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14.
I20260709 12:19:21.639634  4911 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:21.644140  4919 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:21.646860  4984 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:19:21.731271  4908 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.032s	sys 0.044s
I20260709 12:19:21.647557  4888 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:21.651247 31971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71e0cf115549454c894335e70c43843a"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:21.723636  4933 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:21.658206 31972 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:21.659418 32046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.009s	sys 0.027s
I20260709 12:19:21.776844 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:21.777537 32653 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:21.661677 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.779103 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.623986  4918 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.665205  4532 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:21.666961 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 643) took 1355 ms. Trace:
I20260709 12:19:21.668588 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.784143 32659 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:21.784431 32047 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:21.672968 31968 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:21.788029  4963 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.641855  4870 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.788960  4984 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:21.789490  4984 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:21.628990  4901 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:21.706506 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2252) took 1142 ms. Trace:
I20260709 12:19:21.661703  4921 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.666625  4710 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.767335  4929 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.720630  4952 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.764667  4966 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:21.626201  4961 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:21.712507 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 374363us
I20260709 12:19:21.721314 31969 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:21.729224 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.639134  4591 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:21.732142 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99598.6us
I20260709 12:19:21.731535 31965 rpcz_store.cc:276] 0709 12:19:20.560096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.853755 (+293659us) service_pool.cc:224] Handling call
0709 12:19:20.948521 (+ 94766us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:20.989736 (+ 41215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.989784 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:20.990530 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 12:19:21.345555 (+355025us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:21.434047 (+ 88492us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.439913 (+  5866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.440141 (+   228us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.634255 (+194114us) raft_consensus.cc:1685] finished
0709 12:19:21.634290 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.635283 (+   993us) spinlock_profiling.cc:243] Waited 94.3 ms on lock 0x7b5400314f50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:21.635433 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10259,"prepare.queue_time_us":85080,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":26808,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":24308,"spinlock_wait_cycles":216934656,"thread_start_us":334152,"threads_started":1,"wal-append.queue_time_us":575248}
W20260709 12:19:21.763751 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 135418us
I20260709 12:19:21.784180 32041 rpcz_store.cc:276] 0709 12:19:20.311175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.311308 (+   133us) service_pool.cc:224] Handling call
0709 12:19:20.311610 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.038616 (+727006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.038650 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.039132 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:19:21.395914 (+356782us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:21.395954 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.396021 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.666692 (+270671us) raft_consensus.cc:1685] finished
0709 12:19:21.666718 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.666891 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":726643,"raft.queue_time_us":728927,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":356381,"threads_started":1,"wal-append.queue_time_us":610954}
I20260709 12:19:21.837674 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:21.647280  4830 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:21.842770 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.773047 31971 raft_consensus.cc:606] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:21.846863 31971 raft_consensus.cc:614] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:21.848768 31971 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:19:21.652274 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.023s	sys 0.007s
W20260709 12:19:21.849658 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.023s	sys 0.007s
W20260709 12:19:21.850066 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 780877us
I20260709 12:19:21.732081  4857 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:21.775050 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:21.711599 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 574443us
W20260709 12:19:21.768036  4927 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.004s
I20260709 12:19:21.788375 31968 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [12.7-16.9]
I20260709 12:19:21.790732 31967 rpcz_store.cc:276] 0709 12:19:20.564309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.956248 (+391939us) service_pool.cc:224] Handling call
0709 12:19:20.956623 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.956857 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:21.348278 (+391421us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.705725 (+357447us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.705863 (+   138us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:21.705914 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.706023 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.706439 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31036,"prepare.queue_time_us":56908,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":1033,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4156,"thread_start_us":715398,"threads_started":2}
I20260709 12:19:21.790901 31966 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:21.814576 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.814982 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:21.820644 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:21.835479 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:21.776376  4933 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:21.790256  4870 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:21.790669  4984 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.822789  4966 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:21.825790 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 96652.2us
I20260709 12:19:21.853533 32115 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:19:21.839952 31969 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:19:21.798856  4998 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:19:21.821668 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.798004  4885 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:21.836217 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:21.791209  4710 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.794575  4997 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:21.790602  4901 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:21.790977  4921 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:21.857200 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.878393 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:21.981225 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.982717 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:21.802543  4908 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:21.827221 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:21.867336 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:21.907370 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66524.1us
I20260709 12:19:21.913542 31969 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:21.994750 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:21.906531 32658 raft_consensus.cc:2368] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:19:21.908651 32115 pending_rounds.cc:85] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 7
I20260709 12:19:21.996079 32115 pending_rounds.cc:107] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:19:21.996515 32115 raft_consensus.cc:2889] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:21.908218 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.944276 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.998474 32045 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:21.809197  4952 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.827419 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:21.947039 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 85901.2us
W20260709 12:19:21.767379 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 143347us
I20260709 12:19:21.904663  4966 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:21.942790  5004 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:19:21.963967 32040 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:21.862713 31967 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:21.932387 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:21.987751 32043 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:22.028590 32041 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:21.988876 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 160521us
I20260709 12:19:21.830367  4918 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:21.894611  4729 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.139s
I20260709 12:19:22.031479  4952 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:19:22.034929 32043 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:22.035207  4729 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:22.035780  4729 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:22.035977 32112 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 12:19:22.036410 32112 pending_rounds.cc:85] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 7
I20260709 12:19:22.036643 32112 pending_rounds.cc:107] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:19:22.036911 32112 raft_consensus.cc:2889] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.037225 32112 pending_rounds.cc:107] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:19:22.037496 32112 raft_consensus.cc:2889] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.037633  4911 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:22.040288  5000 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:22.042030  5007 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:22.042389  5006 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:19:22.042629  5007 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.901978  4961 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:22.044811  5007 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.048002 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2253) took 1479 ms. Trace:
I20260709 12:19:22.049256 32116 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:22.007606  4908 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:22.051127  4908 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.052482  4908 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:21.875699  4967 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.373s	user 0.004s	sys 0.000s
I20260709 12:19:21.906672  4993 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.198s
I20260709 12:19:21.877884  4777 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 12:19:21.978782  4955 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:22.057950 32040 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:21.855809  4857 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:21.826656 32047 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:22.005088 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:21.865940 31965 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:21.841837  4900 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:19:21.920073  4885 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:22.067497 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 58293.6us
I20260709 12:19:21.904368  4984 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.899780  4868 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:21.870247 31968 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:22.026908  4821 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:21.897940  4926 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.829s	user 0.008s	sys 0.000s
W20260709 12:19:22.024977 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 146668us
W20260709 12:19:22.027829 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 119711us
I20260709 12:19:21.947656  4997 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:22.023087  5011 raft_consensus.cc:2764] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:22.050205 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2267) took 1508 ms. Trace:
I20260709 12:19:22.087984  4952 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:21.874526 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:22.093751 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 219315us
I20260709 12:19:22.100790 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.064121 32042 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:22.102737 32041 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:22.103931 32047 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:22.104216 32041 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:21.915930 31971 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:22.053984 32039 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:19:22.032039  4998 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:21.941288  4591 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:19:21.896052  4676 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:19:22.081524 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 76561.9us
I20260709 12:19:22.152842 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:22.086163 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:22.154431 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68391.4us
I20260709 12:19:22.155857 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:22.005098  4927 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:19:22.048534 31970 rpcz_store.cc:276] 0709 12:19:20.568768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.115604 (+546836us) service_pool.cc:224] Handling call
0709 12:19:21.115971 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.724841 (+608870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.047657 (+322816us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54000b6654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:22.047921 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398501120}
I20260709 12:19:22.088682 32045 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:22.103161 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2280) took 1026 ms. Trace:
I20260709 12:19:22.106421 32041 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:22.036659  4729 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:21.902881  4933 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.084241  4888 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:22.144966 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 284497us
I20260709 12:19:22.088157  4997 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:22.158892 32045 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:22.246541 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:22.091740 32046 rpcz_store.cc:276] 0709 12:19:20.542160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.545235 (+  3075us) service_pool.cc:224] Handling call
0709 12:19:20.545531 (+   296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:21.048190 (+502659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.048220 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:21.050875 (+  2655us) log.cc:844] Serialized 52 byte log entry
0709 12:19:21.776850 (+725975us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:21.776910 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.777014 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.049926 (+272912us) raft_consensus.cc:1685] finished
0709 12:19:22.049953 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.050131 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":573941,"raft.queue_time_us":515628,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":19067,"thread_start_us":649500,"threads_started":1,"wal-append.queue_time_us":752832}
I20260709 12:19:22.158470 32044 rpcz_store.cc:276] 0709 12:19:21.077044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.077190 (+   146us) service_pool.cc:224] Handling call
0709 12:19:21.130678 (+ 53488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.130868 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:21.680112 (+549244us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.102831 (+422719us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.102885 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:22.102942 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.103007 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.103111 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":610098,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":88,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":26746,"thread_start_us":817930,"threads_started":2}
W20260709 12:19:22.212838 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 134386us
W20260709 12:19:22.212908 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 122071us
I20260709 12:19:22.109033  4952 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:22.208053 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50924.4us
I20260709 12:19:21.882395 31865 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.088232  5014 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.136s
I20260709 12:19:22.142771  4771 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:22.230583 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 77870.5us
I20260709 12:19:22.248224 32115 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:19:22.249099 32121 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:19:22.262892 32115 pending_rounds.cc:85] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 7
I20260709 12:19:22.207600  4919 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:22.249864 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.267066 32120 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:22.091181  4926 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:22.231659 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.253227 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.269456 32118 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:22.238633 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2515) took 1210 ms. Trace:
I20260709 12:19:22.258749 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:22.235030 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 633) took 1421 ms. Trace:
I20260709 12:19:22.206949  4933 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.272784 31966 rpcz_store.cc:276] 0709 12:19:20.813436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.457159 (+643723us) service_pool.cc:224] Handling call
0709 12:19:21.457614 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.870492 (+412878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.234695 (+364203us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400002154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:22.234943 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765898496}
I20260709 12:19:22.222940  5009 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:22.077759  4955 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.161432  4894 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.151932  4676 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:22.263248 32115 pending_rounds.cc:107] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:19:22.263151 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:22.263041 32121 pending_rounds.cc:85] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
I20260709 12:19:22.307171 32121 pending_rounds.cc:107] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:19:22.307583 32121 raft_consensus.cc:2889] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.228956 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:22.269929 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:22.270401 32116 rpcz_store.cc:276] 0709 12:19:21.027792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.036838 (+  9046us) service_pool.cc:224] Handling call
0709 12:19:22.234275 (+1197437us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400147dd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:22.238555 (+  4280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2325660416}
W20260709 12:19:22.276870 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.276486 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:22.589800 32658 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
W20260709 12:19:22.216203  4998 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:22.246563  4868 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:22.580571  5016 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.434s	user 0.000s	sys 0.001s
I20260709 12:19:22.254958  4956 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.262526  4791 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.261525  4952 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.370703 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:22.311875 32115 raft_consensus.cc:2889] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.161201  4885 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.493657  4901 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:22.145253  4857 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.276470  4684 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 12:19:22.244925  4907 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:19:22.302389  4900 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:19:22.432590  5025 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:22.310037  5009 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.390654  4834 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:22.577651 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 314543us
I20260709 12:19:22.425755  4921 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.265261  5007 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.105s
W20260709 12:19:22.328317 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.428552  4935 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.001s
I20260709 12:19:22.262326  4722 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:19:22.258025  4888 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.371829  4894 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:22.299088  5013 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:22.277639  4771 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:22.409145  4862 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:22.492694  4955 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:22.494807  4926 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:22.580210 31969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.012s	sys 0.001s
I20260709 12:19:22.585589 31964 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:19:22.529413  4927 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:22.370431 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:22.708158 31969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.013s	sys 0.001s
I20260709 12:19:22.708838 31966 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:22.667342  4834 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:22.411038 31936 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:22.276917  4993 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.710431  4868 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:22.352084 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.591374 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.599685 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 329820us
I20260709 12:19:22.255303  4861 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.602931 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 374053us
W20260709 12:19:22.609516 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 238856us
I20260709 12:19:22.512329 31865 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 6 to 9, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 9 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.606877  4821 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.665508 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:22.719389 32085 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:22.650875  4919 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.720713 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:22.681813 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 415661us
I20260709 12:19:22.695979 32117 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:19:22.723331 32117 pending_rounds.cc:85] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
I20260709 12:19:22.723708 32117 pending_rounds.cc:107] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:19:22.723990 32117 raft_consensus.cc:2889] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:22.696456 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 446899us
I20260709 12:19:22.607103  4883 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:22.607594  4791 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.664389  4999 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:22.664340  4901 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:22.670892  5007 consensus_queue.cc:1243] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:22.692709  4684 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.717118 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 299388us
I20260709 12:19:22.746253 31936 leader_election.cc:400] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:19:22.746726 31936 leader_election.cc:403] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:19:22.669624 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.748265 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.702347  4935 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.727746 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.751909 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.732s (SENT) [suppressed 18 similar messages]
I20260709 12:19:22.732535  4934 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:22.708099  5037 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:22.719162  4927 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.734186 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:22.735018 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69590.3us
I20260709 12:19:22.730926  4955 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.761013 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:22.670579  5025 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.718001  4821 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:22.729256 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:22.776994 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.703660  4894 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.725401  4861 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.757308  4771 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.711956  4834 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.729127  4883 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:22.710922  4993 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.708458  4862 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.760465 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:22.738341  4999 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:22.708761  5013 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.758946  5037 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:22.750641  4919 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:22.739219  4944 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:19:22.743911  4918 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:19:22.759611  4722 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:19:22.760660  5036 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:22.786036 32116 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:22.759382  4935 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.782127 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.763012  4934 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.776141  4907 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:22.798746 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:22.756105  4901 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.844074 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.845184 32121 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 18 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:22.799639 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:22.815780 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 88343.2us
I20260709 12:19:22.816075 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.759826  4866 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:22.758785  5044 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:19:22.780350  4821 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.821336  4868 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:22.755913  4684 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.831140 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.881570 32116 raft_consensus.cc:2410] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:19:22.785889 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56708.3us
I20260709 12:19:22.882197 32120 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:22.847172 32119 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:22.882881 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:22.883806  4934 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:22.883855  4888 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:22.824522  4919 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.884496  4866 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.885850 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:22.886155  4919 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.886234  5007 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:22.886608  4866 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.886746  4919 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:22.887137  4866 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:22.887466  4919 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.888010  4919 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.816578  4950 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:22.833529 32112 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:22.824386  4921 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:22.889235  4950 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:22.889339  4921 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:22.783476  4883 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.767225  4955 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.886495 32659 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
W20260709 12:19:22.823989  5027 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:19:22.828626  4918 consensus_queue.cc:1243] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:22.904532  4684 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.905543  4893 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:19:22.907160  5042 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:22.907800  5037 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.909147  5037 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.911100 32657 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.912416 32114 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:22.890190  4950 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.899318 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.890190  4921 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:22.901549  4934 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:22.890892  5043 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.890039  4919 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:22.915800 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.916399 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 634) took 2102 ms. Trace:
W20260709 12:19:22.925510 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 164566us
I20260709 12:19:22.880590  4868 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:22.933961 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 182061us
W20260709 12:19:22.964574 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.951697 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:22.929762 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:22.949549 31963 rpcz_store.cc:276] 0709 12:19:20.813738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.457487 (+643749us) service_pool.cc:224] Handling call
0709 12:19:21.457767 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.579892 (+122125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.579939 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.579992 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:21.580045 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.916177 (+1336132us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000b5754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:22.916218 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.916334 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280316672}
I20260709 12:19:22.973668 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:22.979326 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83451.6us
I20260709 12:19:22.758884 31865 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 14, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 14 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:22.981338 32119 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 11: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:22.982656 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:22.924942  5030 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:19:22.987555  4684 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.875191  5035 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.087s
W20260709 12:19:22.987954 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 198018us
I20260709 12:19:22.921329  4934 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:23.000885 32657 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:22.990468  4579 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:22.874541  4907 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.978790  4868 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:23.013554 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:23.023659 32045 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:23.024112 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:23.024745 32046 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.979051  4926 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:23.039577  5054 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.040463  4951 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:23.053886  4710 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:23.063583 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45690.6us
I20260709 12:19:23.069926 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:22.991361  4684 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:22.997345  5052 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.989650  5044 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:19:23.061209 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 131340us
I20260709 12:19:22.989866  5036 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:19:23.037020  4791 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.079494 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:23.018504  4972 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 12:19:23.083760 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 132117us
I20260709 12:19:22.893435  4999 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.059921  4951 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:23.086984 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:23.086018 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:23.099052 32042 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:23.064575 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:23.066156  5050 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:23.096444 32047 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:23.077387  4861 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.065073  4926 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.079136  5014 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:23.084487  5052 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.130087 31970 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:22.965068  5043 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.113482 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:23.119989 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.115545 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:23.115208 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:23.114406 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.124135 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:23.136041 31966 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.127213 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.132805 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.139117  4676 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.140187  4676 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.140244 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.143486 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 800) took 1678 ms. Trace:
I20260709 12:19:23.143970 32655 rpcz_store.cc:276] 0709 12:19:21.464766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.464943 (+   177us) service_pool.cc:224] Handling call
0709 12:19:21.478635 (+ 13692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.711316 (+232681us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:21.764606 (+ 53290us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.210104 (+445498us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.210518 (+   414us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.210733 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.210780 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:22.210828 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.142609 (+931781us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b180026e900. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419
0709 12:19:23.143213 (+   604us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54004a2c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:23.143264 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.143397 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58685,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":47442,"raft.run_wall_time_us":32717,"spinlock_wait_cycles":672097792,"thread_start_us":395558,"threads_started":1,"wal-append.queue_time_us":511609}
I20260709 12:19:23.095923  4955 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.089576  5009 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.103392  4791 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:23.152038  4791 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:23.136234  5052 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:23.152473  5047 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:19:23.097932  4729 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:23.155390 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:23.156973 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.166072 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.167176 32112 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:23.160178 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.102281  4927 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.171834 32119 raft_consensus.cc:1217] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-17.8]   Dedup: 10.6->[15.7-17.8]
I20260709 12:19:23.114248  4579 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:23.117967  4999 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:23.172303 32119 pending_rounds.cc:85] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 6
I20260709 12:19:23.116313  5025 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:23.173465 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.087504  4821 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.183328 32119 pending_rounds.cc:107] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:23.185608 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.204126 32119 raft_consensus.cc:2889] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:23.133810  4972 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.131137  5035 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:23.130578  5050 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:23.066555  5054 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.152534  4791 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.171416  5009 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:23.214563 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 76334.2us
W20260709 12:19:23.215024 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59721.5us
I20260709 12:19:23.163365  4901 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.085314  4934 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.213292 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67389.9us
I20260709 12:19:23.212742 32120 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:23.246531 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:23.253770 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.202256  5013 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:23.215109  5031 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 12:19:23.175786  4579 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:23.271966 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.275759 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:23.212311  5067 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:23.319990 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.317701 32046 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:19:23.320885 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45226.2us
I20260709 12:19:23.245425  5036 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:23.331019 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.277935  5030 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:19:23.267764  5047 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:19:23.245476  4862 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:23.387578 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.005s	sys 0.013s
W20260709 12:19:23.388141 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.005s	sys 0.013s
I20260709 12:19:23.236480  4972 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.177358  4729 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.322577  4908 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.270s	user 0.016s	sys 0.000s
W20260709 12:19:23.370466 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.402335 32046 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:23.222237  4710 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.107s
W20260709 12:19:23.387004 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.246265  5065 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:23.327136  5067 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:23.330005  5056 raft_consensus.cc:2764] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:23.340791  4934 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:23.253455  5050 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.417816  5078 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:23.418511  5078 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.420846  5078 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.420974 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2454) took 1969 ms. Trace:
I20260709 12:19:23.421420 32043 rpcz_store.cc:276] 0709 12:19:21.452081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.036783 (+584702us) service_pool.cc:224] Handling call
0709 12:19:22.037147 (+   364us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:22.076647 (+ 39500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.076683 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:22.087709 (+ 11026us) log.cc:844] Serialized 78 byte log entry
0709 12:19:22.673036 (+585327us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:22.910521 (+237485us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.069131 (+158610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.069265 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.420759 (+351494us) raft_consensus.cc:1685] finished
0709 12:19:23.420779 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.420919 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30927,"prepare.queue_time_us":173019,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":269254,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":60236,"thread_start_us":820924,"threads_started":3,"wal-append.queue_time_us":653786}
I20260709 12:19:23.423862 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.423949 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.429445  5057 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.430040  5057 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.432129 32046 raft_consensus.cc:1217] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-14.9]   Dedup: 10.7->[12.8-14.9]
I20260709 12:19:23.432345  5057 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.245447  5035 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.434238 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 802) took 1336 ms. Trace:
I20260709 12:19:23.435134  5025 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.435637  5035 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.434634 32113 rpcz_store.cc:276] 0709 12:19:22.097812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.097966 (+   154us) service_pool.cc:224] Handling call
0709 12:19:22.098275 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.213016 (+114741us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:22.599974 (+386958us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.876923 (+276949us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.880360 (+  3437us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.880542 (+   182us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:22.880587 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.880671 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.434023 (+553352us) raft_consensus.cc:1685] finished
0709 12:19:23.434043 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.434162 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163260,"prepare.queue_time_us":270416,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":170084,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":18305,"thread_start_us":518554,"threads_started":2,"wal-append.queue_time_us":378703}
I20260709 12:19:23.435729  5025 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.436995 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.437603 32113 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:23.438614 31966 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:23.432670 32046 pending_rounds.cc:85] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 7
I20260709 12:19:23.439193 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.439574 32046 pending_rounds.cc:107] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:19:23.439828 32046 raft_consensus.cc:2889] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:23.446038 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.447290 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:23.449865 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:23.450811 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 327768us
I20260709 12:19:23.451815 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:23.453347 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 206896us
I20260709 12:19:23.454879 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:23.400243  5059 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:23.456354 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 474651us
I20260709 12:19:23.215727  4951 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.385365  4919 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
W20260709 12:19:23.530606 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.452029  5030 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:23.537294 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.442153  4862 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:23.163271 31865 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 14, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 14 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:23.361339  5036 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.450773  5065 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:23.560639  5064 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
W20260709 12:19:23.564303 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.535017  4927 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:19:23.584064 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 132300us
I20260709 12:19:23.585915 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:23.540678  4907 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.583199  4926 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.554958  4934 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:23.596755 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:23.578837  4951 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.575402  4684 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.584301  5059 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.582826  4927 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:23.447013  4579 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:23.595403  5048 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.419s	user 0.006s	sys 0.011s
W20260709 12:19:23.622783 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.623464 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:23.631862 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 176971us
I20260709 12:19:23.633322 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:23.634485 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.650893  5080 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:23.651815  5080 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:23.652200  5080 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:23.652580  5080 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.653940  5087 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:23.655140  4676 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.514s	user 0.003s	sys 0.007s
I20260709 12:19:23.618017  4934 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.658636  5086 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.659385  5086 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:23.659288  4934 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.659850  5086 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.660245  5086 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.660784  4972 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.661206 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.663491  4972 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:23.664422  5080 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.666563  5080 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.598623  5074 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.670583 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 632) took 2942 ms. Trace:
I20260709 12:19:23.698103 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.699515 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:23.698103 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:23.711992 32114 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.718812 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.719714 32655 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:23.706604 31964 rpcz_store.cc:276] 0709 12:19:20.728178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.392726 (+664548us) service_pool.cc:224] Handling call
0709 12:19:21.393081 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.708386 (+1315305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.670295 (+961909us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400068e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:23.670521 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2741620992}
I20260709 12:19:23.632598  4927 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:23.690876  5020 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.293s	user 0.008s	sys 0.000s
W20260709 12:19:23.673934  5066 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:19:23.733989 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:23.725638 31964 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:23.735821 31968 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:19:23.680150  5043 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.727509 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:23.677527  4952 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.023s	user 0.008s	sys 0.001s
W20260709 12:19:23.670288  4883 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.765s	user 0.006s	sys 0.000s
I20260709 12:19:23.728216 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.729117 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.767367 31969 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
W20260709 12:19:23.674798  5017 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.459s	user 0.000s	sys 0.003s
W20260709 12:19:23.679860  4997 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.417s	user 0.007s	sys 0.004s
W20260709 12:19:23.692698  4933 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.394s	user 0.010s	sys 0.001s
W20260709 12:19:23.693842  5037 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.784s	user 0.008s	sys 0.004s
W20260709 12:19:23.694764  4966 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.635s	user 0.008s	sys 0.004s
I20260709 12:19:23.687332  4935 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.685994  4579 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:23.673379  5036 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.786826 32112 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:23.680676  5054 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.726610 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:23.685232  5025 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.741344 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.741689 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.800773 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.742249 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.747079 31967 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:23.691108  4729 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:23.747776 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 114539us
W20260709 12:19:23.782045  4868 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.000s
I20260709 12:19:23.726408  4921 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.773494 31969 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:23.781298 32114 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:23.738840  4676 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.813743 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:23.738139  5084 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:23.803207 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:23.820390 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:23.585892  5067 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.775333  5009 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.799733 31965 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
W20260709 12:19:23.799677 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 73145.8us
I20260709 12:19:23.723695  5074 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.693341  5086 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.680052  5065 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.821597 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.822144 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.822154 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.835826 31969 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:23.782790  4908 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.823236 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 901) took 1103 ms. Trace:
I20260709 12:19:23.826341 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:23.769686  4952 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.807639  4984 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.723s	user 0.014s	sys 0.000s
I20260709 12:19:23.810405  4935 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:23.825065  4834 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.033s	user 0.011s	sys 0.000s
I20260709 12:19:23.836738 32652 rpcz_store.cc:276] 0709 12:19:22.720287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.721824 (+  1537us) service_pool.cc:224] Handling call
0709 12:19:22.722173 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.915988 (+193815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.916019 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.924873 (+  8854us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.067613 (+142740us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:23.067692 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.067887 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.822961 (+755074us) raft_consensus.cc:1685] finished
0709 12:19:23.823000 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.823167 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2222,"raft.queue_time_us":99224,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":22943,"thread_start_us":282437,"threads_started":2,"wal-append.queue_time_us":258366}
I20260709 12:19:23.776070  4883 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.777220  4997 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.839162 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:23.812134  4955 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:23.784147  4966 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.834372  5059 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:23.858825 32041 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:23.824893  5091 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:23.840991  5068 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:23.828352  5090 raft_consensus.cc:2764] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:23.818073  4676 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.814837  4688 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:19:23.817762  4868 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.831928  4961 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.435s	user 0.018s	sys 0.004s
I20260709 12:19:23.840652  4591 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:23.854732  5052 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
I20260709 12:19:23.848405  4935 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.860771 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:23.859074  4999 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
I20260709 12:19:23.849287  5079 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.868772 32039 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:23.835484  5044 consensus_queue.cc:1175] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3836f9b5cd36488da56d4540dc9d80ec" responder_term: 16 status { last_received { term: 14 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:23.859372  5059 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:23.826846  4921 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.873746  4883 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.860394  4955 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.909050 32654 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
W20260709 12:19:23.869395  4961 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.499s	user 0.019s	sys 0.005s
I20260709 12:19:23.892418  4676 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.842655  4908 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.899514 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.872218 32660 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:23.830240  5084 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:23.906831 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.856015  4834 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:23.783424 31865 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 9 to 11, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 11 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:23.923321 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:23.868983  5091 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:23.908463  4935 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:23.926224  5089 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:19:23.931829 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.875447  5050 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.904189  5009 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:23.904597  5059 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:23.916620  5002 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.894s	user 0.007s	sys 0.006s
I20260709 12:19:23.868305  5026 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.899046  5052 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.942699 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:23.943511  5096 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:23.947754  5005 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.649s	user 0.004s	sys 0.010s
I20260709 12:19:23.911895  4591 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:19:23.945590  5092 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.942229  5080 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.961611  5074 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:23.962378  5096 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:23.964175  5009 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:23.998203  4791 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.005s
I20260709 12:19:24.001027 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.008590 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2293) took 2382 ms. Trace:
I20260709 12:19:24.009120 32039 rpcz_store.cc:276] 0709 12:19:21.626128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.280262 (+654134us) service_pool.cc:224] Handling call
0709 12:19:22.390674 (+110412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.896538 (+1505864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.008243 (+111705us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54000587d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:24.008512 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3398620416}
I20260709 12:19:24.010283  5009 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.012085  4997 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:23.903852  5079 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.016477  4791 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.016396  5109 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.025506  4791 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.026973 31967 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:23.963476  5026 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.028035 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2281) took 2950 ms. Trace:
I20260709 12:19:24.025409  5109 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.031646 31865 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:24.033447  5109 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.033624  4821 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.034024  4955 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:23.987846  5080 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.029352 32038 rpcz_store.cc:276] 0709 12:19:21.077440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.077573 (+   133us) service_pool.cc:224] Handling call
0709 12:19:21.077881 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.101423 (+ 23542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.101454 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.101909 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:19:21.658305 (+556396us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:21.658576 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.658669 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.869553 (+2210884us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:24.020746 (+151193us) raft_consensus.cc:1685] finished
0709 12:19:24.020782 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.027932 (+  7150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112728,"raft.queue_time_us":46502,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":22943,"spinlock_wait_cycles":10075904,"thread_start_us":478478,"threads_started":1,"wal-append.queue_time_us":681326}
I20260709 12:19:24.016719  5086 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.042233  5096 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.000069  5091 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.050112  5026 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.062418 32042 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:24.065305 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.068886 32042 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:24.082015 32046 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:19:24.084033 32045 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:24.085040 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.083237 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.089648 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.091452 32046 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:19:24.092710 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.092338 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:24.093190 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.095731 31970 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 12:19:24.099560 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.100176 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:24.093571 32042 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:24.067200  4868 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.102913 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:24.081192  4928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.007s	sys 0.013s
I20260709 12:19:24.104153 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:24.106735 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.107169 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.108517 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.110518 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:24.111116 32112 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.112242 32039 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:24.115423 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.116032 32039 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:24.116678 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.117383 32045 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:24.121726  5084 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.122296  5068 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:24.122797  5068 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.066896 31865 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 7 to 14, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 14 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:24.105877  4928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.008s	sys 0.013s
W20260709 12:19:24.104512  4911 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.897s	user 0.016s	sys 0.011s
W20260709 12:19:24.139092  5104 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:19:24.139485  4777 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:24.142603  4927 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.144021  4935 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.145479 32045 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:19:24.146637 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:24.153611 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:24.155934 32112 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:24.158665 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.136337 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.164683 32039 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:24.244428 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.235347 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:24.262773 32045 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:24.157269  5052 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.166538  4906 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
W20260709 12:19:24.257596 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 94950.4us
I20260709 12:19:24.280037 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:24.234683 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:24.281021 32113 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:24.282198 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:24.282389  5076 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:19:24.283406 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:24.284080 32113 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:24.243088 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 807) took 1448 ms. Trace:
I20260709 12:19:24.157234  4821 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:24.238355 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.262848 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:24.285923 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:24.263758 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:24.286887 32112 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.272970  5068 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.275038  5035 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:24.305789 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.307742 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:24.309991 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.162899  4911 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.012s	user 0.016s	sys 0.011s
I20260709 12:19:24.235965 31865 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 7 to 8, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 8 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:24.287829 32118 rpcz_store.cc:276] 0709 12:19:22.794184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.794499 (+   315us) service_pool.cc:224] Handling call
0709 12:19:22.794865 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.878056 (+ 83191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.878083 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.878136 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:22.878181 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.242763 (+1364582us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b54003e49d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:24.242827 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.243012 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190824832}
I20260709 12:19:24.288594 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:24.235347  5080 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.326531 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.291211 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.292253 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:24.301254  5096 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.330885 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:24.302420  4821 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.316991 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.323738  4853 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:24.294533  4775 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:24.320237  5035 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.320758 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.321444 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.159166  4532 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 12:19:24.348212 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.293303  5059 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.322520  5068 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.323182 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.331447 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.307586 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.356181 31968 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:24.314849 32112 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:24.356871 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:24.356927 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.346848 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2474) took 1251 ms. Trace:
I20260709 12:19:24.357901 32044 rpcz_store.cc:276] 0709 12:19:23.095405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.321126 (+1225721us) service_pool.cc:224] Handling call
0709 12:19:24.346783 (+ 25657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:24.358560 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:24.359249 31968 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:19:24.343324  4775 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.360175  4775 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:24.362486  5007 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:24.347398  4853 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:24.363977  4853 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.346558  5107 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.366178  4834 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.366657  4894 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.368475  5035 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:24.368829  5099 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.410s	user 0.000s	sys 0.001s
I20260709 12:19:24.353900 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.341218  4906 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.356864 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.332224  5026 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:24.364428  4821 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.354975 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:24.373845  5059 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.366266  5107 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.383363  5076 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:24.383548  5107 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.379477 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.387310 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:24.394662  5035 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:24.395061  5036 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:24.395205  5035 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:24.395509  5036 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.395887  5036 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.396116 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.396154  5035 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.397133  4984 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:24.390715  5074 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:19:24.400118  5074 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:24.400543  5074 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.400732  4676 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:24.401353  4676 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:24.401541  4984 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:24.401649  4791 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:24.402920  4791 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:24.402489  4676 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.403339  4791 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:24.403999  4676 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.404382  4791 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.405799  4791 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.401399 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:24.400961  5074 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.342734  5096 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.378713  5026 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.378592  5044 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:24.410034  5007 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.415974  5046 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.433748  5026 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:24.437251  5046 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.439466  4935 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.440580  5046 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.387014  5069 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.379801  4906 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.411993 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:19:24.431501 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52274.3us
I20260709 12:19:24.391034  4532 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:24.444092 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:24.445452 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.445086  4906 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.445674  5096 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.444622  4532 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.467747 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.438089  5044 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:24.466971 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43605.6us
I20260709 12:19:24.472429  4993 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.441380  5007 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.476784  5079 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.458002  5082 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:24.479274 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:24.458626  4919 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:24.483664 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.484270 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.463780  4927 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.480149  4532 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:24.477313  4911 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:24.446246  5069 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.515461 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:24.515811  4900 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.543368 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.545233 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.519381  5074 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.520020  4734 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:24.554136 31967 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:24.558408 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:24.522558  5110 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.288s	user 0.003s	sys 0.000s
W20260709 12:19:24.523847  5060 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.370s	user 0.001s	sys 0.001s
W20260709 12:19:24.524711  5019 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 2.313s	user 0.001s	sys 0.003s
I20260709 12:19:24.528789  5044 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.525952  4911 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:24.530406 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.574730 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.536096 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:24.546617 32038 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:19:24.578691 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.579341 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.579239 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.579850 32038 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:24.547288 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:24.549566 32043 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:24.559207 31970 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:24.581103 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.581252 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.583366 31971 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:24.564945 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.584653 32046 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 11.
I20260709 12:19:24.526165  4993 raft_consensus.cc:2764] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:24.566402 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:24.564287  4984 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.509150  5082 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:24.592836 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.569173 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.593632 32044 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:24.569787 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.594797 32045 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:24.595049 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:24.507714  5096 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:24.566908 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:24.598497 32659 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:24.572511  5015 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:24.595890 32652 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:19:24.591111 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:24.603972 32043 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.547487  5107 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:24.596331 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 112196us
I20260709 12:19:24.569363  4901 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:24.551326  4900 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.600925 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.625389 32046 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:24.596153 31970 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:24.575711  4734 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:24.610913 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.583149 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:24.615368 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:24.627568 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44500.7us
I20260709 12:19:24.627967 32044 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:24.630505  4993 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:24.591763  4984 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:24.599296  5082 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.634644 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:24.637195 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.598336  5096 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.645603  5053 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:24.651930 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.654238 32651 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:24.589681  5044 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:19:24.602936  5015 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:24.656566 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.670146 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.671070 32044 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:24.657766 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:24.613369  4901 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.608745  5052 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:24.664659 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.673477 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2556) took 1904 ms. Trace:
I20260709 12:19:24.649914  5035 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.650617  4984 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:24.665900  5119 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:19:24.679828 32115 rpcz_store.cc:276] 0709 12:19:22.769356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.769456 (+   100us) service_pool.cc:224] Handling call
0709 12:19:22.769818 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.781726 (+ 11908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.781770 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.782330 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.987640 (+205310us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:22.987695 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.987761 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.523292 (+1535531us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:24.660856 (+137564us) raft_consensus.cc:1685] finished
0709 12:19:24.660887 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.673388 (+ 12501us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21094,"raft.queue_time_us":17300,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":29133,"spinlock_wait_cycles":42332416,"thread_start_us":155165,"threads_started":1,"wal-append.queue_time_us":266370}
W20260709 12:19:24.686519 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.654129  4729 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.691795 32043 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:24.692713 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.684722 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:24.696077 32112 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:19:24.696568 32112 pending_rounds.cc:85] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 6
I20260709 12:19:24.696844 32112 pending_rounds.cc:107] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:19:24.697139 32112 raft_consensus.cc:2889] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:24.697504 32112 pending_rounds.cc:107] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:19:24.697773 32112 raft_consensus.cc:2889] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:24.673040  4929 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:24.660941  5096 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.632562  4919 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:19:24.606102  5036 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.659696  5079 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.659235  5053 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.678725  5093 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:24.728888 32112 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:24.728639  5079 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:24.730895  5123 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:19:24.731073 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2517) took 3702 ms. Trace:
I20260709 12:19:24.731534 32117 rpcz_store.cc:276] 0709 12:19:21.028356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.034739 (+  6383us) service_pool.cc:224] Handling call
0709 12:19:21.035023 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.724839 (+1689816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.730817 (+2005978us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540012acd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:24.730991 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3815105408}
I20260709 12:19:24.706144  4821 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.734640  4821 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.735045  4821 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.735532 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 641) took 1612 ms. Trace:
I20260709 12:19:24.735888 31963 rpcz_store.cc:276] 0709 12:19:23.123441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.123659 (+   218us) service_pool.cc:224] Handling call
0709 12:19:23.124005 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:23.321381 (+197376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.321426 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:23.665497 (+344071us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:23.666046 (+   549us) log.cc:844] Serialized 52 byte log entry
0709 12:19:23.834926 (+168880us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:23.856940 (+ 22014us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.859096 (+  2156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.959730 (+100634us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b480036c0d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:24.368235 (+408505us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:24.735225 (+366990us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540034d854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:24.735272 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.735317 (+    45us) raft_consensus.cc:1685] finished
0709 12:19:24.735332 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.735466 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6935,"prepare.queue_time_us":21907,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":47,"raft.queue_time_us":4496,"raft.run_cpu_time_us":1660,"raft.run_wall_time_us":128719,"spinlock_wait_cycles":470620032,"thread_start_us":168501,"threads_started":1,"wal-append.queue_time_us":255143}
I20260709 12:19:24.728631  5053 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:24.743040  5079 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:24.744740  5029 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 2.016s	user 0.000s	sys 0.012s
I20260709 12:19:24.678748  4901 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:24.748373  4821 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:24.753690  5128 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:19:24.722820 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.686439  5108 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:24.685277  5121 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:19:24.755968 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.775317 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.775758 31963 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:24.776292 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.766706 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:24.780834 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:24.781972 31963 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.726727  5007 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:24.781044 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 120796us
I20260709 12:19:24.683318  5052 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:24.773619  5124 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:19:24.704926  5127 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
W20260709 12:19:24.779090  5105 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:19:24.796058  5118 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.357s	user 0.000s	sys 0.003s
I20260709 12:19:24.795622  4775 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.795678 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:24.759492  5093 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:24.754544  4734 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.756361  5015 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.774109  5125 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:24.808409 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.813862 31963 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:24.797065  5074 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.829456 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.830391 32038 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:24.831426 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:24.832074 32038 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:24.821094  4857 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.849175  5069 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.857290 32655 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:24.860874  5107 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.866499 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:24.867799 32651 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
W20260709 12:19:24.868682 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 60397.8us
I20260709 12:19:24.870047 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.839550 32038 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:24.872403 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:24.775246  4821 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.848068  5036 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.879370 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:24.887652 32038 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:19:24.881588 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 121601us
I20260709 12:19:24.863541  5015 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.822767  4900 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:24.840648  5125 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.900022 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:24.900867 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:24.883513  5065 raft_consensus.cc:2764] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:19:24.885128  4791 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.904806 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:24.912791 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:24.888048  5107 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:24.909415 32039 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:24.893095  5068 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:24.914948  5075 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:24.910776  4771 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:24.889948  5053 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.924849  5075 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:24.915828  4944 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:24.906554  4900 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:19:24.928486  5075 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.941787  5062 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:24.942348  5062 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:24.918489  4929 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:19:24.946775  4929 consensus_queue.cc:1243] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:24.947168 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1012) took 2168 ms. Trace:
I20260709 12:19:24.947343  4929 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:24.947912  4929 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:24.949108  4929 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.947594 32657 rpcz_store.cc:276] 0709 12:19:22.778653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.778775 (+   122us) service_pool.cc:224] Handling call
0709 12:19:22.779061 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.001308 (+222247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.946864 (+1945556us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:24.947078 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303274752}
I20260709 12:19:24.925734  5035 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:24.925652  4771 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.950975  5035 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:24.943305  5062 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.951305  4771 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.951491  5035 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:24.951792  5036 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.915091  4791 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.956097 32113 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:19:24.956055  5096 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:24.924091  5068 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:24.957835  4966 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:24.960268  5052 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:24.964017 32660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 904) took 1540 ms (client timeout 1734 ms). Trace:
I20260709 12:19:24.966590 32656 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:24.968679  5033 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 2.232s	user 0.000s	sys 0.003s
I20260709 12:19:24.969465  4901 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.970342  5053 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.973388 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 809) took 2001 ms. Trace:
I20260709 12:19:25.012053 32116 rpcz_store.cc:276] 0709 12:19:22.971940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.972115 (+   175us) service_pool.cc:224] Handling call
0709 12:19:22.972426 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.080615 (+108189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.080646 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:23.080694 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:23.080724 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.973140 (+1892416us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001487d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:24.973189 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.973322 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248344448}
I20260709 12:19:24.928887  4944 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:24.994246 32113 pending_rounds.cc:85] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 9
I20260709 12:19:24.973639  4906 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:25.002789 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.001597 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2459) took 3418 ms. Trace:
I20260709 12:19:25.006734  4908 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.996584  4775 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:24.905014  5093 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:25.008122 32660 rpcz_store.cc:269] 0709 12:19:23.423456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.423572 (+   116us) service_pool.cc:224] Handling call
0709 12:19:24.963824 (+1540252us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54004a2c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:24.963961 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008487680}
W20260709 12:19:25.001832  5026 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.005s
I20260709 12:19:25.014453 32113 pending_rounds.cc:107] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:19:25.031170 32113 raft_consensus.cc:2889] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:25.031424 32115 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:25.018452 32040 rpcz_store.cc:276] 0709 12:19:21.583637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.059077 (+475440us) service_pool.cc:224] Handling call
0709 12:19:22.059548 (+   471us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:22.082976 (+ 23428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.083031 (+    55us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:22.083996 (+   965us) log.cc:844] Serialized 78 byte log entry
0709 12:19:22.423137 (+339141us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:22.714141 (+291004us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.715566 (+  1425us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.720078 (+  4512us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.676040 (+955962us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b48000fe358. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:24.937693 (+1261653us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800092e00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:19:25.001190 (+ 63497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.001289 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.001343 (+    54us) raft_consensus.cc:1685] finished
0709 12:19:25.001361 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.001538 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14684,"prepare.queue_time_us":827492,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":5435,"raft.queue_time_us":333498,"raft.run_cpu_time_us":2697,"raft.run_wall_time_us":983666,"spinlock_wait_cycles":37584512,"thread_start_us":583982,"threads_started":3,"wal-append.queue_time_us":451987,"wal-append.run_cpu_time_us":7538,"wal-append.run_wall_time_us":2024303}
W20260709 12:19:25.015645 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.015973 31967 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:24.996591  5134 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.014602  5062 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.050760 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2285) took 1195 ms. Trace:
I20260709 12:19:25.054714 31965 rpcz_store.cc:276] 0709 12:19:23.855583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.855774 (+   191us) service_pool.cc:224] Handling call
0709 12:19:23.856061 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:23.861708 (+  5647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.861760 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:24.740359 (+878599us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b480036c0d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:24.791606 (+ 51247us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:24.809521 (+ 17915us) log.cc:844] Serialized 52 byte log entry
0709 12:19:24.993252 (+183731us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:24.993315 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.994950 (+  1635us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.050494 (+ 55544us) raft_consensus.cc:1685] finished
0709 12:19:25.050517 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.050693 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2396,"raft.queue_time_us":3590,"raft.run_cpu_time_us":4911,"raft.run_wall_time_us":878318,"spinlock_wait_cycles":14393472,"thread_start_us":135074,"threads_started":1,"wal-append.queue_time_us":203472}
I20260709 12:19:25.057428  4771 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.059872  5096 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.015439  5125 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:25.063074 32623 leader_election.cc:336] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.732s (SENT)
I20260709 12:19:25.015823  4944 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:25.052832  5068 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.15, Last appended by leader: 15, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:24.998049  5036 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.043756  5046 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:25.071934 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 159225us
I20260709 12:19:25.075798 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:25.083761 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:25.085173 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:25.075245  4961 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.086540 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:25.048278  5026 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.084396  4710 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:25.098587  4710 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.098973  4710 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:25.088199  4900 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:25.104213 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2468) took 2015 ms. Trace:
I20260709 12:19:25.107296  4791 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:25.094230  4931 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:19:25.088356  4734 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.088486  5096 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:25.116808  5036 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.121376  4901 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.104844 32047 rpcz_store.cc:276] 0709 12:19:23.088363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.857379 (+1769016us) service_pool.cc:224] Handling call
0709 12:19:24.857769 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.858154 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.858189 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.858678 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.979820 (+121142us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:24.979868 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.979957 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.103801 (+123844us) raft_consensus.cc:1685] finished
0709 12:19:25.103834 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.104131 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":120832,"threads_started":1,"wal-append.queue_time_us":185590}
I20260709 12:19:25.083446  4984 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.126859 31971 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:25.127991  5026 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.132457  4771 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:25.136597 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.137573 32038 raft_consensus.cc:2376] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15. Re-sending same reply.
I20260709 12:19:25.138837 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.139489  5076 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:25.139889 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.140069  5076 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.140362 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.141589 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.141808 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.142552 32044 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:25.143515 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.145059 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.145845 32040 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:25.138922 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.142673 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.149019 32042 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:25.078104  5046 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.151674 32046 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:25.155103 31965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:25.155746 31965 raft_consensus.cc:606] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:25.156129 31965 raft_consensus.cc:614] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:25.161837 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.164527 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.165205 32047 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:25.168967 32046 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:25.169377 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.171870 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.172597 32038 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:25.114300  5082 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.176421  5082 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:25.177230  5072 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.776s	user 0.005s	sys 0.000s
I20260709 12:19:25.178470 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2548) took 2022 ms. Trace:
I20260709 12:19:25.178890 32119 rpcz_store.cc:276] 0709 12:19:23.155882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.171260 (+ 15378us) service_pool.cc:224] Handling call
0709 12:19:23.171640 (+   380us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:23.205054 (+ 33414us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:23.253366 (+ 48312us) log.cc:844] Serialized 14 byte log entry
0709 12:19:24.951851 (+1698485us) spinlock_profiling.cc:243] Waited 984 us on lock 0x7b180033d9c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a
0709 12:19:24.952085 (+   234us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:24.952704 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 12:19:25.074636 (+121932us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:25.074675 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.074807 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.178078 (+103271us) raft_consensus.cc:1685] finished
0709 12:19:25.178181 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.178391 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13193,"prepare.queue_time_us":62158,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":130606,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":96043008,"thread_start_us":207595,"threads_started":2,"wal-append.queue_time_us":585360,"wal-append.run_cpu_time_us":1124,"wal-append.run_wall_time_us":1031410}
W20260709 12:19:25.152159  4532 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.000s
W20260709 12:19:25.182579 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.182853 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.183681 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.185071 32046 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.185072 32042 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:25.191840 32038 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:25.193076 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.195003 32042 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:25.184886  4532 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.184625  4771 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:25.202978 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.208384 32046 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:25.205561 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:25.219462  4885 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.221110  4532 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.188524  5091 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:19:25.222241  5091 consensus_queue.cc:1243] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:19:25.222791  5091 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:25.223124  5091 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:25.224095  5091 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.225404  5091 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.210719  4771 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.232266 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:25.240867 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2259) took 3757 ms. Trace:
I20260709 12:19:25.241407 31972 rpcz_store.cc:276] 0709 12:19:21.483550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.810479 (+326929us) service_pool.cc:224] Handling call
0709 12:19:21.810896 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.751435 (+940539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.751467 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.751510 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:22.751539 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.240626 (+2489087us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54001e91d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:25.240678 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.240807 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162542848}
I20260709 12:19:25.239400  4777 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:25.235793  4919 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:25.252418 32045 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:25.258656 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.259598  5093 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:25.259975  5093 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.260325  5093 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:25.260306 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:25.261332  5075 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.261858 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.262789  5068 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.263548  5068 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.264529 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.265786  5068 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.265879  4579 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.266077 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:25.266530  4927 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.267594  5052 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.268574  5052 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:25.268160  5093 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.271339  5093 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:25.262756  4966 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:25.272097  4966 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.266567  4579 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:25.273545  4579 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.274487  4966 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.278280  4927 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.279006 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.281023 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:25.281275 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:25.281999 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:25.285718 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.286226 31968 raft_consensus.cc:2368] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:25.238816  4771 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.283604 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:25.288661 31963 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:25.291253 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:25.292089 31965 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.287830  4734 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:19:25.252920  4777 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:25.295063  4777 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.296484  4777 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.296742 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.298363 32120 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:19:25.298985 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:25.299856 31972 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:25.300117  4911 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.300688 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:25.301429  4911 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.301774 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:25.298959 32120 pending_rounds.cc:85] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 7
I20260709 12:19:25.302629 32120 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:19:25.302887 32120 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:25.303337 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:25.304198 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2551) took 1950 ms. Trace:
I20260709 12:19:25.304623 32120 rpcz_store.cc:276] 0709 12:19:23.354238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.398201 (+ 43963us) service_pool.cc:224] Handling call
0709 12:19:23.398585 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.303335 (+1904750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.303966 (+   631us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54002b52d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:25.304134 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4367890176}
I20260709 12:19:25.305540  4966 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:25.305876 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.311041 32040 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:25.306512  5073 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.315402 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:25.318655 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:25.317951 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:25.319664 32119 raft_consensus.cc:2368] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:25.319693 32116 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:19:25.320865 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:25.321079 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:25.321470 31968 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.318560 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:25.323851 32114 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:25.324189 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.324476 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:25.325268 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.325850 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:25.326776 32119 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:25.322350 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:25.287065 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.328649 31972 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:25.322964 32118 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:25.383787 32013 leader_election.cc:400] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
W20260709 12:19:25.378762 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53692.7us
I20260709 12:19:25.279965  4919 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.356464 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.238111  4944 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.390965 32112 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:25.279881  5062 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:25.413691 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 126763us
I20260709 12:19:25.416392 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.398015 31968 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:25.288964  5068 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.390416 31972 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:25.388707 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:25.463392 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:25.390843  5152 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:19:25.288146  5084 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:25.476097 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.340945  4885 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.390516  5013 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.330686  4952 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.506958 32038 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:25.512038 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:25.513795 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:25.408504  5062 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.520259 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.422297  5087 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.411101  4919 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.525818  5009 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.534499  5067 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.535008 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.536029 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2291) took 3914 ms. Trace:
I20260709 12:19:25.558914 32041 rpcz_store.cc:276] 0709 12:19:21.621394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.159144 (+537750us) service_pool.cc:224] Handling call
0709 12:19:22.159453 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.859545 (+1700092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.535801 (+1676256us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54001a72d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:25.535973 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3907715840}
I20260709 12:19:25.528483 32038 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:25.537269 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47544 (request call id 1017) took 1812 ms. Trace:
I20260709 12:19:25.560422 32654 rpcz_store.cc:276] 0709 12:19:23.725025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.727259 (+  2234us) service_pool.cc:224] Handling call
0709 12:19:25.537102 (+1809843us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400323f54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:25.537211 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380683008}
I20260709 12:19:25.538506 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2491) took 1228 ms. Trace:
I20260709 12:19:25.560935 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:25.541141  5153 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:19:25.357614  4532 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:25.554209 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:25.397997 31965 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:25.331116  5035 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.540426 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2288) took 1616 ms. Trace:
I20260709 12:19:25.575778 31964 rpcz_store.cc:276] 0709 12:19:23.924167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.924328 (+   161us) service_pool.cc:224] Handling call
0709 12:19:23.924757 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.934368 (+  9611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.934417 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:23.934489 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:23.934537 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.540168 (+1605631us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b5400455bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:25.540222 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.540362 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21524352}
I20260709 12:19:25.561414 31966 rpcz_store.cc:276] 0709 12:19:24.309801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.359394 (+ 49593us) service_pool.cc:224] Handling call
0709 12:19:24.359713 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.364236 (+  4523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.364270 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.364777 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.465324 (+100547us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:24.502470 (+ 37146us) log.cc:844] Serialized 14 byte log entry
0709 12:19:24.502838 (+   368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.502919 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.007422 (+504503us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b18002d1180. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:25.534904 (+527482us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b18002d1180. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:25.538193 (+  3289us) raft_consensus.cc:1685] finished
0709 12:19:25.538222 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.538429 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7412,"prepare.queue_time_us":34420,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":8462,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":1509,"spinlock_wait_cycles":43988096,"thread_start_us":100127,"threads_started":1,"wal-append.queue_time_us":146949}
I20260709 12:19:25.569849 32042 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:25.572930 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.573107 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.527179  4729 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.581423 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.543359  5062 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.579218 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.582156 32042 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.586549 32041 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:25.585034 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.314630  5073 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.602694  5035 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.609725 32042 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:25.531735  4885 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.524649  5084 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.625789 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:25.614367  5081 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 2.030s	user 0.004s	sys 0.000s
I20260709 12:19:25.638726 32112 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:19:25.555613  4956 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.588441  4532 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.589732  4729 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.647619 32117 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.624585  4921 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.657271 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2546) took 2502 ms. Trace:
W20260709 12:19:25.654544 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 265892us
I20260709 12:19:25.657792 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:25.656479 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.657711 32121 rpcz_store.cc:276] 0709 12:19:23.155285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.160792 (+  5507us) service_pool.cc:224] Handling call
0709 12:19:23.161170 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.161575 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.161615 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.162171 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.526597 (+1364426us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b18004b8b60. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1
0709 12:19:24.526746 (+   149us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:24.526786 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.526850 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.422421 (+895571us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:25.657067 (+234646us) raft_consensus.cc:1685] finished
0709 12:19:25.657088 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.657216 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105647,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":44994,"spinlock_wait_cycles":23796224,"thread_start_us":154590,"threads_started":1,"wal-append.queue_time_us":296487}
I20260709 12:19:25.660923 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:25.643366  5076 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.656108  5082 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:25.667830 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:25.676158 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:25.668956  4993 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:25.664973  4944 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.690891  5013 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:25.671499  5082 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:25.697402  4710 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.001s
I20260709 12:19:25.696563  5013 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.716598  5082 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.715225  5013 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.716153  4729 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.734961  5076 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.731779  5084 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:25.741142  5091 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.003s
W20260709 12:19:25.814191 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.824664 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 408231us
I20260709 12:19:25.828658 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.837052 32041 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:25.848107 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.850138 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.851624 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:25.852463 32043 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:25.857029 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:25.860014 32117 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:25.656944  4579 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.873396 32121 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:25.712749  4710 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.878342  4956 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.882736  4993 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:25.884768  4993 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.887365  5078 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.888608  5078 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:25.910115  4579 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:25.917936 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55084.4us
W20260709 12:19:25.920192 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 91901.8us
I20260709 12:19:25.929067 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.930676  5166 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.938067 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.939124  5166 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.941502 32010 leader_election.cc:400] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:25.941917 32010 leader_election.cc:403] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:25.942420  4929 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.943091  5166 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:25.947499  5078 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:25.949527 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:25.953337 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.953938 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.954041  5166 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:25.958096 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:25.941901  4710 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.961613  5166 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:25.962692  5166 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:25.964807  5166 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.970585  4956 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:25.972541  5091 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:25.975600  5091 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:25.975826 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.976999  5091 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.978289  5147 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:25.978977  5165 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:25.979622  5165 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:25.979705  5147 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:25.980304  5165 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.980397  5147 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:25.981863  5147 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:25.983672  5147 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.983680 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.986706  5086 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.972735  4885 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.990468 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:25.994967  5086 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:25.995313  4532 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:25.995710 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1010) took 3234 ms. Trace:
I20260709 12:19:25.997586 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:26.008019  5086 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:25.996199 32658 rpcz_store.cc:276] 0709 12:19:22.761102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.761316 (+   214us) service_pool.cc:224] Handling call
0709 12:19:22.761683 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.820100 (+ 58417us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:23.075469 (+255369us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.255119 (+179650us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:23.255173 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:23.255217 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.965528 (+1710311us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x7b180038e4e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db
0709 12:19:25.176700 (+211172us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:25.995150 (+818450us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b5400526cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:25.995242 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.995468 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":271692,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":248666,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":31170,"spinlock_wait_cycles":161103232,"thread_start_us":431989,"threads_started":3,"wal-append.queue_time_us":202480}
I20260709 12:19:26.028636  5086 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:26.041150 32043 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:26.031293  5165 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.057076 32121 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:26.059386 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.063776  4961 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.064867  5086 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.064697  4961 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.070190  5091 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.074167  4906 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:26.077050 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.078084  5166 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:26.079214 32039 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:26.080796  5067 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.010s
I20260709 12:19:26.081331 32039 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:26.081909 32120 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:19:26.082296 32120 pending_rounds.cc:85] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
I20260709 12:19:26.082617 32120 pending_rounds.cc:107] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:19:26.082916 32120 raft_consensus.cc:2889] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:26.084692  5079 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.085243  5079 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.088393 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:26.089993 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 369589us
W20260709 12:19:26.169269 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 76804.9us
W20260709 12:19:26.178565  4919 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.008s
I20260709 12:19:26.179322  4919 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.181795  4894 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:26.182539  4777 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.885s	user 0.018s	sys 0.007s
I20260709 12:19:26.184968  4894 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:26.185482  4894 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:26.186650 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.186724  4894 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.187680 31970 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:26.188328 31966 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:26.188716  4894 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.189625  4961 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.189950 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.190966  5165 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:26.191191 31964 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:26.193833 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:26.195300 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.206113  4777 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.213388 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:26.224710  5144 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:26.226048 31968 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:26.228324  4961 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.242750  5144 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:26.250370  5062 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.653s	user 0.015s	sys 0.004s
W20260709 12:19:26.253968  5082 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.005s
I20260709 12:19:26.254577  5082 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.257159  5082 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.263350 32040 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:26.272482  5082 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.273280  5082 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.274524  4919 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:26.276842  5062 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.296401  5062 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:26.297490 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:26.298653 32040 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:26.299549 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:26.302692  4888 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:26.304327  4888 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:26.304778  4888 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.305176  4888 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.305258 32116 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:26.305815  4993 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:26.306397  4993 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:26.307741  4993 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.309559  4993 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.331584 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:26.335048  4906 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:26.335829  4906 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:26.337146  4906 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.338006  4894 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:26.339388  4894 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:26.244047  4961 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.339779  4894 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:26.291015  4777 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:26.353029 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 253845us
W20260709 12:19:26.361279 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 190768us
I20260709 12:19:26.366443  5086 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:26.367168  5144 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:26.356407  4532 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:26.368546 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.368315  4532 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.371833  5171 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.372699  5171 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:26.366690 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.375869  4984 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:26.391456  5147 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:26.392102  5084 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:26.392904  5084 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.393877  4927 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.389309  5171 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.394702  4927 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.395239 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:26.395561  5013 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.005s
I20260709 12:19:26.396077  5013 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.397138  4927 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.397212 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.392115  5075 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.398180  4894 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.403514 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2509) took 1099 ms. Trace:
I20260709 12:19:26.414470  5082 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.367246  5086 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.398300  5013 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.403932 32045 rpcz_store.cc:276] 0709 12:19:25.300790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.135681 (+834891us) service_pool.cc:224] Handling call
0709 12:19:26.166399 (+ 30718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:26.167220 (+   821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.167268 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:26.168134 (+   866us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.341930 (+173796us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:26.341984 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.342118 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.389402 (+ 47284us) raft_consensus.cc:1685] finished
0709 12:19:26.389424 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.400161 (+ 10737us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17193,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":45067,"thread_start_us":173260,"threads_started":1,"wal-append.queue_time_us":207681}
I20260709 12:19:26.395565  5084 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:26.422370 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.427917  5086 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:26.429041 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.429242  4984 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.421265  4956 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:19:26.431249 32117 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:26.405429  4777 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.435354 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.435508  4579 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:19:26.440491  5082 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.442258 32118 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:26.442167  5082 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.435362  5144 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:26.443845  5144 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.448357 32112 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:26.452669 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:26.453293 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.453837 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.454025 31963 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.398710  5075 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.469393 31966 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:26.455396  4894 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.474527 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.465865 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.472509 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.430141  4984 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.476468 31964 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:26.475561 32117 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:26.478251 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:26.492822  5144 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:26.495254 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2505) took 1290 ms. Trace:
I20260709 12:19:26.495684 32042 rpcz_store.cc:276] 0709 12:19:25.204572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.611417 (+406845us) service_pool.cc:224] Handling call
0709 12:19:25.611841 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.612346 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.612388 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.616482 (+  4094us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.494491 (+878009us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b5400484f20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1466b8 00007f228c1cf92b 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c
0709 12:19:26.494593 (+   102us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:26.494648 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.494757 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.494808 (+    51us) raft_consensus.cc:1685] finished
0709 12:19:26.494823 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.495064 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":73505664,"thread_start_us":354020,"threads_started":1,"wal-append.queue_time_us":352930}
I20260709 12:19:26.497611 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:26.498479 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.500803 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:26.501709 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.502476  4532 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:26.503746 32120 raft_consensus.cc:2410] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 12:19:26.504992 32121 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:26.473276  4956 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:19:26.506531 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:26.508414 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102845us
I20260709 12:19:26.508854  5075 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:26.508932 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.509370 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.517856 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.518625 31965 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:26.519660 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.520247  4984 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.521118 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.521865 31965 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 12: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:26.502485  5086 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:26.522763 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.523264  5086 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.524094 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.524765 31969 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:26.525617 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.525867 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.526633 31970 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:26.527496 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.528075 31970 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:26.528996 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.529320 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.529657 31970 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:26.493033 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2561) took 2378 ms. Trace:
I20260709 12:19:26.509896 32039 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
W20260709 12:19:26.530725  5177 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.205s	user 0.002s	sys 0.000s
I20260709 12:19:26.497676 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:26.536854 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.531355 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.537324 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:26.531320 32113 rpcz_store.cc:276] 0709 12:19:24.114609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.375502 (+260893us) service_pool.cc:224] Handling call
0709 12:19:24.375827 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.031968 (+656141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.492594 (+1460626us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400149bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:26.492853 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333582208}
I20260709 12:19:26.530759 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.542125 32040 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
W20260709 12:19:26.543565 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46145.3us
I20260709 12:19:26.550763 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.551174 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:26.553195 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.521087  4984 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.556102 31964 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:26.555308 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.526093  5086 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:26.558214 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.537063  5084 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:26.561882 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 143899us
I20260709 12:19:26.538735  5144 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.557309 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.563601 31966 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 15 index: 16. (index mismatch)
I20260709 12:19:26.564316 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:26.554919 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.560029 31969 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:26.554448 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.569610 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:26.569892 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.571429 32039 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:26.571862 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62651.2us
I20260709 12:19:26.573405 32042 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:26.573844 32118 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:26.578989 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.582168 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.586736 32117 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.587435 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:26.591261 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 243894us
I20260709 12:19:26.595577 32121 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:19:26.567873  5091 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:19:26.591825 32112 raft_consensus.cc:2376] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17. Re-sending same reply.
I20260709 12:19:26.581835  5147 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:26.599560 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45210.5us
I20260709 12:19:26.600450 32118 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:26.587913  5084 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.600656  5182 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.609494  5182 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:26.609928  5144 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:26.610030  5062 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:26.610445  5182 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.611824  5182 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.615000 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:26.616350  5054 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.603276 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:26.619236 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2509) took 1334 ms. Trace:
I20260709 12:19:26.619632 31968 rpcz_store.cc:276] 0709 12:19:25.285268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.420631 (+135363us) service_pool.cc:224] Handling call
0709 12:19:25.551077 (+130446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.245061 (+693984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.618928 (+373867us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400079ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:26.619159 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550952832}
W20260709 12:19:26.621887  5176 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.346s	user 0.002s	sys 0.000s
I20260709 12:19:26.623801 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2496) took 1452 ms. Trace:
I20260709 12:19:26.624189 32047 rpcz_store.cc:276] 0709 12:19:25.171480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.171594 (+   114us) service_pool.cc:224] Handling call
0709 12:19:25.984974 (+813380us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:19:26.216964 (+231990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.217378 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:26.217730 (+   352us) log.cc:844] Serialized 14 byte log entry
0709 12:19:26.352144 (+134414us) log.cc:844] Serialized 14 byte log entry
0709 12:19:26.353354 (+  1210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:26.353410 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:26.353451 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.353513 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.442441 (+ 88928us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b48000fe358. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:26.528708 (+ 86267us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b180035ecc0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:26.623491 (+ 94783us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400033ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:26.623598 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22991,"prepare.queue_time_us":6709,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":1524,"raft.queue_time_us":5901,"raft.run_cpu_time_us":2379,"raft.run_wall_time_us":93760,"spinlock_wait_cycles":570279040,"thread_start_us":131023,"threads_started":1,"wal-append.queue_time_us":178357}
I20260709 12:19:26.627117 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2586) took 1002 ms. Trace:
I20260709 12:19:26.627530 32116 rpcz_store.cc:276] 0709 12:19:25.625212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.625484 (+   272us) service_pool.cc:224] Handling call
0709 12:19:26.626931 (+1001447us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54002fb054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:26.627062 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1281171200}
I20260709 12:19:26.584575  5166 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.630820  5166 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.631358 32660 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:26.633049 32117 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:26.633602 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:26.603323 32119 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:26.573422  5183 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.232s
I20260709 12:19:26.603446  5189 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:26.638552 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.639485 32044 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:26.604637  5084 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:26.616357 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 65636.1us
I20260709 12:19:26.604236  5079 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.641652  5183 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:26.607007 32658 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:26.644122  5186 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:19:26.643900  5013 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:26.601524 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:26.668144 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68868.9us
W20260709 12:19:26.668424 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 98987us
I20260709 12:19:26.631263  5166 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:26.673674 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 72221.8us
I20260709 12:19:26.642083  5183 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:26.608891  5078 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:26.680294 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:26.683194 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 676) took 1479 ms. Trace:
I20260709 12:19:26.685070  4888 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.689785  4888 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.690662 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 823) took 2400 ms. Trace:
I20260709 12:19:26.691114 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.691481  5183 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.697253  4984 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:26.705055  5046 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.669098  5054 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:19:26.707357 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.708838 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:26.711004 32660 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:26.686127 32041 rpcz_store.cc:276] 0709 12:19:25.204191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.598171 (+393980us) service_pool.cc:224] Handling call
0709 12:19:25.612713 (+ 14542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.837861 (+225148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.660522 (+822661us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54000334d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:26.683102 (+ 22580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512704384}
I20260709 12:19:26.691061 32115 rpcz_store.cc:276] 0709 12:19:24.290282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.694500 (+404218us) service_pool.cc:224] Handling call
0709 12:19:24.694822 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.038423 (+343601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.690185 (+1651762us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b540010ead4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:26.690458 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773640064}
I20260709 12:19:26.682575 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.677886  4532 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:26.679078  5086 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:26.680842  5084 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:26.720444 31963 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:26.683797  5078 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:26.728426 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51817.6us
I20260709 12:19:26.735689 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:26.715498  5046 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:26.678630  4927 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:19:26.680488  4919 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:19:26.715559  4532 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:26.715764  4777 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.717824  5086 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:26.749981 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61452.7us
I20260709 12:19:26.737004  5084 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:26.738890  5190 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.742364  5046 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:26.768169 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:26.773733 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 82723.3us
I20260709 12:19:26.748050  5086 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.785002 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:26.780519  4984 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.792991  4984 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.794404  5127 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:19:26.795588  4984 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.758842  5054 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:26.798921 32654 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
W20260709 12:19:26.800028 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84343.8us
I20260709 12:19:26.801947  4532 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.802260 32658 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:26.803048  4532 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:26.804352 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:26.806609 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:26.808395  5078 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.809723  5078 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.806627 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:26.814406 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:26.815127 32658 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:26.740823  5075 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:19:26.820853  5084 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:26.826113 32657 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:19:26.826864  4961 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:26.828219 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:26.829125 32651 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:26.833755  5191 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:19:26.835639  5187 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:26.836211  5187 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:26.837463  5187 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.839490  5187 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.840255  5082 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:26.739205  5189 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:19:26.847290  5147 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:26.779940  4956 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:26.765321  5190 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.783473 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:19:26.863397 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 332320us
W20260709 12:19:26.864861 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.858357 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:26.874413 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 72957.8us
I20260709 12:19:26.874426 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:26.784296  5046 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:26.877912 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 115104us
I20260709 12:19:26.873462  5190 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:26.878386 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 113062us
I20260709 12:19:26.879022  5075 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:26.850211  5147 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:26.893280 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:26.895599 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 186069us
I20260709 12:19:26.900210 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:26.906826 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:26.909626 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:26.906440 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:26.887126  5183 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.882397  4956 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:26.888984  5013 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:26.911741 32657 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:26.890242  5147 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:26.900467  5075 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:26.931569 32660 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 12.
I20260709 12:19:26.915429  4579 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:26.937743 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:26.943434 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:26.947408 32660 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:26.947710 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 99978us
I20260709 12:19:26.944701 32045 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:26.958418  5053 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.959008  5053 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:26.959425  5053 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.961874  5073 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:26.940762  4927 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:19:26.962965  5200 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:26.963598  5147 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.16, Last appended by leader: 16, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.965044  5147 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.968165 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:26.969708  5075 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:26.970896 32658 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:26.939476  4579 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:26.982599 32651 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:26.981091 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:26.985062 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:26.990392 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101792us
W20260709 12:19:26.991864  5198 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:26.978193  4929 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.985880  4956 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.003952  4929 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:27.045780 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.046911 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:26.982978  4579 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:27.052342 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 262682us
W20260709 12:19:27.052428 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 114405us
I20260709 12:19:27.060336 32115 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
W20260709 12:19:27.061774 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80759.3us
I20260709 12:19:27.062247 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.062544 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:27.067004 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.050618  4961 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.085860 31964 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:27.071143  5082 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:27.093273 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.113346  5025 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:27.114116 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.114042  5025 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.115214 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.116354  5025 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:27.116806 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.117600  4984 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:27.091893  5082 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.119011  4993 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:27.120117  4993 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:27.120338  4888 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:27.120784  4888 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:27.118340  4777 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:27.123553 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76733.6us
I20260709 12:19:27.124166 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:27.130962 31963 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:27.126827  5001 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:27.130448  5205 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:27.135414 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.141811 32660 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:27.143575  5205 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:27.144635  5205 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.148761  4961 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:27.149389  4961 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:27.150426  4961 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.121940  5086 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.119036  5082 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.155623 32651 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:27.157150 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:27.162434  5212 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:19:27.162957 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.163290 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:27.155830 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101814us
I20260709 12:19:27.164829 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.166215 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.167564 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.168843 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.170243 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.170627 32658 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:27.171770 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.172391 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.172986 32660 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:27.173866 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.128018  5189 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:27.123132  4532 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:19:27.190857 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.191174  4532 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:27.191932  4532 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:27.192068 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.192682 31972 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:27.193549 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.193150  4532 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.194160 31963 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:27.196000 32118 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:27.196280 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:27.197139 32043 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:27.199626 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.200359 31963 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:27.201062 31971 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:27.200378 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.202692 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:27.202831 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.204166 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:27.204463 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.205526 31970 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:27.207274 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:27.208158 32112 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:27.209224 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:27.210526  5219 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:27.191289 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:27.211797 32040 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:27.210997 32121 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:27.212831 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:27.212880  5222 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:27.213537 32113 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:27.214727 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:27.216563 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46657us
I20260709 12:19:27.218885 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:27.188273 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:27.224711 31964 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.221338 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:27.135725  5001 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.242583 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.246037 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:27.249212 32118 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:27.250173 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:27.253899 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:27.249281 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:27.258718 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:27.259285 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:27.261068  4532 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.261143 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:27.261861 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.262743 32112 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:19:27.262524  5211 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:27.245865  5001 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:27.265041 31968 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:27.265156 32121 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:27.265539 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.267419 32118 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:27.138952  5053 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.266148 31970 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:27.161340  5166 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.278710 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:27.279981 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.280407 32041 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:27.281443 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:27.283170 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:27.283949 32046 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:27.278167 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.285025 32043 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:27.286432 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.287132 31965 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:27.281348 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:27.290688 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.291755  5134 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:27.291972 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:27.293562  5209 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:27.293989  5209 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:27.295136 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.295698 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.296267  5190 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.297439  5053 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:27.298367  4919 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:27.298794  4919 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:27.299302 31963 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:27.301800  5215 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:27.302474  5215 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:27.302969  5215 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:27.306727 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.308830  5084 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:27.309285  5084 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:27.245568  5082 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.310163  5084 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.311246  5084 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:27.312618  5082 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:27.315801 32115 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:27.267831  5054 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:19:27.333704 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 112516us
I20260709 12:19:27.409586 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:27.334545 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53253.3us
I20260709 12:19:27.410848 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.427802 32658 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:27.428540  5216 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.429576 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.430733 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:27.431695  5013 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.432440  5013 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:27.433590  5013 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:27.288014  5211 consensus_queue.cc:1243] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:27.317222  5190 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:27.456400 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:27.435143  5013 consensus_queue.cc:1243] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:27.457659  5205 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.408607  4894 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.472295 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:27.468423 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:27.475647 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.286406  5189 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.442804  5053 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:27.328790  5222 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.143s
I20260709 12:19:27.442703  5217 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.495880 31970 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:27.497658  4993 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:27.498514 32121 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:27.500183  5209 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.488976 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:27.505616 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:27.523677 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.315304  5046 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.545789 32038 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:27.549901 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.550771 32660 raft_consensus.cc:2376] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12. Re-sending same reply.
W20260709 12:19:27.551333 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 285208us
I20260709 12:19:27.554543 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:27.556797 32047 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:27.485033  5053 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.557217 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.488610  5025 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:27.572660 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.502341  5222 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:27.570484  5217 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.563555 32657 raft_consensus.cc:2376] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16. Re-sending same reply.
I20260709 12:19:27.579010 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.581596 32660 raft_consensus.cc:2376] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17. Re-sending same reply.
I20260709 12:19:27.587174 32047 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:27.564215  5053 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:27.503595  4894 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.502562  5205 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.594554 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.577399  5222 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:27.598670 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 167353us
I20260709 12:19:27.564893  5054 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:19:27.586253  5091 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.624619 32655 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
W20260709 12:19:27.652374 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.647064 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:27.659891 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:27.669143 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.592229  5086 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:27.711872 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 101553us
I20260709 12:19:27.712865 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:27.707381 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:27.646970 32085 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.548s (SENT)
I20260709 12:19:27.593357  5025 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:27.739550 32651 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:27.742986 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.743393 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:27.744243 32658 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:27.746058  5224 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:19:27.746773  5237 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:27.772934  5237 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.773459  5237 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.748162 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:27.779291 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 132339us
W20260709 12:19:27.780396 32085 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 1.636s (SENT) [suppressed 18 similar messages]
I20260709 12:19:27.781188  5013 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.585457  5217 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.785797  5217 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:27.789219 32085 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:27.790024  5217 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.791989  5013 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:27.793581 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:27.646466  5191 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:27.741134  4579 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.675s	user 0.005s	sys 0.005s
I20260709 12:19:27.710062  4777 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:27.810530 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:27.811362 32657 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:27.733116  4956 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:27.770910  5147 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.805s	user 0.008s	sys 0.004s
I20260709 12:19:27.703063  5091 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:27.831445 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.834492 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 253487us
I20260709 12:19:27.845341 32115 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:19:27.819722  5013 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:27.846856 32115 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:27.848070 32115 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:27.849216  4919 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.850620  4919 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:27.850914  5228 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
W20260709 12:19:27.851910  5227 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:19:27.855309  5237 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.746803  4993 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:19:27.796054  5086 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:27.884572 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 72165.2us
I20260709 12:19:27.816241  5166 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:27.813043  4777 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:27.826068  5165 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:27.894068 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:27.911754 31964 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.001s
I20260709 12:19:27.812352  5054 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:19:27.819293  4579 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.912438 31964 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
W20260709 12:19:27.892151  5001 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.001s
W20260709 12:19:27.916288 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 212796us
I20260709 12:19:27.890235  4777 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.832821  4956 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:27.816814  5191 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:27.913652  5234 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:19:27.828440  5222 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.932214 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:27.919124  5189 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:27.895759  5165 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.928460  5237 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:27.893687  5086 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:27.951316 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 195457us
W20260709 12:19:27.952723 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 218669us
I20260709 12:19:27.923264  5001 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.947149  5166 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.951372  5222 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.916723  5025 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:27.951728  5147 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.982892 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:27.963438  4894 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:27.979666  5046 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:27.988309  5235 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
W20260709 12:19:28.009195 32046 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.678s	user 0.005s	sys 0.001s
I20260709 12:19:28.009549 32046 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:27.986117  5237 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.963766  5211 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:19:27.993515  5205 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.016822 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.017928  5046 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:28.020255  4894 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:28.036561 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:28.020691  5236 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:19:28.028648 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 70418.6us
W20260709 12:19:28.029745  5231 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:19:28.030754  5240 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:19:28.002960  5245 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:28.031522  5001 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:28.031765  5241 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:19:28.002749  5219 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.046784  5219 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:28.047230  5219 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.047724  5245 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.001143  5147 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.034652 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:27.986526  5166 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.041765 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:27.985692  5243 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:19:28.035660  5211 consensus_queue.cc:1243] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:28.032739  5091 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.032548  5189 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:28.067144 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 79433.1us
I20260709 12:19:28.040320  5205 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.065752 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.068868 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.019866  5237 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.078293  5075 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:28.067731  5250 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:28.092190 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55707.7us
I20260709 12:19:28.074651  5046 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:28.074971  5091 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:28.077880  5001 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:28.097504 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:28.098166  5127 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:28.098567  5127 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:28.076153  5211 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:19:28.095783  4956 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.096311  5046 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:28.081171  5189 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.121652  5086 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.123652  4919 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.125900  5244 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:28.126485  5244 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.129132  5209 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:28.105922  5053 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:28.132345  5210 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.168s	user 0.010s	sys 0.013s
I20260709 12:19:28.137039  5232 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:28.137665  5215 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.834s	user 0.006s	sys 0.005s
I20260709 12:19:28.138564  4894 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:28.138947  5249 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:19:28.139155  4894 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.141666  5251 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:28.142416  5254 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:28.142925  4894 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.144869  5237 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:28.145426  5046 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.146270 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.146792  4777 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.147114 32657 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:28.148887  4777 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:28.132417  5053 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.103752  4579 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.150936 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2594) took 1292 ms. Trace:
I20260709 12:19:28.152019  4888 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.151321 32117 rpcz_store.cc:276] 0709 12:19:26.858523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.858649 (+   126us) service_pool.cc:224] Handling call
0709 12:19:26.858984 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:26.874348 (+ 15364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.874377 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:26.877800 (+  3423us) log.cc:844] Serialized 26 byte log entry
0709 12:19:27.127642 (+249842us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b1800370180. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1
0709 12:19:27.127784 (+   142us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:27.127812 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.127863 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.131504 (+1003641us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:28.150701 (+ 19197us) raft_consensus.cc:1685] finished
0709 12:19:28.150730 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.150879 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4315,"raft.queue_time_us":13569,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":7795,"spinlock_wait_cycles":18769152,"thread_start_us":27712,"threads_started":1,"wal-append.queue_time_us":86274}
I20260709 12:19:28.118621  5246 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:28.154111  5246 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:28.155126  5246 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.160660  5134 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:28.161271  5209 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:28.161178  5134 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.109702  5165 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.099375  5127 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.110879  4984 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:28.190328  5207 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 1.225s	user 0.007s	sys 0.007s
I20260709 12:19:28.137620  5232 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:28.153453  4888 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:28.190718  4956 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.190779  4984 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:28.191357  4956 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.191979  4984 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.192529  5189 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.193450  4956 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.194849  5247 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.195391  5247 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.196964  4956 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:28.197477  4956 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.119205  5166 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:28.198236  5084 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.887s	user 0.012s	sys 0.003s
I20260709 12:19:28.184185  5134 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.081588  5205 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.102079  5091 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.104398  5191 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:28.114992  4532 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.853s	user 0.011s	sys 0.000s
I20260709 12:19:28.183913  5147 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.186517 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.106736  5250 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.185097  4961 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:28.187897 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 148610us
I20260709 12:19:28.183315  5209 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.288617  5250 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.190114  5127 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.289764  5209 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.285168  5222 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:19:28.291142  5222 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:28.291486  5222 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:28.292361  5222 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.293543  5222 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:28.197589  5247 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.142995  5254 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.295591  5245 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.202638  5244 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.218034 32121 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:28.223125 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.222069 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:28.257519 32117 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:28.260743 32114 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:28.220530  5237 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:28.264562  5205 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.264846  5192 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:28.265410  5191 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.266455  5229 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:19:28.268460  4956 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.275156  5046 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.261893  5219 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:28.285491 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 99024.9us
I20260709 12:19:28.286012 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:28.281723  5232 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.285374  4961 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:28.299525 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 224512us
I20260709 12:19:28.296991  5254 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.335210 32116 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:28.357998 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.360049 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:28.360428 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.352147 32115 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:28.362027 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.362752 32115 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:28.363785 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.352197 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:28.187237  5215 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.368570  5243 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:28.369025  5243 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:28.369415  5243 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.371415 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.372745 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.382565  4777 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:28.383327  4777 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:28.384686  4777 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.360663 32116 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.339151 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:28.386687 32117 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:28.390529  5215 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.374078 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:28.392204  5252 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:28.392707  5252 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:28.393198  5252 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.410722  4777 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:28.417404 32116 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:28.313912  5053 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:28.421310 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 135397us
I20260709 12:19:28.422818  5191 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.431182  5191 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.431504  5091 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.336971  5147 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.439273 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:28.433578  5192 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:28.338094  5237 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:28.435237 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81708.2us
I20260709 12:19:28.336052  5244 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.443382 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.471072 32116 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:28.336629  5226 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:28.444301 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.450924 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.452405 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:28.455827 31967 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:28.476348 32657 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:28.477437 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:28.478298 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.478732 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.479092 32660 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:28.479348 32655 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
I20260709 12:19:28.480048 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.481086  5147 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.478206  5226 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.483366  5250 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.483467  5147 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.483973  5250 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.484649  5226 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.485743  5219 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.486912 32046 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:28.443557  5053 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.488943  5053 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:28.493639 32117 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:28.495625 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.494253 32044 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:28.455446  5252 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.457090 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:28.469573 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:28.507392 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49379.1us
I20260709 12:19:28.507539  5075 raft_consensus.cc:2764] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:19:28.507818  5256 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.508153 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:28.508332  5256 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:28.508816  5256 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:28.508900 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 51898.3us
I20260709 12:19:28.510558  4929 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:28.510496  5254 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:28.511231  4929 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:28.512099  4929 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.513119  5166 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:28.514525  5232 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.510042 32013 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:28.514544  5127 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:28.515480  5165 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:28.516003  5165 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:28.509995 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:28.517040  5165 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.518512  5062 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:28.519039 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:28.518955  4961 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.520001  4956 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:28.520414  4961 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:28.520444  4956 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.521085  4956 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.521731 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.522353 32658 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:28.525197 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.525799 32655 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:28.527145 32658 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 12:19:28.528879 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.529630 32657 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:28.515858  5127 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:28.531500  5127 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:28.533269 31936 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:28.534464 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.536067 31966 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:28.443686  5189 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:28.458287  5209 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:28.450048  5183 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:28.474452  5144 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:28.467031  5191 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:28.553210 31936 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:28.553850 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.573113 32117 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:28.555049 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:28.556108 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:28.512157  5254 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:28.575109 31967 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:28.559139 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:28.558954 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:28.561537 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:28.576406 32047 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.576537 32046 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.563230 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:28.578841 32657 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:28.565917 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:28.579883 32039 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:28.567595 31969 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: [14.15-15.16]
I20260709 12:19:28.548115  5091 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.548044  5166 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.587286 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:28.588730 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.556586  5247 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:28.590911  5046 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:28.550041  5219 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:19:28.498371  5244 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.609586 31970 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.604740 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.558893  5215 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:28.606475 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 50474.3us
W20260709 12:19:28.611228 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.611631 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:28.566028  5086 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:28.568310  5144 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:28.574905  5254 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:28.624578 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:28.628934 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:28.612916 32117 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:28.582546  5084 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.607187 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2588) took 1361 ms. Trace:
I20260709 12:19:28.623708 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:28.624004 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:28.592221  5247 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.630908 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:28.647987 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.638177 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:28.566871  5134 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:19:28.629942  5254 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.625689  5211 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.653160 32042 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.652066 31972 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.636615 32046 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:28.609259  4956 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.656225 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:28.656471  4956 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.656854 32046 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:28.657392  4956 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:28.657828  4956 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.637406  5086 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.629813  5166 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.624294 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:28.660022 31971 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.638087 32113 rpcz_store.cc:276] 0709 12:19:27.245962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.246793 (+   831us) service_pool.cc:224] Handling call
0709 12:19:27.247124 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.257405 (+ 10281us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:28.280963 (+1023558us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b1800119800. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf
0709 12:19:28.281172 (+   209us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.606772 (+325600us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:28.606829 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:28.606872 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.606950 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.607117 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":201392,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":2557,"spinlock_wait_cycles":98391296,"thread_start_us":569719,"threads_started":2}
I20260709 12:19:28.658195  4956 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.573259  5183 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.609052  4532 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:28.663303 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 263389us
I20260709 12:19:28.560685  5189 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:28.621029 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 552147us
I20260709 12:19:28.651616 31968 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.677446 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.600334  5250 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.680742 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.620836  5258 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:28.686698 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:28.688483 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:28.592687  5091 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.648463  5046 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.587751  5127 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.663972  5247 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:28.752683 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 199604us
W20260709 12:19:28.772806 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 95368.6us
W20260709 12:19:28.774467 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:28.802719 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 114296us
I20260709 12:19:28.728720 32038 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.627626  5215 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.760757 32041 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.621783  5191 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.724800  5086 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.836642 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.727619  5166 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:28.753818  5251 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:28.724009  5258 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.725666  5250 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:28.663326  5084 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:28.835875  4888 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.623s	user 0.005s	sys 0.005s
I20260709 12:19:28.856387 32113 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:28.895911 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.898736 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.898530 32085 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:28.901757 32657 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:28.903472 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.904139  4929 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:28.898662  5053 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.907279  5053 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.909073 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:28.909816 32658 raft_consensus.cc:2376] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15. Re-sending same reply.
I20260709 12:19:28.905725 32113 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:28.911010 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:28.911675 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.912599 32113 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:28.913791 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.917773 32113 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:28.919028 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:28.919602 32651 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:28.919005 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.921319 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:28.927345 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.928623 31969 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.851202  5165 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:28.858839  4579 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.855963  5086 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:28.933903 32655 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:28.835434  5046 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:28.947367 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 101050us
I20260709 12:19:28.858422  5258 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.871107  5260 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:28.937145 32118 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:28.927976  5231 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:19:28.966259 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 329692us
I20260709 12:19:28.930815  5255 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:28.979143 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:28.911007  5053 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:28.933683  5165 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.990578 32117 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:28.990686 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:28.974030 31972 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:28.992158 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:28.993687 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2591) took 1746 ms. Trace:
I20260709 12:19:28.927232  5237 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.012553 32118 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:28.949467  5252 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:28.867795  5250 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:28.839905  5144 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.023399 32118 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:28.995450  5127 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.013094 32120 rpcz_store.cc:276] 0709 12:19:27.246840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.246981 (+   141us) service_pool.cc:224] Handling call
0709 12:19:27.257966 (+ 10985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.268275 (+ 10309us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:27.574293 (+306018us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.863675 (+1289382us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.991223 (+127548us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:28.991352 (+   129us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:28.991453 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.991547 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.993586 (+  2039us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1134896,"prepare.queue_time_us":2671714,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":99506,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":24716,"spinlock_wait_cycles":22264064,"thread_start_us":475547,"threads_started":3,"wal-append.queue_time_us":336784}
I20260709 12:19:29.032891  5242 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:29.035665 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:29.036355 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.009750  5073 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:29.024500  5053 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:29.021838 32042 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:28.964699  5086 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:28.964936  4579 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.018352  5165 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.043201 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.034132  5260 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.057215 31969 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:29.057790  5281 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:29.041769  5209 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:29.058385  4956 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.058663  5025 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.058772  5286 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:29.060284  5265 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
W20260709 12:19:29.061087  5264 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 12:19:29.061267 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.031553  4888 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.034716  5251 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.116225 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.123751 32657 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:29.124799 31972 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 12.
W20260709 12:19:29.118974 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.010336  5255 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.126349 31971 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:29.108732  5281 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:29.129066 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58810.9us
I20260709 12:19:29.137840 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:29.120893 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.008s	sys 0.020s
I20260709 12:19:29.098531  4929 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:29.140198 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.009s	sys 0.020s
W20260709 12:19:29.141099 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 780605us
I20260709 12:19:29.139012  4929 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.147704 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.135435 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:29.151814 31967 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.115126  5209 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.154629 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:29.155732 32039 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.000s
I20260709 12:19:29.136618  5144 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.132212  5286 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.048590  5242 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.157509 32039 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
W20260709 12:19:29.159042 31970 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.000s
I20260709 12:19:29.158885 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:29.157371 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.018s	sys 0.024s
I20260709 12:19:29.139756  5281 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:29.161996 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.018s	sys 0.025s
I20260709 12:19:29.151414  5252 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:29.165928 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 554327us
I20260709 12:19:29.150930  5091 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.133152  4888 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.109638  5260 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.120954  5025 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.154770  5215 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.188292  5261 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:29.188916  5266 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 12:19:29.190960 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.192227 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.169070 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:29.177546 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.198180 31971 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:29.181018 31970 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:29.181416 32038 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:29.176798 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.200160 32045 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
W20260709 12:19:29.182555  5279 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:19:29.183524 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48160.3us
I20260709 12:19:29.186233  4888 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.163210  5053 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:29.181430 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:29.202664 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:29.202986 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.177883  5191 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.205492 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.257681 31970 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:29.257771 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.218081 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.165266  5084 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.259153 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.259166 31966 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:29.259637 31971 raft_consensus.cc:2376] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16. Re-sending same reply.
I20260709 12:19:29.175061  5250 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.263052 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.203615 32114 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:29.234179 31968 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 12:19:29.265280 31968 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
W20260709 12:19:29.236274  5263 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
I20260709 12:19:29.269284  5255 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.237391  5001 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:29.198446  5252 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.238361  5283 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:29.238812  5075 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:29.281796  5075 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:29.282833  5075 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:29.240127  5288 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:19:29.242614 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.286405 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:29.286296  5283 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.287250  5229 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.287645  5229 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.287978  5229 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.288686  5283 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.291847 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.292490 32042 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:29.292582 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:19:29.279858  5254 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
I20260709 12:19:29.294559  4993 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:29.294693  5254 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.295657  5166 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.296170  5166 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.296778  4993 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:29.242700  5226 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:19:29.243592  4961 consensus_meta.cc:287] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.002s
I20260709 12:19:29.246732  5211 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:29.246678  5286 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.248220  4579 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.188928  5261 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.201759  5189 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.279274 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.251077 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:29.251354 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:29.253600  5284 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:19:29.253381 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:29.254410  5268 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:19:29.255173  5272 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
I20260709 12:19:29.255776 32113 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:29.254431 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.171288  5209 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:29.173604  5275 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:19:29.200523  5091 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:29.261683 32047 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.669s	user 0.003s	sys 0.007s
I20260709 12:19:29.256496 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.172128  5086 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.267249 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.424841  5127 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.430644 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.432390 32654 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:29.432926 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.207854  5251 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.425696 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.433211 31967 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:29.433758 32654 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 12:19:29.433877 31969 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:29.434866 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.435021 32654 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:29.435884 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.238045  5270 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:29.436482  5270 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:29.436945 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.437541 31969 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:29.424535 32653 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:29.270668 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.438593 31970 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:29.429070  5285 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:19:29.439381 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:29.305125  5295 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:19:29.440591 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.440753 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.441411  4888 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.270182  5053 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.273892  5001 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:29.274711  5073 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:29.276201 32043 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.277439 32044 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.279876 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.178172  4894 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:19:29.168212  5134 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:19:29.430460  5255 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.429620  5287 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.425663 32660 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:29.429294  5250 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:19:29.432683  5127 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.441957 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:29.324883  5243 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.955s	user 0.011s	sys 0.000s
I20260709 12:19:29.428479  5246 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.296795  5254 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.330998 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:29.308681 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.442823 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.471741 32039 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:29.458870  4777 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:29.457731  5082 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.451359  5211 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:29.297318  5229 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.443789 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:29.442234 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.370882 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:29.517127 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73419.2us
I20260709 12:19:29.517179 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.516263  5082 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.454582 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2619) took 1009 ms. Trace:
I20260709 12:19:29.308431 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.455541 32045 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:29.453336 32116 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:29.453375 32121 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:29.446029 32117 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:29.520843 32113 rpcz_store.cc:276] 0709 12:19:28.445510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.921458 (+475948us) service_pool.cc:224] Handling call
0709 12:19:29.454501 (+533043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:29.315853 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:29.270578  5222 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.969s	user 0.015s	sys 0.005s
I20260709 12:19:29.310894 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:29.305801 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.374802 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:29.306514 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:29.427059  5256 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.000s
I20260709 12:19:29.463394 32119 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:29.462863 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.442927 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.464735 31970 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:29.305958 32041 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:29.534739 32660 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:29.531056 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 156374us
I20260709 12:19:29.532665 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:29.538458 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:29.385367  5247 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.542569 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.542790 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:29.461406  5255 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:29.280797  5232 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.766s	user 0.002s	sys 0.010s
I20260709 12:19:29.271286 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:29.424873  4532 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.512372  4777 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.298240  5166 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.316831  4956 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.471164 31964 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:19:29.357116 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:29.561506 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 204501us
I20260709 12:19:29.457343  5285 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:29.302824  5046 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.456193  5209 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.456578  5073 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.442058  4888 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.453786  5226 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:29.303544  5183 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.312844  5242 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:29.444049 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 192740us
I20260709 12:19:29.475979  5246 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.467458  5287 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.520612 32651 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20.
I20260709 12:19:29.304445 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.517830 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:29.299045  5252 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.526594  5262 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:19:29.535692 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.543401 31966 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:29.525934  5205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.014s	sys 0.017s
W20260709 12:19:29.549108 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 281044us
I20260709 12:19:29.549786 32657 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:29.550509 32112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.012s	sys 0.025s
W20260709 12:19:29.608093 32112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.013s	sys 0.025s
I20260709 12:19:29.608914 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2593) took 2361 ms. Trace:
I20260709 12:19:29.551282 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.548836 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:29.528667 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.610853 31971 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:29.611069 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62336.5us
I20260709 12:19:29.611718 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.612258 31971 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:29.550027 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.580511  5244 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.555450 31972 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:19:29.454586 32047 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:29.564606 32045 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:29.532814 32652 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:29.298122  4993 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:29.569600  5054 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:29.548813  5086 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:29.588930  5222 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:29.572844 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.575985  5258 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.000s
I20260709 12:19:29.520591  4929 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.527534  5290 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:29.562536  4532 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.469861  5165 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.512956  4961 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:19:29.590072 31968 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:29.316270  5025 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.586841  5300 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 12:19:29.473992  5243 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.592833 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:29.598242 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80482.6us
I20260709 12:19:29.562335  5283 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.314016  5297 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.199s
I20260709 12:19:29.603049 32116 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:29.666911 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.553208 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.667519 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:29.586288  5285 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:29.618775 32038 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:29.622911 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.532541  5254 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.592173  5257 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.626608 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 699) took 1080 ms. Trace:
I20260709 12:19:29.614918  5013 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.594815  5219 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.686571  5219 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.687482 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2365) took 1151 ms. Trace:
I20260709 12:19:29.688002 32041 rpcz_store.cc:276] 0709 12:19:28.535762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.535877 (+   115us) service_pool.cc:224] Handling call
0709 12:19:29.687232 (+1151355us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001a4fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:29.687405 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424898944}
I20260709 12:19:29.689605 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.674150  5285 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.694060 32046 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:29.696687 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 701) took 1109 ms. Trace:
I20260709 12:19:29.697576 32117 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:19:29.547358  5256 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.600665  5046 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:29.700548  5256 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:29.701494 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.639051 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.703225 31966 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.705199  5084 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:29.705730  5084 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:29.604666  5205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.014s	sys 0.017s
I20260709 12:19:29.642251 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.560196  4984 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:19:29.643517  4579 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.711009  5229 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.711156  4579 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.713235 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.713573  4579 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.609336 32112 rpcz_store.cc:276] 0709 12:19:27.247370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.263576 (+ 16206us) service_pool.cc:224] Handling call
0709 12:19:27.263859 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.264067 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:27.282565 (+ 18498us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.097527 (+814962us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18003468a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:28.764619 (+667092us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b18003468a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:19:28.808345 (+ 43726us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.608475 (+800130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:29.608556 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:29.608613 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.608709 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.608848 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1184813,"prepare.queue_time_us":1578842,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":95646,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":483,"spinlock_wait_cycles":14340864,"thread_start_us":1082215,"threads_started":3,"wal-append.queue_time_us":605054}
I20260709 12:19:29.621711  5183 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.716459 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.718003 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2598) took 1419 ms. Trace:
W20260709 12:19:29.653767 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54663.3us
I20260709 12:19:29.586061  5299 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:19:29.595494  5209 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.586385 32121 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:29.635349  5222 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:29.624164  5086 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:29.660386 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67639.8us
I20260709 12:19:29.647388  5290 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.666569 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:29.667809 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.689265  5219 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.720274 32116 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:29.721704  5290 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.666613 32040 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:29.724432 32040 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:29.725594 32040 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:29.726727  5046 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:29.726864 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60381.8us
I20260709 12:19:29.727311  5046 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.727679 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:29.728240  5046 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.730535  5222 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.612617  5053 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.731956 31969 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:29.644161  5258 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:29.672030 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 125973us
I20260709 12:19:29.675689 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.820009 32651 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:29.676609 32047 rpcz_store.cc:276] 0709 12:19:28.537389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.537526 (+   137us) service_pool.cc:224] Handling call
0709 12:19:29.617642 (+1080116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:29.820842 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.674790 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.679977 32115 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:29.681546 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:29.666600  5283 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:29.682562  4532 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.682614  5075 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.683316 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.827059 31971 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:29.596451  5001 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.833576 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:29.833943 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.019s	sys 0.027s
I20260709 12:19:29.834395 31971 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:29.835181  5303 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:19:29.835363 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.836098 31971 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:29.837998 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2621) took 1305 ms. Trace:
I20260709 12:19:29.855611 32115 rpcz_store.cc:276] 0709 12:19:28.532617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.034449 (+501832us) service_pool.cc:224] Handling call
0709 12:19:29.065930 (+ 31481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:29.836454 (+770524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.837753 (+  1299us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540012c0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:29.837938 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1404027136}
I20260709 12:19:29.856966 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.857542 32115 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:29.859419 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:29.860006 32044 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.865448  5013 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:29.865918  5013 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.866518  5165 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.867542  5255 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.868191 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.871019 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.872179 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.872838 31969 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:29.873523 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.874758  5247 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.875576  5127 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.876449  5084 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.877521  5025 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:29.877938  5025 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:29.878993  5246 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.878854  5025 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.681573  5257 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.880082  5025 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.882638 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:29.734894  5226 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.887202  5226 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.657173  5281 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.855084  5211 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.890190  5211 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:29.736768 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.671259 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:29.894709 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2519) took 1442 ms. Trace:
I20260709 12:19:29.896848  5082 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.897528 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.898269 31969 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:29.666407  5250 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.899039 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.737157 32119 rpcz_store.cc:276] 0709 12:19:28.298335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.298772 (+   437us) service_pool.cc:224] Handling call
0709 12:19:28.299048 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:29.533026 (+1233978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.717797 (+184771us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400139554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:29.717948 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677011712}
I20260709 12:19:29.903040 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.697132 32046 rpcz_store.cc:276] 0709 12:19:28.587752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.657664 (+ 69912us) service_pool.cc:224] Handling call
0709 12:19:29.693803 (+1036139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:29.694597 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.696444 (+  1847us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001a4fd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:29.696615 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371466112}
I20260709 12:19:29.908689 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.909454 32046 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19.
I20260709 12:19:29.733744  5287 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.805810  5086 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:29.810230 32121 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:29.818699 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:29.742811 32041 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:29.912434 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:29.913259 32113 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:29.913327 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:29.822595 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.915479  5243 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:29.915967 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.834453 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.920064 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.825517 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.920979 32045 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:19:29.650926  4777 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.620311  4993 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:29.830907 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:29.936594 32651 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:29.831198  4888 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.834395  5283 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.840042 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.810954  5251 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.843664  5222 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:29.834040  5290 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:29.847982 32042 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:19:29.849531  5258 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:29.818776  5189 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:29.852481  5260 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:29.813047  5263 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:19:29.855067 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:29.737730 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:29.808748 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.004257 31971 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:29.921754 32044 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:29.894161 32039 raft_consensus.cc:2368] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:29.922168 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:29.910210 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:29.925957 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.019s	sys 0.027s
I20260709 12:19:29.929467 32045 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22. Ops: [20.15-21.16]
W20260709 12:19:30.007654 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 846810us
I20260709 12:19:29.881963  5257 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:29.931311 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 112734us
I20260709 12:19:29.959810 31964 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:30.014688 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2533) took 1174 ms. Trace:
I20260709 12:19:29.843686 32652 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:29.960528 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:29.966349 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.575s (SENT)
I20260709 12:19:29.968551  5205 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:29.988283 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.083214 31967 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:29.988948 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:29.989539 32660 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:29.989651 32115 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:30.084483 32655 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:29.878008  5084 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.895097 31965 rpcz_store.cc:276] 0709 12:19:28.452443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.452592 (+   149us) service_pool.cc:224] Handling call
0709 12:19:28.452892 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:28.489245 (+ 36353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.489278 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:28.489935 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:19:29.281519 (+791584us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b180031ab00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:19:29.642859 (+361340us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:29.818812 (+175953us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.819045 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.819120 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.894385 (+ 75265us) raft_consensus.cc:1685] finished
0709 12:19:29.894497 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.894652 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1140,"prepare.queue_time_us":55657,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":29416,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7585,"spinlock_wait_cycles":5490560,"thread_start_us":517630,"threads_started":2,"wal-append.queue_time_us":1176258}
I20260709 12:19:29.875609  5192 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:30.085407 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.003372 32038 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:29.889971  5281 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:29.968969  5251 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:30.010699 32658 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:30.016130 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:29.972818 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2623) took 1439 ms. Trace:
I20260709 12:19:30.014696 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.019194  5222 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.093125  5192 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:30.109371  5192 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.109934  5054 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:30.110635  5192 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.110692  5054 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.112977  5258 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:30.113655  5054 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.015174 31964 rpcz_store.cc:276] 0709 12:19:28.840742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.839774 (+999032us) service_pool.cc:224] Handling call
0709 12:19:30.014610 (+174836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.121366 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2350) took 1102 ms. Trace:
I20260709 12:19:30.020323 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:30.020659  5290 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:29.999755 32656 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:30.122895 32112 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:30.021204 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 283534us
W20260709 12:19:30.124356 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 158069us
I20260709 12:19:30.124750 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:30.125346 32623 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.631s (SENT)
I20260709 12:19:30.125739 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.021745 32114 raft_consensus.cc:2368] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15.
I20260709 12:19:30.022065  5299 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.024296 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.026719 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:30.046485 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.135097 32044 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.064254 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2374) took 1306 ms. Trace:
I20260709 12:19:30.136060 32042 rpcz_store.cc:276] 0709 12:19:28.758112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.294145 (+536033us) service_pool.cc:224] Handling call
0709 12:19:29.340849 (+ 46704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.972922 (+632073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.063940 (+ 91018us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b540003a054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:30.064184 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165742976}
I20260709 12:19:30.008862 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:30.081156 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:29.989835 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.918236  5073 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.141058 32046 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:30.085807 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 330338us
I20260709 12:19:30.009902 31968 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:19:30.141587 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 132790us
I20260709 12:19:30.142463 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2342) took 1548 ms. Trace:
I20260709 12:19:30.142899 31968 rpcz_store.cc:276] 0709 12:19:28.593672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.907150 (+1313478us) service_pool.cc:224] Handling call
0709 12:19:30.009443 (+102293us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:19:30.009614 (+   171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.142106 (+132492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.142394 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40274176}
I20260709 12:19:30.088575 32011 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.099571 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.144522 32038 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:30.144832 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56378.1us
I20260709 12:19:30.145628 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:30.101783 32120 rpcz_store.cc:276] 0709 12:19:28.533259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.038609 (+505350us) service_pool.cc:224] Handling call
0709 12:19:29.062441 (+ 23832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.185716 (+123275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.185773 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.186365 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.555999 (+369634us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:29.556057 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.932745 (+376688us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:29.932819 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.932885 (+    66us) raft_consensus.cc:1685] finished
0709 12:19:29.932905 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.972756 (+ 39851us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1657,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":282618624,"thread_start_us":369265,"threads_started":1,"wal-append.queue_time_us":438779}
I20260709 12:19:30.104852  5287 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:30.105690  4993 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.113576  5258 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.116037 31965 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:30.117775 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2535) took 1150 ms. Trace:
W20260709 12:19:30.128391  5144 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.966s	user 0.010s	sys 0.000s
I20260709 12:19:30.100242  4888 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:30.121807 31964 rpcz_store.cc:276] 0709 12:19:29.018993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.120412 (+1101419us) service_pool.cc:224] Handling call
0709 12:19:30.120764 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.120968 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.121006 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:30.121062 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:30.121115 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.121183 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.121298 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.178658 31963 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:30.127281  5251 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:30.019434 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.126086 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.127624 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.127707 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:30.132645 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 118009us
W20260709 12:19:30.137776 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56755.1us
I20260709 12:19:30.193679 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.148872 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:30.151825 32038 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:30.014940  5310 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:30.153390 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:29.827567  5270 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.118912  5209 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.169677 32011 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20.
I20260709 12:19:30.210271 32011 leader_election.cc:403] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:30.173759 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2528) took 1567 ms. Trace:
I20260709 12:19:30.082774  5205 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:30.211215  5205 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.211021 31965 rpcz_store.cc:276] 0709 12:19:28.606568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.098904 (+1492336us) service_pool.cc:224] Handling call
0709 12:19:30.115728 (+ 16824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.166773 (+ 51045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.173698 (+  6925us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.212791 31965 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:30.213481 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 680) took 1066 ms. Trace:
W20260709 12:19:30.214671  5304 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:19:30.216034 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2624) took 1682 ms. Trace:
I20260709 12:19:30.216400 32118 rpcz_store.cc:276] 0709 12:19:28.533911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.042011 (+508100us) service_pool.cc:224] Handling call
0709 12:19:29.042374 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.112748 (+ 70374us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:29.136790 (+ 24042us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.598727 (+461937us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.599943 (+  1216us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.919824 (+319881us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:29.929989 (+ 10165us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.930184 (+   195us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:29.930222 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.930289 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.215829 (+285540us) raft_consensus.cc:1685] finished
0709 12:19:30.215852 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.215975 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50880,"prepare.queue_time_us":19511,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":10538,"raft.queue_time_us":72959,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11291,"spinlock_wait_cycles":9669504,"thread_start_us":437709,"threads_started":1,"wal-append.queue_time_us":512580}
I20260709 12:19:30.220048  5247 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:30.221537 31938 leader_election.cc:400] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15.
I20260709 12:19:30.221932 31938 leader_election.cc:403] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:30.222108 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2536) took 1198 ms (client timeout 1503 ms). Trace:
I20260709 12:19:30.201116  4888 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:30.222591 31963 rpcz_store.cc:269] 0709 12:19:29.023773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.060923 (+ 37150us) service_pool.cc:224] Handling call
0709 12:19:30.221917 (+1160994us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:30.222026 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2429744512}
I20260709 12:19:30.223935 31963 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:30.224488 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2353) took 1020 ms. Trace:
I20260709 12:19:30.224651  4888 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:30.224810 31963 rpcz_store.cc:276] 0709 12:19:29.203921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.223487 (+1019566us) service_pool.cc:224] Handling call
0709 12:19:30.223792 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.224282 (+   490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.224438 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.213843 31965 rpcz_store.cc:276] 0709 12:19:29.146874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.212216 (+1065342us) service_pool.cc:224] Handling call
0709 12:19:30.212565 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.213216 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.213408 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:30.225440 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76693.3us
I20260709 12:19:30.226351  4532 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:30.226483 31965 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:30.226930  4532 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:30.227806  4532 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.167066 31967 rpcz_store.cc:276] 0709 12:19:28.967782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.116819 (+1149037us) service_pool.cc:224] Handling call
0709 12:19:30.117178 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.117378 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.117425 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:30.117481 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:30.117527 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.117594 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.117710 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.228575  5261 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.229902 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.166181  5091 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.230701  5091 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.231068  5091 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.231295  5209 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.232123 32656 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:30.175582 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.151556  5254 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:30.022969  5313 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:30.184355 31971 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:30.246457 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.246999 31971 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:30.247982 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.248572 31971 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:30.250730 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:30.252867 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.253667 32114 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:30.253729 32039 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:30.254206 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.231245  5001 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.259212 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.131847  5222 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.203449 32044 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:30.204960  5314 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.205753 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.141417  5281 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:30.140695  5299 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:30.201967 32045 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:30.288445 32119 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:30.249331  5144 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:30.169358  5046 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.152472  5257 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:30.179729 31964 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 12:19:30.293804 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 679) took 1147 ms. Trace:
I20260709 12:19:30.294214 31964 rpcz_store.cc:276] 0709 12:19:29.146515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.179022 (+1032507us) service_pool.cc:224] Handling call
0709 12:19:30.179376 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:30.293446 (+114070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.293738 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.226408 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:30.234985 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 107466us
I20260709 12:19:30.150869  5300 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.238351 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.239384 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.300863 31964 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:19:30.301167 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74841.3us
I20260709 12:19:30.301891 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.240648 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:30.234458 32043 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:30.242028 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.238720 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:30.244887 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.304526 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:30.304904 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.150859  5165 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:19:30.186127  5287 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:30.256369 31971 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:30.196481 31966 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:30.309794 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2349) took 1297 ms. Trace:
I20260709 12:19:30.258567 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.310171 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.311456 31968 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:30.290786  5299 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.176496  5082 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:19:30.247484  4888 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.241533  5254 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:30.291756  5086 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.306998 32113 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:30.312566 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.313275 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.280566 32120 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:30.310127 31966 rpcz_store.cc:276] 0709 12:19:29.012744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.616499 (+603755us) service_pool.cc:224] Handling call
0709 12:19:30.309739 (+693240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:30.299690  5257 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.251121  5286 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:19:30.243453  5310 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.318812 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.208571  5246 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:19:30.344983 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.415403  5219 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:30.415498 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70612.2us
W20260709 12:19:30.415884  5211 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.001s
I20260709 12:19:30.416467 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.417333  5219 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.347707  5189 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:30.418918  5189 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20.
W20260709 12:19:30.350525  5269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.026s	sys 0.077s
W20260709 12:19:30.420056  5269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.026s	sys 0.078s
I20260709 12:19:30.352349  5315 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:30.354038  5287 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.355222  5165 raft_consensus.cc:2764] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:30.359589  5258 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:30.302465  5046 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.360883  4993 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.423867  5046 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:30.350265  4888 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.311902  5252 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:30.362504  5257 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.362528  4984 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.343878 32118 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:30.350477  5285 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.003s
I20260709 12:19:30.434361 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2603) took 1202 ms. Trace:
W20260709 12:19:30.396325  5226 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.002s
W20260709 12:19:30.398677 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.299789  5314 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.400388 31970 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:30.400831  5299 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.407402  5270 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:30.409632  5251 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:30.414464  5183 consensus_meta.cc:287] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.000s
I20260709 12:19:30.417616  5283 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.343395  5254 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:30.420786 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.432848  5252 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:30.417987 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.429520 32115 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:30.365732 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:30.434935 32112 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:30.437744  4888 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:30.395860  5250 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.439616 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 672) took 1295 ms. Trace:
I20260709 12:19:30.471915 31968 rpcz_store.cc:276] 0709 12:19:29.144360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.176915 (+1032555us) service_pool.cc:224] Handling call
0709 12:19:30.311233 (+134318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.311905 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.439413 (+127508us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b5400314f50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:30.439544 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178010496}
W20260709 12:19:30.441803 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.444249 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:30.399626  5281 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:30.451566 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 829) took 1146 ms. Trace:
I20260709 12:19:30.431290  5211 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.458060 31963 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:30.427525 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:30.458891 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.437822  5285 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:30.489404 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61957.3us
I20260709 12:19:30.462025 31966 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:19:30.495204 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:30.496035 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 186474us
I20260709 12:19:30.443404 32116 rpcz_store.cc:276] 0709 12:19:29.231905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.721014 (+489109us) service_pool.cc:224] Handling call
0709 12:19:29.721334 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.721701 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.721731 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.722239 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.921483 (+199244us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:30.430904 (+509421us) log.cc:844] Serialized 14 byte log entry
0709 12:19:30.433947 (+  3043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.434030 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.434078 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:30.434099 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.434265 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7970,"prepare.queue_time_us":72480,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":16028,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":3176,"thread_start_us":656460,"threads_started":2,"wal-append.queue_time_us":348357}
W20260709 12:19:30.469688 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51777us
I20260709 12:19:30.473366 32117 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:30.491920 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.478461 32658 rpcz_store.cc:276] 0709 12:19:29.303252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.304187 (+   935us) service_pool.cc:224] Handling call
0709 12:19:30.449011 (+1144824us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:30.449124 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949495168}
I20260709 12:19:30.460182  5183 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:30.467846  5252 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.521677 32047 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:30.529031 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:30.467224  5046 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.523212 32046 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:30.525204 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:30.526641 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 380598us
I20260709 12:19:30.478008  5226 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:30.533094 32038 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:30.533190 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:30.534271 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:30.538816 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.542881 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.514230  5144 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:30.513931  5073 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:30.581312 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.582121 32045 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
W20260709 12:19:30.549609 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.568317 32041 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:30.583359 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.520823  5054 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.580832 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.587258 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:30.587802 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 461741us
I20260709 12:19:30.588603 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.581823 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.590960 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.587937 32040 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:30.596204  5073 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:30.596792  5073 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:30.597142  5269 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:30.518348  5211 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.578569 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:30.581044 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:30.576483 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:30.612613 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.616122 32039 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:30.591740 32043 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:30.543010  5192 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:30.615052 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.619576 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:30.617940 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2532) took 1010 ms. Trace:
W20260709 12:19:30.584409  5013 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.691s	user 0.016s	sys 0.000s
I20260709 12:19:30.620481 32041 rpcz_store.cc:276] 0709 12:19:29.607323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.914295 (+306972us) service_pool.cc:224] Handling call
0709 12:19:29.914594 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.583861 (+669267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.617694 (+ 33833us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54001a59d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:30.617886 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1501035392}
I20260709 12:19:30.531805  5260 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:19:30.624605 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.629451 32042 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:19:30.602474  5183 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:30.626081 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.593663  5054 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.616981 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:30.626631 32040 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:30.542797  5046 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:30.633191 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.633278 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:30.629973 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:30.618882  5192 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:30.668500 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:30.636245  5326 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:30.637351  5314 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:30.659188 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:30.663285 32041 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:30.664599 31963 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.673501 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:30.636003 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 45994.8us
I20260709 12:19:30.632532  4888 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:30.627822  5250 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:30.670009 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:30.650465 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 456819us
I20260709 12:19:30.644104  5054 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.663316  5281 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:30.670939  5192 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.597860  5073 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.689373 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:30.698402 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:30.681713 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.632668  5183 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:30.689186 32114 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:19:30.699415  5183 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.699082  4532 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.696604 31935 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19.
I20260709 12:19:30.700627 31935 leader_election.cc:403] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:30.680083 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.684664 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:30.701653 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.687269  5258 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.703864 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.706012 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:30.668675  5247 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:19:30.686820  5250 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.629469  5013 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.691157  5281 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.677675  4888 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:30.701797  5321 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:19:30.692508  5216 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:30.718672 32657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.021s	sys 0.020s
W20260709 12:19:30.723296 32657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.021s	sys 0.020s
W20260709 12:19:30.722108 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.703316  5189 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:30.726416 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.708431  5251 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:30.742048 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.806082 31972 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:30.719425  5084 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:30.710438  5250 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.709903  5025 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.721712  5281 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.749107 31963 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:30.700922  5144 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:19:30.746953 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2562) took 1209 ms. Trace:
I20260709 12:19:30.707742  5127 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:30.711925  5086 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.822402 31965 rpcz_store.cc:276] 0709 12:19:29.537136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.227334 (+690198us) service_pool.cc:224] Handling call
0709 12:19:30.227637 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:30.258436 (+ 30799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.258467 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:30.285800 (+ 27333us) log.cc:844] Serialized 26 byte log entry
0709 12:19:30.687952 (+402152us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:30.688525 (+   573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.688601 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.746674 (+ 58073us) raft_consensus.cc:1685] finished
0709 12:19:30.746702 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.746882 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13604,"raft.queue_time_us":13702,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":26559,"thread_start_us":341573,"threads_started":1,"wal-append.queue_time_us":431937}
I20260709 12:19:30.741406  5315 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:30.807142  5324 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:19:30.833698 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:30.836135 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.709442  4532 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.744210  5189 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.726390  5216 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:30.741087  5310 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.843015 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:30.804731  5330 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.807664  5285 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.807071  4929 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:30.846053 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.816323  5165 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:30.738514  5299 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.727456  5332 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.861765 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.816078  5222 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.714082  5258 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.815834  5283 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:30.841634 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:30.862703 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 159640us
I20260709 12:19:30.848030  5287 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:30.845428  5251 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:30.852216  5216 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.853811  5086 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.874907 32010 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.850764  5189 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.882923 31972 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:19:30.893041 31972 pending_rounds.cc:85] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 6
I20260709 12:19:30.842702  5331 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:19:30.890357 32040 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:30.891403 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:30.892103 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 186193us
I20260709 12:19:30.864719  5211 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:30.871678  4961 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:30.872339  5165 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:30.842885  4888 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.863850  4929 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.864152  5332 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:30.889091 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 185408us
I20260709 12:19:30.898408 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.893275 31972 pending_rounds.cc:107] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:19:30.918502 31972 raft_consensus.cc:2889] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:30.901566  5328 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:30.884450  5335 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.887180 31963 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:19:30.908728 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.893997 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:30.895661  5330 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:30.871847  5243 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:19:30.873437  5222 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.924999 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:30.917678 31935 leader_election.cc:400] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
W20260709 12:19:30.896987  5334 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:19:30.885571  5260 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:19:30.925684 31935 leader_election.cc:403] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:30.925932 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:30.906405  5209 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 12:19:30.906298  5283 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:30.934556  5165 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:30.935698  4929 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:30.937242  5219 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.519s	user 0.000s	sys 0.010s
I20260709 12:19:30.937438  5257 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.938441  5001 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:30.939567  5205 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.728s	user 0.008s	sys 0.001s
I20260709 12:19:30.927946  4888 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:30.946719  5329 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:19:30.900207  4961 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.925866 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:30.975239  4929 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:30.920162  5332 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:30.975132 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.975459 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2345) took 2306 ms. Trace:
I20260709 12:19:30.978904  5252 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:30.979871  5327 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:19:31.053231  5252 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:31.043999 31969 rpcz_store.cc:276] 0709 12:19:28.669156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.094209 (+1425053us) service_pool.cc:224] Handling call
0709 12:19:30.094583 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:30.094925 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.094951 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:30.119172 (+ 24221us) log.cc:844] Serialized 52 byte log entry
0709 12:19:30.534743 (+415571us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:30.534798 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.534887 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.975231 (+440344us) raft_consensus.cc:1685] finished
0709 12:19:30.975253 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.975404 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9835,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":415100,"threads_started":1,"wal-append.queue_time_us":594407}
I20260709 12:19:30.980039  5086 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:30.927606  5285 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:30.931321  5341 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:31.066452  4532 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.066536  5341 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.068044  5341 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:31.069141  5254 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.001s
I20260709 12:19:30.989398 31963 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:31.028462  5258 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.069608  5254 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.028702  5219 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.919569  5226 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.071887  5226 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:30.949615  5209 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15.
I20260709 12:19:30.891335  5251 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.079742 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:31.080338  5251 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.080436 32652 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:30.949965  5344 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:31.008956  4993 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.002s
I20260709 12:19:31.001261  5330 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:31.081786  5344 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.013813  5299 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:31.031991  5257 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.032410  5001 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.082531  5299 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:31.082930  5344 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.083599  5299 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.084376  5001 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.033149  5205 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.033890 32656 raft_consensus.cc:2368] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
W20260709 12:19:31.042624  5091 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.004s
I20260709 12:19:31.017939  5054 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:31.037061 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 111249us
I20260709 12:19:31.087860  5054 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.088023 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:31.088471 32046 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:31.088649 31964 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:31.088784 32656 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:30.969189 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51663.9us
I20260709 12:19:31.078768 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.035938  4961 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.090829 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.107537 32039 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:31.108229 31971 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:31.010080  5046 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:31.046696  5286 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:31.049191 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2563) took 1434 ms. Trace:
W20260709 12:19:31.049389 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 123526us
I20260709 12:19:30.949348  5348 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:31.081856  5330 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:31.122570 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 197276us
I20260709 12:19:31.084872  5299 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.042724 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 952) took 1101 ms. Trace:
I20260709 12:19:31.091171 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:31.094775 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:31.100891 32121 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:31.126744 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.128283 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.129159 32112 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:31.108331 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.133104 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.107787 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:31.116964 31967 rpcz_store.cc:276] 0709 12:19:29.614289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.228868 (+614579us) service_pool.cc:224] Handling call
0709 12:19:30.229191 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:30.249452 (+ 20261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.249481 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:30.285793 (+ 36312us) log.cc:844] Serialized 26 byte log entry
0709 12:19:30.695371 (+409578us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:30.695435 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.695537 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:31.048865 (+353328us) raft_consensus.cc:1685] finished
0709 12:19:31.048899 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:31.049110 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19892,"raft.queue_time_us":20402,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":26546,"thread_start_us":407325,"threads_started":1,"wal-append.queue_time_us":620418}
I20260709 12:19:31.135540 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.139428 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.141026 32112 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:19:31.142215 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.142539 32041 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:19:31.142894 31968 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:31.143502 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:31.145332 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:31.145875 32044 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:31.148715 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.150877 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:31.150959 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:31.151850 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.077672  5216 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.085693  5091 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.176707 32117 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:31.153053 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.153693 32116 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:31.158186 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.160730 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.164490 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.179548 32112 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:31.155951 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:31.183496 32115 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 14: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:31.172895 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.118129 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.123646 32657 rpcz_store.cc:276] 0709 12:19:29.935024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.967785 (+ 32761us) service_pool.cc:224] Handling call
0709 12:19:30.000319 (+ 32534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:30.011345 (+ 11026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.011375 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:30.087609 (+ 76234us) log.cc:844] Serialized 26 byte log entry
0709 12:19:30.723836 (+636227us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:30.723876 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.723954 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:31.036158 (+312204us) raft_consensus.cc:1685] finished
0709 12:19:31.036180 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:31.036305 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10651,"raft.queue_time_us":10753,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":3763,"thread_start_us":537477,"threads_started":1,"wal-append.queue_time_us":587516}
I20260709 12:19:31.130728  5054 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.187043 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.153776  5209 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:31.078727  5192 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.195261  5269 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:31.196169  5192 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.196534  5335 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:31.196919 32660 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:31.197827 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 79813.8us
I20260709 12:19:31.191442 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.175387  4929 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.158511  5046 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:31.194072  5082 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.143540  5330 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:31.158702  5252 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.054859  5281 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.201391 32047 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:31.168682  5211 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.175658  5243 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.177006  5025 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19.
W20260709 12:19:31.210826 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.182657  5222 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.220726 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.190768  5351 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.147428  5144 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:19:31.185248  5310 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.211984 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.215636 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.232326 31968 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.186018  5073 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.217358 32112 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:31.143615  5258 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:19:31.210531  5209 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.238857 32657 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:19:31.239259 32657 pending_rounds.cc:85] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 12
I20260709 12:19:31.239470 32657 pending_rounds.cc:107] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:19:31.239686 32657 raft_consensus.cc:2889] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:31.188266  5345 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:19:31.188519  5286 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:19:31.207391  5046 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.190529  5260 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:19:31.292104  5260 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:31.292980 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.294457  5287 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:31.211477  5350 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:31.295558  5350 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:31.295998  5350 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.296379 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.292193 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:31.297860  4993 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:31.298408  4993 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.299218  4993 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.300916  4984 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:31.301503  4984 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.301770 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:31.302619 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.303073 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.303702  4984 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:31.304005 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.304561 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.304814  5192 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:31.305290  5192 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.305671  5192 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:31.307515  5091 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.243204  5209 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.308475  5209 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.309517  5001 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:31.309991  5001 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.310359  5001 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:31.313299  5345 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:31.313728  5345 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.314584  5345 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.198830  5261 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:19:31.315893  5345 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:31.317126  5205 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.221421  5243 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:31.318686  5243 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.320538  5255 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:31.321043  5255 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:31.321995  5255 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.323427  5255 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.244899 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.203042  5127 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:19:31.324985 32112 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:31.230888  5183 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
W20260709 12:19:31.331985 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2351) took 2303 ms (client timeout 2699 ms). Trace:
W20260709 12:19:31.255892 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43961.7us
W20260709 12:19:31.333073 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 160226us
I20260709 12:19:31.333438 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.330221 32119 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:31.275717 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.235711  5073 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.277918  5247 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:31.279855  5319 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.993s	user 0.003s	sys 0.015s
W20260709 12:19:31.283926  5333 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.389s	user 0.000s	sys 0.001s
I20260709 12:19:31.285207  5281 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.243912  5222 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.241163  5226 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.290534  5082 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:31.326372  5046 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:31.186911  5189 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:31.327047 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.330832 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.218290  5346 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.334079 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.338023 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.341714 31971 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:19:31.413888 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 79919.6us
W20260709 12:19:31.332383 31970 rpcz_store.cc:269] 0709 12:19:29.028733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.867952 (+839219us) service_pool.cc:224] Handling call
0709 12:19:31.331790 (+1463838us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b5400315454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:31.331929 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221839232}
I20260709 12:19:31.207643  5332 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.241279  5348 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.413889 31969 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:31.351351 31968 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:31.354429 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.438469 31965 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:31.439735 31966 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:19:31.355892 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.236274  5351 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.240758  5252 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:31.325452  5283 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.081s
W20260709 12:19:31.359311 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 158505us
I20260709 12:19:31.294884  5211 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.457824  5211 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.460182  5211 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.364792 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1049) took 1428 ms. Trace:
I20260709 12:19:31.345146  5286 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 12:19:31.462029  5286 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:31.460908 32653 rpcz_store.cc:276] 0709 12:19:29.936188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.936325 (+   137us) service_pool.cc:224] Handling call
0709 12:19:29.967561 (+ 31236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.967925 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.967959 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.988611 (+ 20652us) log.cc:844] Serialized 26 byte log entry
0709 12:19:30.210975 (+222364us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b18002c5840. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:19:30.624925 (+413950us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:30.624965 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.625045 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:31.283361 (+658316us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:31.364511 (+ 81150us) raft_consensus.cc:1685] finished
0709 12:19:31.364543 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:31.364717 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5073,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":26745,"spinlock_wait_cycles":96011264,"thread_start_us":413776,"threads_started":1,"wal-append.queue_time_us":806392}
I20260709 12:19:31.462446  5286 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.454278  5332 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.463423  5286 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.464972  5286 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:31.332914  5260 consensus_queue.cc:1175] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f05e4741d35c4cf3a31358c56e65b30c" responder_term: 19 status { last_received { term: 19 index: 9 } last_committed_idx: 7 last_received_current_leader { term: 19 index: 9 } }
I20260709 12:19:31.468626  5189 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.438752  4984 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:19:31.469874 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.474283  5216 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.407835  5222 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:31.479761  5342 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.608s	user 0.000s	sys 0.002s
I20260709 12:19:31.480906  5222 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.481297  5084 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:31.481698  5084 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:31.482553  5084 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.484006  5084 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.442438  5243 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.492780 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2622) took 1176 ms. Trace:
I20260709 12:19:31.493294 32113 rpcz_store.cc:276] 0709 12:19:30.316186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.358888 (+ 42702us) service_pool.cc:224] Handling call
0709 12:19:30.359283 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:30.433827 (+ 74544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.433876 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:31.026710 (+592834us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:31.031427 (+  4717us) log.cc:844] Serialized 26 byte log entry
0709 12:19:31.144317 (+112890us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:31.144357 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:31.492460 (+348103us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b54001482d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:31.492516 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:31.492562 (+    46us) raft_consensus.cc:1685] finished
0709 12:19:31.492578 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:31.492719 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5177,"raft.queue_time_us":95,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3994,"spinlock_wait_cycles":182554112,"thread_start_us":112497,"threads_started":1,"wal-append.queue_time_us":223779}
I20260709 12:19:31.498922  5209 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.383816  5054 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:31.501348  5209 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.391121  4929 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.345252  5025 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.502947  5025 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:31.503432  5025 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.340616  5269 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.507797  5269 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.508805 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:31.345000  5073 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.514542  5283 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:31.514683  5332 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:31.515620  5332 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:31.440976 31967 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:31.516170  5283 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:31.516626  5283 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:31.345465  5247 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.517338  5247 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.230650  5250 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.517681  5283 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.379168  5350 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.520150  5250 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.452178  5127 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.521035  5127 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.522145  5350 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.523458  5127 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.434240  5358 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.454228  5351 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.454702  5252 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.525194  5330 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.349197  5183 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.527873  5183 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.443097  5144 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:31.421002 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87638.3us
I20260709 12:19:31.496673  5001 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:31.497418 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.533640 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.445130  5346 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.535080  5001 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.536089  5346 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.536320 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.536828 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.537482 32117 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:31.379585  5290 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:31.434465  5331 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.518997  5283 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.538455 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.559798 32119 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.561422 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:31.499554  5338 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.613s	user 0.000s	sys 0.002s
I20260709 12:19:31.501394  5054 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:31.545454 32660 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:19:31.571579 32660 pending_rounds.cc:85] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 15
I20260709 12:19:31.571889 32660 pending_rounds.cc:107] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:19:31.536230 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:31.572170 32660 raft_consensus.cc:2889] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:31.577844 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.556267 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 105655us
I20260709 12:19:31.570246 32655 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:19:31.583464 31972 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:31.586889 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.587500 31963 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:31.550987  5252 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:31.590185 32653 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:31.591343 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.597397  5073 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.588238 32119 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:31.516173  5332 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:31.612841 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:31.614806 31968 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:31.552556  5290 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.615846 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.615770 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.618806 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.627298 31972 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:31.628110 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.633363 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.586582  5189 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:31.636119 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:31.637494 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.644090 32120 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:31.645026 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.569128  5086 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:19:31.619010  5209 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.612105  5287 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.618602  5053 raft_consensus.cc:2764] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:31.573216  5192 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.590051  4984 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:31.662353 31972 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:19:31.568861  5331 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.632395  5252 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.636118  4993 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:31.672562 32117 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:31.694693 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:31.697420 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:31.698575 32047 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19.
I20260709 12:19:31.665465 31968 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:31.612063  5341 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:19:31.611891  4961 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.712456 32118 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:31.711931 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.687462 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:31.714489 31968 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:31.690630 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:31.715137 32046 raft_consensus.cc:2376] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18. Re-sending same reply.
I20260709 12:19:31.715302 32040 raft_consensus.cc:2376] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13. Re-sending same reply.
I20260709 12:19:31.691951 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.714126  4961 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:31.708681 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:31.716423 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.717032 32044 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.718959 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.720229 31968 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:31.707655 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 258014us
I20260709 12:19:31.689862 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.724160 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.708453 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:31.726522 32041 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:31.696081  5332 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.706739  5290 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.728695  5290 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.729524  5350 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:31.705688  5344 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:19:31.731302  5281 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:31.731753  5331 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.731956  5281 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:31.733107  5144 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.732957  5281 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.733397 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:31.681529  5054 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.734117 32038 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
W20260709 12:19:31.731495  5367 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:19:31.736387 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.685351  5250 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:31.736989 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.742056 32042 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:31.686246  5189 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:31.759909 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:31.765309 32041 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
W20260709 12:19:31.755607 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63744.4us
I20260709 12:19:31.693424  4993 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.678730  5209 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.772020 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.756125  5358 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.688647  5252 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:31.764582 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.730099  5350 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.767454 31972 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:31.769235 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:31.783679 32038 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:31.756286  5001 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.740284 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.775807 32044 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:31.776666 31968 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:31.796365 32112 raft_consensus.cc:2368] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:31.796833 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.797629 31968 raft_consensus.cc:2410] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:19:31.690807  4984 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.779074 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.784227 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.794932 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.801419 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:31.801600 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.803057 32623 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19.
I20260709 12:19:31.803376 32623 leader_election.cc:403] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:19:31.809577 32117 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:31.809890  5073 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:31.809821 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.810586 31968 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:31.810940  5363 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:19:31.812986  5205 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:31.813562  5205 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:31.814477  5205 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.816735  5209 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.817840 31968 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:31.817943  5258 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.818602  5258 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.818923 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.819589 31968 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.774729  4993 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.825414 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.822819 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:31.823531 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.852881 32116 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.824532 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.760587  5250 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.773492  5344 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:31.763783  5046 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:31.859789 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.773216  5362 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.854015 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:31.850878 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47955.2us
I20260709 12:19:31.857590 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.786856  5252 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.775022  4961 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:31.862249 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.855468  5358 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:31.916076 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:31.861085  5329 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:31.821202  5165 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:31.820946  5258 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.850173  5226 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:31.861783  5255 consensus_meta.cc:287] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.538s	user 0.005s	sys 0.007s
I20260709 12:19:31.856192  5222 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:31.929138 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:31.918107 31968 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:31.918165 32117 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:31.915673  5254 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:31.984615 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:31.978417 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.982079 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:31.988175 32115 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:31.934950 31972 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:31.938133 32116 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:31.858878  5344 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:31.940366 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.012043 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:31.972571 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.922269  5378 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:19:31.923141  5209 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.827558  5001 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.800329  5348 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.981688 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.917486  5330 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:32.024277 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:31.982446 31967 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:31.992022 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.038842 31972 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:31.995326  5377 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:19:32.039443 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:32.039680 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:32.003291  5250 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:31.919584  5362 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:32.047966 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.003754  4984 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.052893 32039 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:19:32.005971  5286 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.006s
I20260709 12:19:31.929473  4961 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:31.941084  5358 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:32.012646 31964 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:32.067728 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:32.016352 31968 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:31.977931  5165 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:32.025316 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.076188 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 861) took 1619 ms. Trace:
I20260709 12:19:31.827224  5287 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 12:19:31.989604 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.044536  5330 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:32.044746 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:31.938889  5189 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.010571  5192 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:32.048076 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 119011us
I20260709 12:19:31.941490  5257 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:32.050596 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.051612 32045 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:32.002075  5025 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.054044 32113 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:32.069805 31968 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:32.039740  5254 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:32.073082 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.076607 32114 rpcz_store.cc:276] 0709 12:19:30.456593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.456737 (+   144us) service_pool.cc:224] Handling call
0709 12:19:30.491628 (+ 34891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:30.699532 (+207904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.075913 (+1376381us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54002b66d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:32.076128 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453617920}
I20260709 12:19:32.052663  5362 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.023064  5209 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.096417 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:32.067049  5348 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:32.101136 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.107278 32119 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:32.115525 31972 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:32.124179 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.124881 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.127022 32114 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:32.099237  5247 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
I20260709 12:19:32.128437  5359 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:32.128657 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:32.129801  5053 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.131445 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:32.105695  5205 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:32.101126  5257 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:32.077476  5344 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.235561 32659 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:32.106837  5258 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:32.187696 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:32.178711 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:32.127125  5254 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:32.106419  5330 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:32.148304 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.258477 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 171312us
I20260709 12:19:32.179180 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:32.136773 32114 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:32.114161  5192 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:32.293814 32653 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
W20260709 12:19:32.295650 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.296607 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.177963 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:32.204463 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:19:32.299393 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 95077us
W20260709 12:19:32.206959  5283 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.002s
I20260709 12:19:32.135891 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:32.306667 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 171657us
W20260709 12:19:32.249296  5381 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:19:32.134176  5053 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.182128  5222 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19.
I20260709 12:19:32.293741 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:32.337916 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44268.8us
I20260709 12:19:32.282063  5344 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.245496  5205 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.218093  5359 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:32.188619  5350 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:32.338692 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:32.341430 32658 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:32.185555  5001 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.166525  5046 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.281679  5390 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.226709  5281 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.159400  5287 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.353580 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:32.345638 32114 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:32.295763  5391 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.145308  5250 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:32.159476  4993 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.126645  5358 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.16, Last appended by leader: 16, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.346699  4777 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.134907  5247 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.313285  5226 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:32.313956  5335 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:32.310494  5382 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:32.172288  5393 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.177s
I20260709 12:19:32.346896  5205 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.342478  4961 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:32.343225  5283 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.363246 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.312037  5192 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:32.348857  5347 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:32.314401  5380 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:19:32.346699  5025 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.355252  5387 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.360813  5222 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:32.361850  5281 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.149709  5165 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:32.384517  5388 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:19:32.385021 31966 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:32.380039  5209 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.422834  5281 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.373374  5257 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.402225  5283 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.428305  5283 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.430771  5192 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.407914  5387 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:32.386768  5386 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:19:32.387056  5393 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:32.436929  5393 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:32.437793  5393 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.439077  5393 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:32.439404  5396 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:32.411863  5315 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:32.440457  5315 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:32.402776  5347 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:32.376920  5250 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.443077  5347 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.362416  5287 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.403990 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:32.447678 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43770.3us
I20260709 12:19:32.387504  4993 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.359261  5390 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:32.414673 32121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2616) took 2168 ms (client timeout 1505 ms). Trace:
I20260709 12:19:32.385862  5335 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:32.413748  5222 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:32.444321  5347 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.416710  5013 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.379810  4777 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.384977  5389 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:32.364519  5391 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:32.450098 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.452497 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:32.455861 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:32.452919 32121 rpcz_store.cc:269] 0709 12:19:30.240003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.240435 (+   432us) service_pool.cc:224] Handling call
0709 12:19:32.408562 (+2168127us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54001491d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:32.408681 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792583552}
I20260709 12:19:32.540606 31966 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:32.466447 31968 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:32.467355 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:32.543183 31963 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:32.416494  5247 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.434535  5387 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.547467 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.548684 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.549461  5396 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.549851  5396 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.550189  5396 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.550807  5350 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:32.551591  5283 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.551868  4777 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.552064  5283 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.533526 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.414403  5399 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.421844  5205 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.467017  5001 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:32.392841  5330 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.475790  5335 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:32.562371  5257 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.390568  5254 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:32.569615 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 269383us
W20260709 12:19:32.574641 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.586663 32044 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:19:32.560184  5399 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.386843  5382 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.589720 32659 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:32.594895 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.552531  5283 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:32.565629  5384 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.469s	user 0.002s	sys 0.000s
I20260709 12:19:32.540887  5389 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.588119 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:32.605245 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.635145 32041 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:32.606611 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:32.411953  5359 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.535713  5226 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.604282  5001 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:32.642642 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:32.651890 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.652704 32041 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:32.653755 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.641654 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:32.654800 32045 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.643316 32042 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:32.647537 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:32.656219 32038 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.658851 32041 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:32.609383  5399 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.601511  5257 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.633576  5345 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.671720  5286 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:19:32.671423  4993 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:19:32.680680  5404 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.681301  5205 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.668181  5192 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:32.681260  5404 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.682184  5192 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.684593  5192 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.685684 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.686525 32119 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:32.687182  5250 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.691124  5345 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:32.670934  5252 consensus_meta.cc:287] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.735s	user 0.012s	sys 0.000s
I20260709 12:19:32.696091 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.707161 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.706271  5382 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.723179 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:32.724094 31970 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:32.725262 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.725907 31970 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.639154  5389 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:32.728780 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.732095 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:32.713361 31972 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:32.708774 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.716059 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2640) took 1577 ms. Trace:
I20260709 12:19:32.734555 32118 rpcz_store.cc:276] 0709 12:19:31.138265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.305699 (+167434us) service_pool.cc:224] Handling call
0709 12:19:31.306085 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:31.712858 (+406773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.715772 (+1002914us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54002fb054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:32.715991 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933981440}
I20260709 12:19:32.734161 31969 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.733474 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.747576 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:32.646939  5013 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 12:19:32.674090  5243 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:32.777773 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:32.626197  5222 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:32.756353  5362 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.636s	user 0.003s	sys 0.008s
I20260709 12:19:32.782980 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.792412 32652 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:19:32.754158  5395 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.158s	user 0.000s	sys 0.002s
W20260709 12:19:32.781769  5398 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.177s	user 0.004s	sys 0.004s
I20260709 12:19:32.737236  5404 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.811362 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:32.700199  5391 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.831158 32011 leader_election.cc:400] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
I20260709 12:19:32.831737 32011 leader_election.cc:403] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:32.754084  5401 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.052s	user 0.005s	sys 0.000s
I20260709 12:19:32.752913  5315 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.647075  5387 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.784274  5390 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:32.754106  5403 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 12:19:32.787147 31972 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:32.729919  5252 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:32.829185  5408 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.107s	user 0.000s	sys 0.010s
I20260709 12:19:32.755187  5251 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:19:32.858256 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.809883  5250 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.841131  5358 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15.
W20260709 12:19:32.855228  5290 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.126s	user 0.006s	sys 0.007s
I20260709 12:19:32.829845  5243 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.863620 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:32.809444  5362 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.832931  5205 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.854127 32038 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:32.902500 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.882355 31969 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:32.903357 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:32.829715  5332 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.887121 32112 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:32.850164 31970 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:32.907829 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:32.907829 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:32.905157 31972 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:32.903359 32038 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:32.909601 31970 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:32.804476  5001 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:32.917008 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2566) took 1302 ms. Trace:
I20260709 12:19:32.917563 31964 rpcz_store.cc:276] 0709 12:19:31.615088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.068628 (+453540us) service_pool.cc:224] Handling call
0709 12:19:32.068902 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:32.132071 (+ 63169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:32.132107 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:32.145631 (+ 13524us) log.cc:844] Serialized 26 byte log entry
0709 12:19:32.343350 (+197719us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:32.343393 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.916521 (+573128us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b54000f7654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:32.916586 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:32.916656 (+    70us) raft_consensus.cc:1685] finished
0709 12:19:32.916677 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.916931 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4515,"raft.queue_time_us":7758,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":8251,"spinlock_wait_cycles":133967616,"thread_start_us":193619,"threads_started":1,"wal-append.queue_time_us":395324}
I20260709 12:19:32.921438 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2381) took 1373 ms. Trace:
I20260709 12:19:32.921963 31967 rpcz_store.cc:276] 0709 12:19:31.547850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.548039 (+   189us) service_pool.cc:224] Handling call
0709 12:19:31.570833 (+ 22794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:32.038444 (+467611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.921129 (+882685us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54000799d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:32.921359 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945750784}
I20260709 12:19:32.923530 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.913132 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:32.913754 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 359340us
I20260709 12:19:32.932845 32042 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:32.941450 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 212730us
I20260709 12:19:32.840579  5335 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:32.883311  5243 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:32.963436 31964 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:32.958891 32119 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:32.959806 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:32.961155 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2629) took 1358 ms. Trace:
I20260709 12:19:32.962297 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:32.962370 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2653) took 1380 ms. Trace:
W20260709 12:19:32.920467 31938 connection.cc:256] RPC call timeout handler was delayed by -1.33066669200025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 12:19:32.967118 32043 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:19:32.969303 32114 rpcz_store.cc:276] 0709 12:19:31.602452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.312272 (+709820us) service_pool.cc:224] Handling call
0709 12:19:32.344457 (+ 32185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:32.360557 (+ 16100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.960811 (+600254us) spinlock_profiling.cc:243] Waited 946 us on lock 0x7b540010efd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:32.961072 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176384}
I20260709 12:19:32.968302 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:32.974967 32121 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:32.967054 32118 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:32.969574 32120 rpcz_store.cc:276] 0709 12:19:31.535957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.536084 (+   127us) service_pool.cc:224] Handling call
0709 12:19:32.915834 (+1379750us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54001234d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:32.916012 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243350400}
I20260709 12:19:32.978159 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:32.978870 32652 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19.
I20260709 12:19:32.981465 31963 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:32.884872  5250 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:32.984613 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.985459 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:32.986361 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:32.966368  5417 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:32.970769  5409 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.078s	user 0.005s	sys 0.003s
I20260709 12:19:32.991824 32040 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:32.965406  5413 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:32.998842 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:32.966750  5243 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:32.987484  5393 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.548s	user 0.005s	sys 0.006s
I20260709 12:19:33.002002 32115 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:32.989516 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 211873us
I20260709 12:19:33.002548 32121 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:32.911548  5332 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.016882 32045 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:33.001812 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:33.038223 32043 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 12. Current term is 13. Ops: [12.12-12.12]
W20260709 12:19:33.058540 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56812.3us
I20260709 12:19:33.059588 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:33.031826 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.023711 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:33.036345 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 93687.6us
I20260709 12:19:33.069518 32040 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:33.067217 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.070632 32047 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:33.019306  5251 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:32.992027  5283 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.073237 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.068341 32046 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:33.071659  5013 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:19:33.075747  5315 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:32.985502  5396 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.024085  5417 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.092453 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:33.104970 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.019559  5422 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.100785 32652 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:33.090885  5332 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:33.067761  5286 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:33.115700 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.101867 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.068439  5290 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:33.113484 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.117755 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:33.099967  5251 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.021452  5413 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.127939 32039 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:33.134626 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.146543 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.208585 32047 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:33.072949  5260 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:19:33.146797 32046 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:33.149703 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:33.145753 32011 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.4:39399 timed out after 2.028s (SENT) [suppressed 9 similar messages]
I20260709 12:19:33.118544  5330 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:33.095679  4961 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 12:19:33.155109  5419 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 12:19:33.155786  5424 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 12:19:33.156539  5426 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:19:33.158672  5428 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:33.160872  5283 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.161243  5252 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:33.161979  5335 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.168241  5396 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.146678  5286 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:33.312197  5286 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.312920  5428 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.176338 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 863) took 1592 ms. Trace:
I20260709 12:19:33.314477 32117 rpcz_store.cc:276] 0709 12:19:31.583707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.080394 (+496687us) service_pool.cc:224] Handling call
0709 12:19:32.080785 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:32.095469 (+ 14684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:32.095521 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:32.095591 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:32.095643 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.176042 (+1080399us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54002f9c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:33.176099 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.176257 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33104768}
I20260709 12:19:33.176719  5362 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:33.181535 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.137576  5332 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:33.182142  5250 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.152560  5290 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:33.120344 32657 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:33.199510 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 443555us
W20260709 12:19:33.195153  5399 consensus_meta.cc:287] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 12:19:33.209466 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:33.287840 32047 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:33.193308  5165 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:19:33.294446 32656 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:33.207038  5251 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:33.283732  5082 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:33.215232  5389 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.301194 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:33.208868  5413 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:33.308645  5347 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.006s
I20260709 12:19:33.290827  5330 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.215041  5084 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
W20260709 12:19:33.301990  5427 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:19:33.310422  5417 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.319015 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:33.337179 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.342746 32654 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.343875 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.344571 32117 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.344969 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:33.337486 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.340396 32113 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.328706 32651 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:19:33.320822 32656 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:33.318682 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:33.349720 32657 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.340595 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.351466 32121 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:33.319828 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:33.352212 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.352598 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.353334 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.339804 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.354295 32118 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:33.299244  5254 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.356088 32114 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:33.359076 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2576) took 1730 ms. Trace:
I20260709 12:19:33.362112 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.363493 32117 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:33.319823  5165 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:33.363761 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:33.356675 31938 leader_election.cc:400] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:33.364437 31963 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:33.342365  5429 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:33.316499  5428 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:33.339067  4777 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.330011  5389 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.364439 31938 leader_election.cc:403] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:19:33.370744 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 236174us
I20260709 12:19:33.361338 31971 rpcz_store.cc:276] 0709 12:19:31.628279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.076617 (+448338us) service_pool.cc:224] Handling call
0709 12:19:32.076952 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:32.077315 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:32.377173 (+299858us) log.cc:844] Serialized 14 byte log entry
0709 12:19:32.558036 (+180863us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:32.596804 (+ 38768us) log.cc:844] Serialized 52 byte log entry
0709 12:19:32.597122 (+   318us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:32.976370 (+379248us) log.cc:844] Serialized 14 byte log entry
0709 12:19:33.358243 (+381873us) log.cc:844] Serialized 14 byte log entry
0709 12:19:33.358792 (+   549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.358864 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:33.358902 (+    38us) raft_consensus.cc:1685] finished
0709 12:19:33.358913 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.359024 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26412,"prepare.queue_time_us":81762,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":1252,"raft.queue_time_us":234337,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":33730,"thread_start_us":1119585,"threads_started":4,"wal-append.queue_time_us":260617}
I20260709 12:19:33.337239  5330 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.357400  5315 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:33.325562  5082 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.366155  5165 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:33.372514 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:33.368964  5429 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:33.318251  5399 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.317788  5290 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:33.398609 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 252887us
I20260709 12:19:33.407328 32011 leader_election.cc:400] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19.
I20260709 12:19:33.408633 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.385834  5315 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:33.406133 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 215010us
I20260709 12:19:33.409902 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.412357 31963 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:33.398056 31964 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:33.385246  4777 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.414657  5315 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:33.425974 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 291391us
I20260709 12:19:33.412019  5330 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:33.427114 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.430107 32047 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:33.432964 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 76478us
I20260709 12:19:33.434047 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.435204 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.436700 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.437418 32043 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:33.438563  5441 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16.
I20260709 12:19:33.439282  5441 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.439638 32656 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:33.439932  5441 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:33.440682  5441 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:33.416348  5347 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:19:33.422628  5165 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.417459  5254 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:33.445317  5254 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:33.446383  5254 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.463179  5269 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.464419  5250 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:33.465237  5250 consensus_queue.cc:1243] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:33.466768  5444 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:33.467211  5444 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:33.467779 32652 pending_rounds.cc:85] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 9
I20260709 12:19:33.468106 32652 pending_rounds.cc:107] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:19:33.468387 32652 raft_consensus.cc:2889] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:33.468528  5054 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.417065  5437 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.492962 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.494395 32047 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:33.424439  5433 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:33.471310 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 401944us
I20260709 12:19:33.505445 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:33.526173 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:33.526907 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.528298  5269 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:33.528514  5082 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.528977  5269 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.529663  5243 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.531955  5243 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.468101  5444 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.536891  5399 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.539479  4777 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.541733 32657 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:33.543949 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:33.544860 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.544961 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.546092 32047 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:33.547819 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.527350 32654 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:33.549610 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 164176us
I20260709 12:19:33.549559 32651 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:33.551363 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.553126 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:33.553333 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:33.555977 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.557569 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:33.559201 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.563328  4961 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.563957  5362 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:33.568804 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.570154  5335 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:33.570744 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.570906  5335 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:33.572088  5341 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.574195 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.574963 32047 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:33.574774 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:33.561929  5315 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:33.557762 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 127349us
I20260709 12:19:33.590514 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:33.587323 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:33.591970 31971 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.593006 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.593878 32045 raft_consensus.cc:2393] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:33.595142 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:33.595767 31965 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:33.579304 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 142373us
I20260709 12:19:33.597113 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:33.597509 32047 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:19:33.597673 32045 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.600536 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:33.422518  5429 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.605773 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:33.607712 31967 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.554740  5437 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.576856  5082 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.577908  5260 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:33.615289  5315 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.573354  5013 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:33.616397 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:33.554625  5428 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:33.617808 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:33.589515  5420 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.551841  5387 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.654857  5413 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.658390  5428 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.661059  5428 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:33.577198  5243 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 12:19:33.623793 31971 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:33.625473 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.630535  5251 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.631903  5269 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.605806  5433 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:19:33.632815  5437 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.634897  5335 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.636382  5448 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:33.653064  5260 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:33.655339 31963 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.654392  5086 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:19:33.623692 32045 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:33.673822 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:33.674660 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81638.7us
I20260709 12:19:33.684734 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.661724  5413 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:33.687579 32120 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:33.671391 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46017.5us
I20260709 12:19:33.671697  5226 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:33.707342 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 111032us
I20260709 12:19:33.708209 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:33.682708  5260 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.718394 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:33.717150  5428 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:19:33.723965  5251 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.715437  5054 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:19:33.706928  5448 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:33.735833  5448 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.737571 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2400) took 1051 ms. Trace:
I20260709 12:19:33.738471  5413 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.739004  5413 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.737977 31972 rpcz_store.cc:276] 0709 12:19:32.686041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.910833 (+224792us) service_pool.cc:224] Handling call
0709 12:19:32.911219 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:33.373673 (+462454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:33.373719 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:33.388980 (+ 15261us) log.cc:844] Serialized 26 byte log entry
0709 12:19:33.610218 (+221238us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:33.610261 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.610356 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:33.737354 (+126998us) raft_consensus.cc:1685] finished
0709 12:19:33.737374 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.737511 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":204157,"spinlock_wait_cycles":5668736,"thread_start_us":584790,"threads_started":2,"wal-append.queue_time_us":315414}
I20260709 12:19:33.743901  5086 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.744133  5452 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.757674  5461 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:33.758862  5461 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:33.759348  5461 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:33.760308  5461 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.761072 31963 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:33.761672  5243 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.762252  5243 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.766041  5243 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:33.769182  5086 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:33.770676  5406 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:33.771258  5406 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.773881  5406 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:33.774461  5251 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.774662 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.775265  5073 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.775079  5251 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:33.776018 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:33.776577 32112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2637) took 2171 ms (client timeout 2147 ms). Trace:
I20260709 12:19:33.777109 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.774832  5269 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.777539  5251 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.777853 32113 raft_consensus.cc:2376] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17. Re-sending same reply.
I20260709 12:19:33.777962  5269 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:33.779304  5421 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.780s	user 0.002s	sys 0.000s
W20260709 12:19:33.776943 32112 rpcz_store.cc:269] 0709 12:19:31.605217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.928920 (+323703us) service_pool.cc:224] Handling call
0709 12:19:33.776410 (+1847490us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:33.776505 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073160064}
W20260709 12:19:33.781426 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73328us
I20260709 12:19:33.775780  5073 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.782986 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:33.789168 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.789371 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.788197 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:33.790603 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 177661us
I20260709 12:19:33.793744 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.796661 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.797554 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:33.795044 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2422) took 1119 ms. Trace:
I20260709 12:19:33.798514 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:33.810609 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.811358 32112 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.810670 32041 rpcz_store.cc:276] 0709 12:19:32.675615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.470230 (+794615us) service_pool.cc:224] Handling call
0709 12:19:33.470607 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.470842 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:33.691369 (+220527us) log.cc:844] Serialized 14 byte log entry
0709 12:19:33.794628 (+103259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:33.794694 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:33.794752 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.794839 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.794977 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12008,"prepare.queue_time_us":252563,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"thread_start_us":300960,"threads_started":2}
I20260709 12:19:33.813092 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:33.815969 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:33.788707  5269 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:33.818702 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.820204 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.820833 32117 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:33.823051 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.823714 32115 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.823032 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:33.825785 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.791740  5422 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:33.830058 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.810339  5461 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.818354  5144 raft_consensus.cc:2764] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:33.793535  5368 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.841568 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.810047  5250 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.812891  5387 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.820940  5406 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:33.823539  5216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.004s	sys 0.007s
I20260709 12:19:33.818286  5283 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.822041  5310 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:33.825582  5428 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:33.865077 32119 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:33.863374 32112 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:33.835412  5422 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:33.906256 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 107820us
W20260709 12:19:33.907615 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 81901.1us
I20260709 12:19:33.819617  5350 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:19:33.835772  5452 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.841989  5420 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:33.854362  5399 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:33.818503  5247 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:19:33.919222 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:33.921653 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.872951  5183 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:33.937034 32115 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
W20260709 12:19:33.911790  5216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.009s	sys 0.008s
I20260709 12:19:33.911957  4929 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.915333  5413 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:33.935981  5455 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:19:33.901741  5189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.480s	user 0.013s	sys 0.005s
I20260709 12:19:33.911872  5463 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.924134  5422 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:33.991398 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72260.9us
I20260709 12:19:33.991466 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:33.936838  5350 consensus_queue.cc:1243] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:33.921717  5428 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.935459  5310 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:33.908371  5387 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.871538  5368 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:33.949251  5399 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.006516 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:33.961663  5420 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:34.009382 32114 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:34.114693 32114 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:19:34.003471  5189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.583s	user 0.014s	sys 0.005s
I20260709 12:19:34.009922 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:34.086702  5445 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:34.023034  5472 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:34.023535  5073 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.041594  5013 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:34.147259 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 326914us
W20260709 12:19:34.043382  5456 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:19:34.044605  5165 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.047322 32121 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:19:34.048535  5473 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:19:34.003396  5341 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:34.001552  5433 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:34.152868  5341 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:34.157150  5341 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.051652 32119 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:34.057026 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2414) took 1460 ms. Trace:
I20260709 12:19:34.158576 32039 rpcz_store.cc:276] 0709 12:19:32.596252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.596379 (+   127us) service_pool.cc:224] Handling call
0709 12:19:32.597725 (+  1346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.207550 (+609825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.056714 (+849164us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54001225d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:34.056946 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199413504}
I20260709 12:19:34.058369 31868 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 10 to 14, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 14 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:32.876829 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:34.159562 32119 pending_rounds.cc:85] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 8
I20260709 12:19:34.160777 32119 pending_rounds.cc:107] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:19:34.161000 32119 raft_consensus.cc:2889] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:34.060837 31865 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 245) took 26349 ms (client timeout 15000 ms). Trace:
I20260709 12:19:34.117311  5246 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:34.117417  5252 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:34.068216  5205 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.069867  5257 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.076747  5345 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.077466 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:34.172776 32652 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:34.004081  5183 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:34.080164 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.082005 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:34.158156  5341 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.083129 32660 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:34.083792  5053 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:34.085017 32657 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:19:34.154361  5189 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.078171 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.162408  5413 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:34.184556 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:34.089313  5469 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
W20260709 12:19:34.091389 32659 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 837) took 1746 ms (client timeout 2028 ms). Trace:
I20260709 12:19:34.093111 32658 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:34.110402 32116 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.006439 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:34.115923 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 106064us
I20260709 12:19:33.949455  5452 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.187670 31868 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 9 to 10, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.140733  5453 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:34.141793  5144 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:34.142642  5420 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.121963  5086 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:19:34.101078  5428 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:34.111800  5399 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.068132  5331 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.203867  5144 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:34.210758  5331 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.211148  5144 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.212345  5144 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:34.165426 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:34.213526 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48155us
I20260709 12:19:34.215003 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:34.171125 32121 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:19:34.174748 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 92848.2us
I20260709 12:19:34.174216  5252 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.174254 32651 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:34.217664 32651 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 12:19:34.217917  5252 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.082351 32113 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:34.171989  5358 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:34.084801  5251 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:34.179033 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.227439 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.180024 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.164752  5246 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.120687  5459 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:19:34.154286  5461 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:19:34.087214 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:34.185442 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.187353 32660 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:34.190152 32047 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:34.191977  5183 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.178153  5257 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:34.177819  5310 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.193698 32043 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:34.151811  5445 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.195268  5368 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.172188  5205 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.272506  5205 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.262001  5428 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.274858 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.275590 32115 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:34.275843  5428 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.152020  5013 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.196487 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2595) took 1642 ms. Trace:
W20260709 12:19:34.188352 32659 rpcz_store.cc:269] 0709 12:19:32.345694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.345814 (+   120us) service_pool.cc:224] Handling call
0709 12:19:34.091160 (+1745346us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54005eabd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:34.091327 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541209088}
I20260709 12:19:34.138936  5387 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.199321 32652 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:34.278461  5013 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.191196 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.202430 32116 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:34.204648 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.207286  5422 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:34.279838 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 88690.1us
I20260709 12:19:34.280522 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:34.281073  5422 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.281666  5422 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.121019  5260 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.282518 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.282649 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.283166 31967 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:34.283414  5417 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:34.283789 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:34.284024 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:34.283826  5260 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.284807 31965 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:34.285131 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:34.285607 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.221756 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.285952 31971 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:34.219273 31868 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 12 to 13, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 13 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:34.236472 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.286718 32041 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:34.286813 31969 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:34.287204 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.288034 31964 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:34.288126 31971 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:34.287452 31868 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 15 to 17, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 17 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:34.236578 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.234920  5479 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:34.225144 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:34.289338 31868 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 9 to 10, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.290867 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:34.291270 32121 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:34.291702 31971 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:34.291970 31967 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:34.241250  4929 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.241328  5251 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.199862  5453 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:34.245918  5252 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:34.246421  5464 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:34.247010  5358 consensus_queue.cc:1243] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:19:34.185420  5335 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:19:34.313966 31868 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.317625 32115 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:34.318635 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.255815  5445 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.203547  5433 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:19:34.187250  5189 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:34.326891  5488 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:34.329265 32046 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:34.197593  5413 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.258285  5359 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:19:34.321233 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.331626 32041 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:19:34.258858 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:34.259317  5407 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.395s	user 0.000s	sys 0.003s
I20260709 12:19:34.236515  5246 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.263813  5452 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.248814  5073 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
W20260709 12:19:34.161520 31865 rpcz_store.cc:269] 0709 12:19:07.711657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.815122 (+103465us) service_pool.cc:224] Handling call
0709 12:19:34.060635 (+26245513us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:26.083352 (+     0us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b1800338da0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:26.083795 (+   443us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:26.084934 (+  1139us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:26.085002 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:19:26.085054 (+    52us) tablet.cc:662] Decoding operations
0709 12:19:26.229751 (+144697us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:26.229842 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:26.229880 (+    38us) tablet.cc:685] Acquiring locks for 30 operations
0709 12:19:26.248922 (+ 19042us) tablet.cc:701] Row locks acquired
0709 12:19:26.248969 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:19:26.249241 (+   272us) write_op.cc:270] Start()
0709 12:19:26.249506 (+   265us) write_op.cc:276] Timestamp: P: 1783599566249166 usec, L: 0
0709 12:19:26.249549 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:19:26.372737 (+123188us) log.cc:844] Serialized 11621 byte log entry
0709 12:19:26.820690 (+447953us) op_driver.cc:464] REPLICATION: finished
0709 12:19:26.844362 (+ 23672us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f22890c50ac 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228fc05ef0
0709 12:19:27.003008 (+158646us) write_op.cc:301] APPLY: starting
0709 12:19:27.003117 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:27.498052 (+494935us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:27.498095 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:28.289955 (+791860us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:28.432102 (+142147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=60,mrs_lookups=0
0709 12:19:28.432179 (+    77us) write_op.cc:312] APPLY: finished
0709 12:19:28.432817 (+   638us) log.cc:844] Serialized 260 byte log entry
0709 12:19:28.759754 (+326937us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:28.857224 (+ 97470us) write_op.cc:454] Released schema lock
0709 12:19:28.895340 (+ 38116us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:29.063434 (+168094us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:29.066436 (+  3002us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22384,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":182005,"apply.run_cpu_time_us":76208,"apply.run_wall_time_us":2024217,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129407,"num_ops":30,"prepare.queue_time_us":67711,"prepare.run_cpu_time_us":23332,"prepare.run_wall_time_us":334439,"raft.queue_time_us":175091,"raft.run_cpu_time_us":17063,"raft.run_wall_time_us":117038,"replication_time_us":570967,"spinlock_wait_cycles":89004160,"thread_start_us":527230,"threads_started":5,"wal-append.queue_time_us":513619,"wal-append.run_cpu_time_us":22887,"wal-append.run_wall_time_us":2926119}]]}
I20260709 12:19:34.270421  5257 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:34.281623 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.240527  5478 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:34.286855 31963 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:34.293715 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68701.1us
I20260709 12:19:34.277119 31968 rpcz_store.cc:276] 0709 12:19:32.554145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.554458 (+   313us) service_pool.cc:224] Handling call
0709 12:19:32.554851 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:32.557680 (+  2829us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:32.591696 (+ 34016us) log.cc:844] Serialized 14 byte log entry
0709 12:19:34.045586 (+1453890us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b180031d260. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:19:34.151525 (+105939us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.160321 (+  8796us) log.cc:844] Serialized 26 byte log entry
0709 12:19:34.160443 (+   122us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:34.160475 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.160539 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:34.196227 (+ 35688us) raft_consensus.cc:1685] finished
0709 12:19:34.196307 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:34.196434 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3303,"prepare.queue_time_us":38980,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2088,"raft.queue_time_us":36985,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5384,"spinlock_wait_cycles":2500480,"thread_start_us":105144,"threads_started":1,"wal-append.queue_time_us":1590064}
I20260709 12:19:34.299016 32656 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:34.299571 32013 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:34.300758 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.300192 31971 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 12:19:34.301863 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:34.303572 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:34.291538 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.310012 31972 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:34.298795  5127 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.289563  5422 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.308475  5479 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:34.403983 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 63307.5us
I20260709 12:19:34.321265 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.323251 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.325348 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:34.405606 31967 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:34.405788 32116 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:19:34.325961  5492 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:34.317564 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.408501 32117 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:34.410710 32121 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:34.328631 31868 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 15 to 16, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 16 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:34.341794 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60240.4us
I20260709 12:19:34.346604 31969 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:34.413022 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.306710  5165 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:34.328640  5189 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.308559  5459 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:34.306514  5387 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:34.308588  5205 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:34.403359 32045 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.308238  5251 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:34.405859 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80592.3us
I20260709 12:19:34.404134  5478 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.415643  5420 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:34.409951 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 110447us
I20260709 12:19:34.308897  5252 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:34.419942 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:34.423060 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:34.284026  5417 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.434336 31965 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:34.400905  5452 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:34.403409  5082 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:34.440162 31868 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 10 to 14, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 14 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:34.406873 32047 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:34.444144 32657 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 11.
I20260709 12:19:34.444334 31963 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:34.430012  5358 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:34.423787  5464 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:34.443272 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.453430 32045 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:34.441411  5205 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.470768 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.416371  5310 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:19:34.409287  5127 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.439045  5459 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.446585  5420 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:34.462085  5260 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:34.465204  5082 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:34.448017 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:34.472927 31868 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 12, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:34.483719 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63882.6us
I20260709 12:19:34.466095  5428 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:34.471585 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48570.7us
I20260709 12:19:34.447299  5387 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:34.474884 32047 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:34.522958 31868 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 7 to 10, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 10 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.427996 32119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:19:34.473717  5358 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:34.479967  5433 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:34.474901  5205 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:34.533022 32119 raft_consensus.cc:606] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:34.465828  5461 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:19:34.543594  5430 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:34.544711  5422 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.546183 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.547148  5358 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.551568  5127 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.481078  5257 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.555193 31868 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.484686  5459 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.519819  5420 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:34.556664  5436 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:34.557192  5347 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:34.557102  5246 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:34.557765 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 109811us
I20260709 12:19:34.502331  5475 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.512651  5362 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:34.480868  5025 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.517666  5183 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:34.584695 32119 raft_consensus.cc:614] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:34.524361  5260 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.612964  5430 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:34.613655  5490 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 12:19:34.619323 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.539309  5464 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
W20260709 12:19:34.620946 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 150278us
I20260709 12:19:34.668834 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.531941  5428 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.606454  5445 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:34.606020  5406 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.627539  5251 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.604276  5474 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:19:34.608199  5452 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:19:34.636492  5417 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.607148  5492 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:34.646380 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.707094 32116 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:34.651548  5489 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:19:34.652163  5478 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.530090  5252 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.525348  5479 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.617386  5082 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.640702 31868 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.657655 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.606236  5433 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:34.668875 32115 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:34.723978  5480 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:34.726586 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.727015  5480 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:34.727190  5082 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.727945 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.672646  5430 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.622180  5436 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:34.730180  5430 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.730906  5436 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.626564  5347 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.619127  5461 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 12:19:34.734215 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.702762  5429 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.611884  5399 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.704944 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.707384 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.699062  5445 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:34.710575 31963 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:34.700846  5496 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
W20260709 12:19:34.711007 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.711668 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:34.711570 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:34.739892 31868 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 11 to 13, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 13 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.712393 32119 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:19:34.648206  5189 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:19:34.664810  5183 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:34.701889  5341 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.002s
W20260709 12:19:34.702056  5222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.533s	user 0.000s	sys 0.016s
I20260709 12:19:34.731460 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.654381  5387 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.724030 32117 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:34.654332  5362 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:34.691977 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71135.2us
I20260709 12:19:34.661262  5025 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.737532 31966 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:34.738720 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:34.746172 32653 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:34.733702  5347 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.654760  5475 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.762363  5025 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.764184  5475 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:34.745095  5222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.576s	user 0.000s	sys 0.017s
I20260709 12:19:34.745771  5499 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.766794  5499 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.767752 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.768980  5499 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.769572 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:34.771631 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.772500 32117 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:34.737046  5436 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:34.722751  5406 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.750779  5445 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.755185  5310 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.756775  5428 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.744761 31868 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 10 to 12, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:34.790091 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:34.759476  5246 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.759773  5362 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.774747 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:34.762413  5399 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.776396 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.794523  5183 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.777038 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.741518  5252 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:34.781723 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.731408  5502 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:19:34.799026 32655 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 13: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:34.786037  5503 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:34.788332 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.802767 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:34.732604  5497 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:19:34.790288 32117 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:34.790745 31966 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.757205  5420 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.791088 31868 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.774747 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:34.795660 32660 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:34.785943  5496 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:34.806402 32654 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:34.782406  5464 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.727880  5480 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.807317 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:34.811414  5459 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.813074  5310 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:34.816259  5310 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.816675  5310 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.817406  5413 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.813520  5498 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.819861  5287 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:34.821552 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2563) took 1938 ms. Trace:
I20260709 12:19:34.821938 32042 rpcz_store.cc:276] 0709 12:19:32.883743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.971736 (+ 87993us) service_pool.cc:224] Handling call
0709 12:19:32.972233 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:33.095102 (+122869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:33.095133 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:33.104088 (+  8955us) log.cc:844] Serialized 26 byte log entry
0709 12:19:33.234747 (+130659us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:33.234810 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.821173 (+1586363us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400379e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:34.821223 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:34.821272 (+    49us) raft_consensus.cc:1685] finished
0709 12:19:34.821285 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:34.821495 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6577,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":2198,"spinlock_wait_cycles":281679232,"thread_start_us":96083,"threads_started":1,"wal-append.queue_time_us":280433,"wal-append.run_cpu_time_us":1800,"wal-append.run_wall_time_us":1048448}
I20260709 12:19:34.815169  5464 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.829874 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:34.829949 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.782771  5499 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:19:34.830909 31972 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:34.831449 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.840451 31969 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:19:34.832072 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.841717  5502 consensus_queue.cc:1243] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:19:34.842278  5502 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:34.844020  5502 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:34.844369  5502 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:34.846542  5504 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:34.793737  5433 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.848816  5464 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.845230  5502 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:34.854473  5464 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.854744 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.855058 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.854938  5464 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.855738 32045 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:34.855978 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.856563 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:34.857225  5464 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.858927  5459 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:34.854756 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.860570  5502 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:34.864089  5260 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:34.863895 31868 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 13 to 15, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 15 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:34.865942 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:34.824324  5399 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:34.867815 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.868557 32117 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:34.824679  5496 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:34.832738  4929 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:34.837579 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.838733 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:34.883065 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44415.3us
I20260709 12:19:34.874897 31966 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:34.875346 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.886670 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.881851 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.879130 32042 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:34.889871 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.892895 31972 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:34.892720 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.893745 32115 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:34.894779 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:34.896436 32623 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:34.900380 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:34.901765 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.906669 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:34.904177 32116 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:34.901001 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.911267 31972 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:34.912461 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.886715 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.915745 31969 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:34.915745 31963 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:19:34.919494 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.923995 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:34.881373  5144 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.926918 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.928563 31963 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:34.913281 31972 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:34.882332  4929 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:34.900178  5246 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.899726  5222 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:34.946420 32651 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:34.899933  5503 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:34.948350  5448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.005s	sys 0.009s
W20260709 12:19:34.954977  5448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.006s	sys 0.009s
I20260709 12:19:34.946511  5433 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.964178  5433 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:34.908324  5399 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:34.975322 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:35.005905 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:34.925472  5499 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:34.936169  5452 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:35.003492  5486 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:19:34.972862 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.008327 31967 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:35.021857 32113 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:35.023079 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:34.968600  5422 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:34.949712  5222 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:35.010174  5346 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.011453  5499 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:35.030819 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 58041.2us
I20260709 12:19:34.947043  5428 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:34.934916  5144 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.039172 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.019117  5452 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.949962  5246 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.018591  5399 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:35.043318 32042 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:35.034157  5222 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:35.042765 32047 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 12:19:35.059504 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:35.060581  5425 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:35.029398  5422 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:35.006676  5503 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.063289 31967 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:35.124657 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:35.113158 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.056591  5452 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:35.040030  5499 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.038335  5464 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.121592 31966 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:35.042989  5362 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:35.115231 32043 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:19:35.135267 31967 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:35.055662  5500 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.150847 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.151810 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:34.951097  5496 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.058117  5082 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:35.153308 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.039618  5346 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:35.140698  5407 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:35.164964 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.179461 31969 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:35.119670  5413 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.127607  5368 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:35.054898  5428 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.177433  5025 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.169489 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:35.197628 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 145730us
I20260709 12:19:35.170557  5082 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.172086 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.206061 32654 raft_consensus.cc:2410] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:19:35.172945 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.206739 31967 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:35.033228  4929 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.117157  5382 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:19:35.062770  5341 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 12:19:35.148746  5205 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.001s
I20260709 12:19:35.144325  5362 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.167380  5255 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:35.198966 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.195487  5496 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.144853  5425 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.106426  5183 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.204193 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.208436  5082 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:35.211081 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.227713 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.154356  5464 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.214564 31964 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:35.220641 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.228921 31967 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 12.
I20260709 12:19:35.229044 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.170501  5331 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.195279  5507 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:35.199481 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.167145  5500 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.223083  5255 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.197069  5025 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.172098  5420 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:35.198884 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.195943  5368 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.194156  5413 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.216747  5436 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:35.222929  5362 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.244087 31969 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:35.246824 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.255910 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:35.256229 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:35.242321  4929 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:35.191987  5422 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.242705  5448 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:35.276839 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.244311  5269 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:35.194162  5399 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:35.249483  5505 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.232s	user 0.000s	sys 0.005s
I20260709 12:19:35.222791  5205 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.244091  5341 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:19:35.273561 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 89170us
W20260709 12:19:35.273712 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74902.7us
I20260709 12:19:35.260012  5504 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:19:35.279350 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 969) took 2548 ms. Trace:
I20260709 12:19:35.291323 32658 rpcz_store.cc:276] 0709 12:19:32.731126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.731286 (+   160us) service_pool.cc:224] Handling call
0709 12:19:34.092742 (+1361456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.188430 (+ 95688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.279158 (+1090728us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54005eabd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:35.279292 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3130078592}
I20260709 12:19:35.242967  5183 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.248845  5255 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:35.281859 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.248761  5025 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.246815  5347 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.294394 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2581) took 1091 ms. Trace:
I20260709 12:19:35.298214 32046 rpcz_store.cc:276] 0709 12:19:34.203034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.227198 (+ 24164us) service_pool.cc:224] Handling call
0709 12:19:35.294225 (+1067027us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400216e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:35.294340 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232658816}
I20260709 12:19:35.294556 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.300763 31966 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:19:35.278365  5517 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.300884 32114 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:35.248354  5330 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:19:35.291816 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.281206  5269 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:35.280535 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:35.316138 31969 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:35.301694 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.316833 31966 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:35.317945 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:35.319032 31966 raft_consensus.cc:2410] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:19:35.248420  5507 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.297535  5516 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.298125  5422 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:35.290427  5341 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:35.296280  5144 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.323411 31966 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:35.327277  5444 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.327939  5425 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:35.328104  5511 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:19:35.328747  5453 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:35.330500  5506 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 12:19:35.321261  5512 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:19:35.346412 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:35.353999 31969 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:35.312214  5497 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:35.326607  5516 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.312650  5479 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:19:35.325323  5503 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:35.363039 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71303.5us
I20260709 12:19:35.367395 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.347476  5524 raft_consensus.cc:2764] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:35.353340  5425 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:35.353595  5346 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.365432  5448 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:35.359567  5436 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:35.358831  5428 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.313565  5517 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.379393  5205 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.438792  5499 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.438866  5025 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.383538  5507 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.441196  5025 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.441566 31966 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:19:35.443149  5452 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.372020  5341 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:35.389415 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.448043  5255 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.448719  5330 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:35.391006 31968 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [16.12-17.13]
I20260709 12:19:35.449219  5330 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.449743  5447 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.417606  5082 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.450270  5447 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.392004  5183 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.457835 32120 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:35.392783  5382 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.394515 32653 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:35.362461  5516 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:35.396088  5514 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:19:35.396687  5347 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.398481  5246 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.398638  5464 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.399905  5335 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:35.401371  5433 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.406652  5084 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.365940  5453 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:35.407814  5437 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:35.408895  5429 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.671s	user 0.004s	sys 0.003s
I20260709 12:19:35.409976 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 980) took 1317 ms. Trace:
I20260709 12:19:35.512142 32657 rpcz_store.cc:276] 0709 12:19:34.092328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.185239 (+ 92911us) service_pool.cc:224] Handling call
0709 12:19:35.409790 (+1224551us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:35.409913 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462411520}
I20260709 12:19:35.410961  5368 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:35.513096 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:35.411791  5502 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 12:19:35.412561 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:35.518342  5509 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.285s	user 0.003s	sys 0.000s
I20260709 12:19:35.519225 31971 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:35.414181 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:35.414722  5425 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:35.415033 32047 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:35.520836 32043 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 13: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:35.521760 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.415270 32041 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:19:35.523057 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:35.523483 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.523767 32043 raft_consensus.cc:2368] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19.
I20260709 12:19:35.524139 32041 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:35.524634 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.525005 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.525498 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 856) took 1454 ms. Trace:
I20260709 12:19:35.416090  5389 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:35.417217 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2609) took 1135 ms. Trace:
I20260709 12:19:35.527336 31965 rpcz_store.cc:276] 0709 12:19:34.281529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.285675 (+  4146us) service_pool.cc:224] Handling call
0709 12:19:34.285934 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.465107 (+179173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.417000 (+951893us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000b6b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:35.417159 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323308416}
I20260709 12:19:35.527871  5425 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:35.417536  5310 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.001s
I20260709 12:19:35.526167 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.362103  5497 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.394507 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.425271  5413 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.423019 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.425027 31967 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:35.424918 31969 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
W20260709 12:19:35.429761 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.525938 32656 rpcz_store.cc:276] 0709 12:19:34.090152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.183511 (+ 93359us) service_pool.cc:224] Handling call
0709 12:19:34.183781 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.408804 (+225023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.525268 (+1116464us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001ebc54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:35.525437 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264519040}
W20260709 12:19:35.542603 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 119669us
I20260709 12:19:35.431180 32114 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:35.363992  5269 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.434181  5436 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.435559  5517 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.436471  5520 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:35.436864 32651 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:35.432940  5331 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.458845 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.460780  5499 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.466823 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:35.494508 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:35.450134  5255 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.459587  5452 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:35.504664 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 139057us
I20260709 12:19:35.594597 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:35.506832 32040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2578) took 2080 ms (client timeout 2132 ms). Trace:
I20260709 12:19:35.507529  5524 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.510609  5330 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.517647 31970 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:35.599699 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:35.600903 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.601491 32044 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:35.521711 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.603384 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.604329 32047 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:35.605048 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:35.415925 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.543241 32043 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:35.606930 31967 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:35.548939 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.553285  5389 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:35.609082  5389 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:35.536485 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 266202us
I20260709 12:19:35.539460  5448 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.609977  5389 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.547451  5310 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.611644  5287 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:19:35.612735 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52647.2us
I20260709 12:19:35.613555  5189 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:35.501859  5525 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 12:19:35.496635  5335 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:35.615837  5082 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.617955 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:35.609968 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.619045 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.551340  5269 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.466244  5260 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:19:35.620561  5255 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:35.636018  5260 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:35.597275  5331 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.636509  5260 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:35.637209 32042 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:35.561400 32046 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:35.534865  5461 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.534336  5475 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.540799  5350 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:35.575276 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.551757  5502 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.637420  5260 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:35.592409 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.592607 31969 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:35.547593  5417 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.595613 32040 rpcz_store.cc:269] 0709 12:19:33.426578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.426759 (+   181us) service_pool.cc:224] Handling call
0709 12:19:35.502991 (+2076232us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:35.506750 (+  3759us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821844224}
I20260709 12:19:35.649070 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.598910  5524 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:35.649268 31969 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:35.499805  5222 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.638470  5001 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:19:35.652375  5222 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.653374 32652 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:35.465014  5382 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.620878 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:35.623836 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.654704 32114 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:35.655665  5382 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:35.656008 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.656445  5524 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.628719 31970 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:35.630785 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.633683 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.659240 32116 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:35.634483 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.555761  5503 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:19:35.660039 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.589702 32038 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:35.639477 32046 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:35.641320 32041 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: [20.15-22.16]
I20260709 12:19:35.558256  5520 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:35.638981 32113 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:35.535576  5084 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:35.646864 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.612811  5310 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.639001  5437 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:19:35.653687  5260 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.659705  5417 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.598328  5347 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:35.661513 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.711527 32038 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:35.639984 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.659734 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:35.712670 32656 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:35.664841 32119 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:19:35.713543 32119 pending_rounds.cc:85] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 15
I20260709 12:19:35.713815 32119 pending_rounds.cc:107] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:19:35.640343  5350 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:35.664368 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:35.669737 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.671029 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.721431 31972 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
W20260709 12:19:35.721706 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50774us
I20260709 12:19:35.671821 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:35.674470 32046 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 12:19:35.696048 32117 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:35.696744 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:35.709481 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.709585 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:35.727505 32657 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:35.714056 32119 raft_consensus.cc:2889] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:35.666906 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46103.7us
I20260709 12:19:35.715529 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.729768 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:35.712136 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.664096  5444 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.747977 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:35.746366 32113 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 12: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:35.638927  5331 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.672380  5300 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:35.746341 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:35.646234  5416 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:19:35.638281  5362 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.757009 31967 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:35.762681 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:35.741526 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.639987  5461 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.641577  5502 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.710390  5496 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:35.640211  5475 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.698262  5503 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 12:19:35.696079  5001 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 12:19:35.755635  5082 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:35.783960 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.754462  5526 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.616621  5335 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:35.782938 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.800109 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:35.802896 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.809353 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.770766  5255 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:35.825147 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.807920  5525 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.826866  5525 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.829504  5525 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:35.802502  5520 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.830636  5516 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.831741  5520 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.831878 32038 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:19:35.833916  5516 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:35.834717  5532 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:35.836314 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:35.840754 32115 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:35.841217 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 340188us
I20260709 12:19:35.754683  5441 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 12:19:35.851313 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.850724  5479 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.844784 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.720715  5347 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.758376  5413 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:19:35.850222 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:35.809614  5331 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:35.808830  5382 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:35.757078  5444 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.830157  5525 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:35.808284  5526 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.870991 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:35.843590  5086 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:35.838243  5255 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:35.843057  5082 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:35.842079  5416 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:19:35.857165  5533 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.881538  5205 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:35.872336  5331 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:35.858779  5447 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.878050  5382 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.820832  5496 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.854911  5499 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:19:35.886919  5525 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.892105  5526 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.954226 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:35.884747  5252 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.895310  5255 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.853763  5479 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:35.836455  5350 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.906674  5205 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.872859  5518 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:19:35.895538  5536 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:35.909826  5497 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:35.944131  5382 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:35.958101  5525 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.961211  5310 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:35.987727 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 162640us
I20260709 12:19:35.949680  5331 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.955055 32119 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
W20260709 12:19:36.023221 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:35.969337  5508 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.912s	user 0.003s	sys 0.000s
I20260709 12:19:35.966780  5205 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.032974 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:35.959750  5252 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.909267  5533 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:35.990413  5536 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:36.049891 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:35.967836  5084 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:36.034289  5350 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:36.075023 32117 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:36.075496 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.107591 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:36.057539  5252 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.093832  5422 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.095070  5362 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:36.094606  5425 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:36.129873 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.097939  5542 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:36.162635  5347 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:36.173117  5534 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.284s	user 0.000s	sys 0.000s
W20260709 12:19:36.190861 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.189036 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:36.195775  5533 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:36.202607  5444 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:36.207134  5548 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:19:36.386615  5547 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.280s	user 0.000s	sys 0.002s
I20260709 12:19:36.046016  5082 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:36.207795  5362 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:36.209658  5542 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.213011  5479 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:36.221457  5537 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
W20260709 12:19:36.222477 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.409540 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.410006 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.277245  5260 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
W20260709 12:19:36.224790  5538 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
W20260709 12:19:36.225679  5540 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
W20260709 12:19:36.222512  5543 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:19:36.236225  5496 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:36.238734 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:36.186985  5536 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:36.285068 32117 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
W20260709 12:19:36.378842  5524 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.722s	user 0.004s	sys 0.007s
W20260709 12:19:36.379392  5520 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.008s
I20260709 12:19:36.137908  5422 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:36.402925 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2683) took 2137 ms. Trace:
I20260709 12:19:36.388566 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 715) took 1503 ms. Trace:
I20260709 12:19:36.218422 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2626) took 1166 ms. Trace:
I20260709 12:19:36.760815 31969 rpcz_store.cc:276] 0709 12:19:35.051509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.651246 (+599737us) service_pool.cc:224] Handling call
0709 12:19:35.651568 (+   322us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:35.666557 (+ 14989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.666602 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:35.675370 (+  8768us) log.cc:844] Serialized 78 byte log entry
0709 12:19:35.950982 (+275612us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:35.951036 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.951123 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.218093 (+266970us) raft_consensus.cc:1685] finished
0709 12:19:36.218123 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.218344 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17365,"raft.queue_time_us":15382,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":28307,"thread_start_us":193387,"threads_started":1,"wal-append.queue_time_us":501554}
W20260709 12:19:36.443327 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 393510us
I20260709 12:19:36.767604 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:36.760665 31963 rpcz_store.cc:276] 0709 12:19:34.885134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.384775 (+499641us) service_pool.cc:224] Handling call
0709 12:19:35.385187 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:35.498825 (+113638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.498868 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:35.499496 (+   628us) log.cc:844] Serialized 52 byte log entry
0709 12:19:35.894429 (+394933us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:36.130879 (+236450us) log.cc:844] Serialized 14 byte log entry
0709 12:19:36.134841 (+  3962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.134953 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.388212 (+253259us) raft_consensus.cc:1685] finished
0709 12:19:36.388234 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.388424 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244074,"raft.queue_time_us":122258,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":16687,"thread_start_us":532031,"threads_started":2,"wal-append.queue_time_us":559963}
I20260709 12:19:36.750813 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2620) took 1842 ms. Trace:
I20260709 12:19:36.751477 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2621) took 1843 ms. Trace:
I20260709 12:19:36.446148 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:36.218219 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2628) took 1166 ms. Trace:
I20260709 12:19:36.768200 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:36.755406 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.785135 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.760459 32112 rpcz_store.cc:276] 0709 12:19:34.265086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.265230 (+   144us) service_pool.cc:224] Handling call
0709 12:19:34.265574 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.266342 (+   768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.266377 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.266812 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 12:19:34.313339 (+ 46527us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:34.416068 (+102729us) log.cc:844] Serialized 14 byte log entry
0709 12:19:36.402298 (+1986230us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b54001e0b20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c14a733 00007f228c1cfdbf 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c
0709 12:19:36.402386 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.402507 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.402570 (+    63us) raft_consensus.cc:1685] finished
0709 12:19:36.402673 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.402848 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20823,"prepare.queue_time_us":102788,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":9129,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":10617,"spinlock_wait_cycles":17154048,"thread_start_us":46187,"threads_started":1,"wal-append.queue_time_us":44113,"wal-append.run_cpu_time_us":3181,"wal-append.run_wall_time_us":1211464}
W20260709 12:19:36.789778 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.777974 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:36.791478 32114 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:36.749245 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.792732 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:36.750195 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.796094 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.776983 31966 rpcz_store.cc:276] 0709 12:19:34.908019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.443800 (+535781us) service_pool.cc:224] Handling call
0709 12:19:35.444099 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.444402 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.444430 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.463751 (+ 19321us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.802026 (+338275us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:35.802072 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.802154 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.751031 (+948877us) raft_consensus.cc:1685] finished
0709 12:19:36.751067 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.751321 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21498,"raft.queue_time_us":6991,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7552,"thread_start_us":299898,"threads_started":1,"wal-append.queue_time_us":477398}
I20260709 12:19:36.801800 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:36.802845 31966 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:36.787294 31969 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:36.808512 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2443) took 1103 ms. Trace:
I20260709 12:19:36.779968 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:36.808957 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:36.809645 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 717) took 1140 ms. Trace:
I20260709 12:19:36.776455 31970 rpcz_store.cc:276] 0709 12:19:35.051991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.658168 (+606177us) service_pool.cc:224] Handling call
0709 12:19:35.658530 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.667557 (+  9027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.667597 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.696157 (+ 28560us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.958987 (+262830us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:35.959047 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.959140 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.217940 (+258800us) raft_consensus.cc:1685] finished
0709 12:19:36.217967 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.218148 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8601,"raft.queue_time_us":13961,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":13170,"thread_start_us":157588,"threads_started":1,"wal-append.queue_time_us":445753}
I20260709 12:19:36.776312 31964 rpcz_store.cc:276] 0709 12:19:34.907774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.439245 (+531471us) service_pool.cc:224] Handling call
0709 12:19:35.439546 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.439840 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.439867 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.440325 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.804494 (+364169us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:36.038224 (+233730us) log.cc:844] Serialized 14 byte log entry
0709 12:19:36.092341 (+ 54117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.092434 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.749971 (+657537us) raft_consensus.cc:1685] finished
0709 12:19:36.750161 (+   190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.750575 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52981,"raft.queue_time_us":55291,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":18172288,"thread_start_us":588341,"threads_started":2,"wal-append.queue_time_us":454864}
I20260709 12:19:36.414098 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 891) took 1054 ms. Trace:
I20260709 12:19:36.411012  5205 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:36.423871  5362 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:36.409166  5359 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.443329  5479 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:36.416656  5515 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.167s	user 0.000s	sys 0.018s
I20260709 12:19:36.414520  5526 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:36.419624  5542 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:37.000679 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.004s	sys 0.012s
W20260709 12:19:37.002008 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.004s	sys 0.012s
W20260709 12:19:37.002903 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 866591us
I20260709 12:19:36.597266  5422 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.004042  5542 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.004770  5422 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.005033  5542 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:36.810127 31963 rpcz_store.cc:276] 0709 12:19:35.668884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.779427 (+1110543us) service_pool.cc:224] Handling call
0709 12:19:36.809501 (+ 30074us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:36.810302 31966 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:36.815156 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:36.808981 31969 rpcz_store.cc:276] 0709 12:19:35.705297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.767588 (+1062291us) service_pool.cc:224] Handling call
0709 12:19:36.808411 (+ 40823us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:36.848817 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.818009 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:36.811367 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.007949 31970 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.008642 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:36.819868 32121 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.028s	sys 0.036s
I20260709 12:19:37.009251 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.010032  5569 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:19:37.010778 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.011025 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.012025 32112 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:37.012233 31969 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:37.012461  5532 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:37.013173 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:36.538538  5269 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.897s	user 0.013s	sys 0.000s
W20260709 12:19:36.499413  5255 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 12:19:36.463008  5524 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.014797  5532 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.014986  5255 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.017616  5524 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.017745  5255 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.018131  5260 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.018663  5260 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.018745 31971 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:37.018958 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2634) took 1150 ms. Trace:
W20260709 12:19:36.601995  5350 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.000s
I20260709 12:19:36.999660  5479 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:36.995561 31968 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.029s	sys 0.029s
W20260709 12:19:37.009467 32121 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.028s	sys 0.036s
I20260709 12:19:37.012845 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2635) took 1143 ms. Trace:
W20260709 12:19:37.012259  5382 consensus_meta.cc:287] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.953s	user 0.003s	sys 0.009s
W20260709 12:19:36.987438 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 195066us
I20260709 12:19:37.005716  5422 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.019469 31966 rpcz_store.cc:276] 0709 12:19:35.868814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.808493 (+939679us) service_pool.cc:224] Handling call
0709 12:19:37.018873 (+210380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.020236 31963 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.819504 32120 rpcz_store.cc:276] 0709 12:19:35.359318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.359473 (+   155us) service_pool.cc:224] Handling call
0709 12:19:35.370006 (+ 10533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.462282 (+ 92276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.413653 (+951371us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b54002fab54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:36.414007 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186881792}
I20260709 12:19:37.034188 31969 rpcz_store.cc:276] 0709 12:19:35.869049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.010558 (+1141509us) service_pool.cc:224] Handling call
0709 12:19:37.012742 (+  2184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:36.588400  5461 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:36.988727 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 186451us
W20260709 12:19:37.046144 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:37.045652 31968 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.029s	sys 0.029s
I20260709 12:19:37.019062  5260 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.038640  5569 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.015733  5532 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:36.995579  5359 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:37.073546  5331 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.006s	user 0.012s	sys 0.001s
I20260709 12:19:37.074352  5331 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.076767 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:37.079048 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2696) took 1005 ms. Trace:
I20260709 12:19:37.079586 32119 rpcz_store.cc:276] 0709 12:19:36.073596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.074862 (+  1266us) service_pool.cc:224] Handling call
0709 12:19:36.104600 (+ 29738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:36.104840 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:36.425105 (+320265us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.078198 (+653093us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.078331 (+   133us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:37.078405 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.078534 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.078793 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9557,"prepare.queue_time_us":436025,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":19728,"thread_start_us":818079,"threads_started":2,"wal-append.queue_time_us":646596}
I20260709 12:19:36.499008  5525 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.083027 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 894) took 1546 ms. Trace:
I20260709 12:19:37.077289  5331 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.091773 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.096838 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.097152  5542 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:37.097887 31966 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:37.098443 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2637) took 1228 ms. Trace:
I20260709 12:19:37.098798 31966 rpcz_store.cc:276] 0709 12:19:35.869561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.057821 (+1188260us) service_pool.cc:224] Handling call
0709 12:19:37.098384 (+ 40563us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.103633  5422 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.104990 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2622) took 2196 ms. Trace:
I20260709 12:19:37.106699 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.074991  5205 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.111904 31964 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:37.106405 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.115025  5526 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.116195  5525 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.116395  5526 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:37.117416 32116 raft_consensus.cc:2368] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:37.117244  5526 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.118427  5542 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.119230  5542 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.120527  5526 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.121842  5542 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.122643  5479 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.123283  5479 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:37.123970  5479 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.130900 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 719) took 1443 ms. Trace:
I20260709 12:19:37.131349 31964 rpcz_store.cc:276] 0709 12:19:35.669680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.817567 (+1147887us) service_pool.cc:224] Handling call
0709 12:19:37.113291 (+295724us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.126937 31966 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:37.113957  5368 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:37.133785 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2645) took 1236 ms. Trace:
I20260709 12:19:37.137344 31966 rpcz_store.cc:276] 0709 12:19:35.897843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.099535 (+1201692us) service_pool.cc:224] Handling call
0709 12:19:37.133724 (+ 34189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.097750  5362 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.149502  5576 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.150494  5576 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.136809  5569 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.157248  5532 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.159274  5580 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:37.159876  5580 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.160234  5580 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.064075  5567 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.148s
W20260709 12:19:37.164660  5519 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.877s	user 0.014s	sys 0.040s
I20260709 12:19:37.094693 32121 rpcz_store.cc:276] 0709 12:19:35.519143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.519479 (+   336us) service_pool.cc:224] Handling call
0709 12:19:35.519756 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.519938 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:35.520238 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.047984 (+1527746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.048110 (+   126us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:37.050286 (+  2176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.050435 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.065842 (+ 15407us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":570,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":178300,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15434,"thread_start_us":1518443,"threads_started":2,"wal-append.queue_time_us":901602}
I20260709 12:19:37.166884  5359 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.167618 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.168841 32112 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:19:37.169016 32121 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:37.175570 32114 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:37.179097 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2702) took 1086 ms. Trace:
I20260709 12:19:37.180600 32114 rpcz_store.cc:276] 0709 12:19:36.096679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.765871 (+669192us) service_pool.cc:224] Handling call
0709 12:19:37.177834 (+411963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.199033 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.096030  5382 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:37.200052 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 93525.2us
I20260709 12:19:37.204432  5425 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:37.205202  5425 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.207432  5425 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.204433 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 742) took 2142 ms. Trace:
I20260709 12:19:37.208907 32045 rpcz_store.cc:276] 0709 12:19:35.049825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.049928 (+   103us) service_pool.cc:224] Handling call
0709 12:19:35.050231 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.050456 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.050484 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.050898 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.205078 (+154180us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:35.205133 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.205234 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:37.163643 (+1958409us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:37.169370 (+  5727us) raft_consensus.cc:1685] finished
0709 12:19:37.169389 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.192328 (+ 22939us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8866688,"thread_start_us":78691,"threads_started":1,"wal-append.queue_time_us":183118}
I20260709 12:19:37.212432 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.215785  5584 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.216645  5584 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.219280  5567 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:37.219955  5567 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.194116  5347 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.202056 32651 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:19:37.223090 31964 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:37.223711 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2638) took 1353 ms. Trace:
I20260709 12:19:37.224062 31964 rpcz_store.cc:276] 0709 12:19:35.869865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.134655 (+1264790us) service_pool.cc:224] Handling call
0709 12:19:37.223598 (+ 88943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.224921  5382 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.225276 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.226413 31964 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 12:19:37.226905 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.226590 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:37.227304  5542 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.239070  5542 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.240455  5226 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:37.241113  5226 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.241529  5542 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.241588  5257 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.227861  5399 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.242583  5399 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.242508  5257 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.229362 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.231137  5425 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:37.237306 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 749) took 1583 ms. Trace:
I20260709 12:19:37.244261 31963 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:37.222343  5584 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.231702 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 748) took 1578 ms. Trace:
I20260709 12:19:37.231855 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2698) took 1157 ms. Trace:
W20260709 12:19:37.237285 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.235299  5567 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.245700 32041 rpcz_store.cc:276] 0709 12:19:35.653513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.229872 (+1576359us) service_pool.cc:224] Handling call
0709 12:19:37.230643 (+   771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.231121 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.231145 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.231236 (+    91us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:37.231278 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.231345 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.231486 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.251010 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.245846 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:37.246373 32114 rpcz_store.cc:276] 0709 12:19:36.074289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.230635 (+1156346us) service_pool.cc:224] Handling call
0709 12:19:37.231120 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.231492 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.231518 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.231577 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:37.231633 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.231694 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.231784 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.244546 32038 rpcz_store.cc:276] 0709 12:19:35.653778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.716225 (+ 62447us) service_pool.cc:224] Handling call
0709 12:19:35.771088 (+ 54863us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.832389 (+ 61301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.233428 (+1401039us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b54001a54d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.237071 (+  3643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139254144}
I20260709 12:19:37.221714  5347 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:37.254134 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.267032  5347 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.254359  5582 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:37.254900 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:37.255599  5226 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.258589  5576 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.260788 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2671) took 1613 ms. Trace:
I20260709 12:19:37.274299 31970 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 12:19:37.261543 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66448us
I20260709 12:19:37.261765  5532 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:37.262916 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2434) took 1809 ms. Trace:
I20260709 12:19:37.263203 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2704) took 1149 ms. Trace:
I20260709 12:19:37.263713  5581 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.264117 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:37.264436 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2428) took 2337 ms. Trace:
W20260709 12:19:37.275084 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2442) took 1570 ms (client timeout 1919 ms). Trace:
I20260709 12:19:37.275414 32121 rpcz_store.cc:276] 0709 12:19:36.114589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.244801 (+1130212us) service_pool.cc:224] Handling call
0709 12:19:37.251601 (+  6800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.262780 (+ 11179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.262810 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.262869 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:37.262899 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.262981 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.263111 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:37.279268 31970 rpcz_store.cc:269] 0709 12:19:35.704487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.810945 (+1106458us) service_pool.cc:224] Handling call
0709 12:19:37.275005 (+464060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.275331 32042 rpcz_store.cc:276] 0709 12:19:35.453302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.606480 (+153178us) service_pool.cc:224] Handling call
0709 12:19:35.606759 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.637622 (+ 30863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.262096 (+1624474us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54001a4fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.262655 (+   559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69470080}
I20260709 12:19:37.244475  5425 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.274436 32115 rpcz_store.cc:276] 0709 12:19:35.646964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.659525 (+ 12561us) service_pool.cc:224] Handling call
0709 12:19:37.260355 (+1600830us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400123754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:37.260593 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285372160}
W20260709 12:19:37.287744 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.289296 32112 raft_consensus.cc:2410] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:19:37.290591 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.277153  5524 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.276415  5532 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.275754  5569 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.279281  5542 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.272779  5260 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.295833 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2636) took 1426 ms. Trace:
W20260709 12:19:37.294647 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49256.2us
I20260709 12:19:37.297392 31966 rpcz_store.cc:276] 0709 12:19:35.869300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.193764 (+1324464us) service_pool.cc:224] Handling call
0709 12:19:37.295782 (+102018us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.298648 32659 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:37.298893  5480 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:37.302048  5257 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.302676 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.303601 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2640) took 1421 ms. Trace:
I20260709 12:19:37.303665  5260 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.303983 31970 rpcz_store.cc:276] 0709 12:19:35.882149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.284749 (+1402600us) service_pool.cc:224] Handling call
0709 12:19:37.303531 (+ 18782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.298461  5536 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.304610  5524 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.307135  5399 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.306838  5536 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:37.308650  5573 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.233s	user 0.002s	sys 0.000s
I20260709 12:19:37.310146 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:37.310859 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56289.5us
I20260709 12:19:37.105463 31968 rpcz_store.cc:276] 0709 12:19:34.908250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.450206 (+541956us) service_pool.cc:224] Handling call
0709 12:19:35.450520 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.450696 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:35.871463 (+420767us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.104329 (+1232866us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.104450 (+   121us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:37.104560 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.104695 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.104921 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":577937,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":16519,"spinlock_wait_cycles":12032896,"thread_start_us":1604030,"threads_started":2,"wal-append.queue_time_us":1196875}
W20260709 12:19:37.305835 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.314643  5587 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:37.315790  5587 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:37.316287  5587 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:37.316762  5586 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.317405  5586 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.318663 32041 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:37.319478 32045 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:37.320601 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.320851 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.321439 32040 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:37.321511 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.322522 32046 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.322573 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.323153 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:37.296051  5082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.020s	sys 0.025s
I20260709 12:19:37.323906  5587 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:37.324668  5082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.020s	sys 0.025s
I20260709 12:19:37.325662  5587 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.326472 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.327284 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:37.327633 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.328109 32115 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.328688 32045 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:37.330622  5479 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.330652 32659 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:37.331583 32120 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:37.331504  5399 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.332242  5536 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.296509 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.332365  5399 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.333596  5542 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.344568 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.345417 31969 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:37.346521  5542 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.334491  5581 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.347597 32119 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.348908  5581 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.349393  5479 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:37.349922 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.350353  5479 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.351032 32041 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.352834 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.336186  5586 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.277905 31965 rpcz_store.cc:276] 0709 12:19:34.926569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.528442 (+601873us) service_pool.cc:224] Handling call
0709 12:19:35.528766 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.529002 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:35.909608 (+380606us) log.cc:844] Serialized 14 byte log entry
0709 12:19:36.078506 (+168898us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.076939 (+998433us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b18003a3060. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:19:37.096298 (+ 19359us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.169532 (+ 73234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.188754 (+ 19222us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:37.201530 (+ 12776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.203824 (+  2294us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.264268 (+ 60444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":968444,"prepare.queue_time_us":2559799,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":111,"raft.queue_time_us":36975,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":10731,"spinlock_wait_cycles":9010176,"thread_start_us":537876,"threads_started":2,"wal-append.queue_time_us":308532}
W20260709 12:19:37.330941  5539 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.302s	user 0.025s	sys 0.032s
I20260709 12:19:37.354463 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.354722  5368 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:37.355367 31965 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:37.355574  5368 consensus_queue.cc:1243] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:19:37.355868 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2646) took 1457 ms. Trace:
I20260709 12:19:37.356055 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 893) took 1928 ms. Trace:
I20260709 12:19:37.357040  5479 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.304576  5582 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:37.357559 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.339184  5429 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:37.358498 32651 pending_rounds.cc:85] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 12
I20260709 12:19:37.356272 31965 rpcz_store.cc:276] 0709 12:19:35.898232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.354107 (+1455875us) service_pool.cc:224] Handling call
0709 12:19:37.355805 (+  1698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.292712  5532 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:37.343314 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51582.1us
I20260709 12:19:37.343873 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.339818 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:37.353788 31969 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:37.327142 31966 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:37.318679 32038 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:19:37.353075 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2707) took 1198 ms. Trace:
I20260709 12:19:37.360946 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2633) took 1492 ms. Trace:
I20260709 12:19:37.361258 32038 pending_rounds.cc:85] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 10
I20260709 12:19:37.362112 31969 rpcz_store.cc:276] 0709 12:19:35.868571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.346060 (+1477489us) service_pool.cc:224] Handling call
0709 12:19:37.360875 (+ 14815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.361383 32114 rpcz_store.cc:276] 0709 12:19:36.398681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.275636 (+876955us) service_pool.cc:224] Handling call
0709 12:19:37.352166 (+ 76530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.352611 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.352641 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.352712 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:37.352753 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.352838 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.352970 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.366456 32119 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:37.367373 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2431) took 2435 ms. Trace:
I20260709 12:19:37.362778 32038 pending_rounds.cc:107] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:19:37.359835 32651 pending_rounds.cc:107] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:19:37.368007 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:37.370054 32651 raft_consensus.cc:2889] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:37.368273  5399 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:37.372188 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.373849 32038 raft_consensus.cc:2889] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:37.369223 31972 rpcz_store.cc:276] 0709 12:19:34.931499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.603183 (+671684us) service_pool.cc:224] Handling call
0709 12:19:35.603462 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.721815 (+118353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.358454 (+1636639us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54000b6154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.367180 (+  8726us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255690112}
I20260709 12:19:37.384804 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.384075  5576 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:37.347640  5500 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.395511 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.356520 32118 rpcz_store.cc:276] 0709 12:19:35.427481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.427633 (+   152us) service_pool.cc:224] Handling call
0709 12:19:35.427963 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.428307 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.428334 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.428773 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.784898 (+356125us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:35.784948 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.785037 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:37.329995 (+1544958us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:37.355631 (+ 25636us) raft_consensus.cc:1685] finished
0709 12:19:37.355650 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.355888 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6464,"raft.queue_time_us":23633,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":981,"spinlock_wait_cycles":36491648,"thread_start_us":355692,"threads_started":1,"wal-append.queue_time_us":470251}
I20260709 12:19:37.369722  5580 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.408861  5574 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.400309 32114 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:37.410842 31965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
W20260709 12:19:37.415297 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.400974 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.397367 32044 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:19:37.416898 31972 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.427279  5580 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.427955 32121 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:37.435199 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:37.437368 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.437793 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:37.438937 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.439247  5480 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.443070  5480 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.443135 32039 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:19:37.439937 31966 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:19:37.440658  5582 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.442054 32118 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:19:37.453784 32046 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:37.395218  5576 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:37.452459 32118 pending_rounds.cc:85] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:19:37.443619 32039 pending_rounds.cc:85] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 13
I20260709 12:19:37.452514 31966 pending_rounds.cc:85] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 13
I20260709 12:19:37.461134 32039 pending_rounds.cc:107] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:19:37.461232 31966 pending_rounds.cc:107] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:19:37.461555 32115 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:37.461688 31966 raft_consensus.cc:2889] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:37.448733  5498 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:37.465338  5498 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:37.465961  5576 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:37.466667  5496 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:37.467372  5496 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.468044  5417 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.443594  5480 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.471045  5496 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.461592 32039 raft_consensus.cc:2889] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:37.474393 32118 pending_rounds.cc:107] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:19:37.474936 32118 raft_consensus.cc:2889] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:37.482762 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.485337 31964 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 12:19:37.487233 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 190983us
I20260709 12:19:37.491315 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2644) took 1593 ms. Trace:
I20260709 12:19:37.491663 31964 rpcz_store.cc:276] 0709 12:19:35.897531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.286872 (+1389341us) service_pool.cc:224] Handling call
0709 12:19:37.491254 (+204382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.500346 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.502710 31964 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.508409  5502 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:37.511099 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.513180 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2427) took 2602 ms. Trace:
I20260709 12:19:37.513731 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:37.517056 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.517820  5582 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.518378  5582 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.519273 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 863) took 2062 ms. Trace:
I20260709 12:19:37.519693 32655 rpcz_store.cc:276] 0709 12:19:35.457254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.457619 (+   365us) service_pool.cc:224] Handling call
0709 12:19:35.458040 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.577425 (+119385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.577472 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:35.577535 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:35.577581 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.518819 (+1941238us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.518881 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.519093 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273817344}
I20260709 12:19:37.520964  5582 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.521713 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:37.524175  5580 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:37.524412  5252 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.386s	user 0.012s	sys 0.009s
I20260709 12:19:37.524952  5252 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.527870  5252 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:37.528064 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.528757 32117 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:37.530431 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:37.531418 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2436) took 2589 ms. Trace:
I20260709 12:19:37.536912 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1067) took 2829 ms. Trace:
I20260709 12:19:37.539795  5335 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.541497 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.541894 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:37.557483 32652 rpcz_store.cc:276] 0709 12:19:34.707488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.708832 (+  1344us) service_pool.cc:224] Handling call
0709 12:19:35.412098 (+703266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.653797 (+241699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.534874 (+1881077us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.536358 (+  1484us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b540061e750. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:37.536643 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170706048}
I20260709 12:19:37.541508 32044 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:37.542644 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.543566 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.543911 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.544582 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.545048 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.545542 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.546051 32042 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 17. Ops: []
I20260709 12:19:37.546198  5574 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.552774 31967 rpcz_store.cc:276] 0709 12:19:34.942230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.607636 (+665406us) service_pool.cc:224] Handling call
0709 12:19:35.607901 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.781978 (+174077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.530959 (+1748981us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400413a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.531244 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336932736}
I20260709 12:19:37.504930 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 718) took 1829 ms. Trace:
I20260709 12:19:37.571458 31966 rpcz_store.cc:276] 0709 12:19:35.669430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.374043 (+1704613us) service_pool.cc:224] Handling call
0709 12:19:37.439536 (+ 65493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.496524 (+ 56988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.498601 (+  2077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.513609 31971 rpcz_store.cc:276] 0709 12:19:34.910787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.520004 (+609217us) service_pool.cc:224] Handling call
0709 12:19:35.520285 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.019320 (+1499035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.512852 (+493532us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400068454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.513112 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3445096704}
I20260709 12:19:37.546581 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.572381 32658 raft_consensus.cc:2410] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:19:37.569789 32656 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:37.581322 31972 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:37.585013 31965 raft_consensus.cc:606] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:37.585489 31965 raft_consensus.cc:614] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:37.585522 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.518085  5500 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.587970 32041 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:37.593592 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.594563 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.596690 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.575287  5560 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:37.601586 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.598106 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.603623 31972 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.604102 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.604890 32118 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:37.606102 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.602548 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:37.614529 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.616844 32040 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.619658 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:37.619910 32045 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.621012 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.621186 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:37.624325 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:37.624439 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103616us
I20260709 12:19:37.497627  5498 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.618204 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.601332  5560 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.613544  5532 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.629840 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.631637 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.633666 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.628548 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.637333 32045 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:37.641691 32042 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:37.646778 32041 raft_consensus.cc:2368] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19.
I20260709 12:19:37.624150  5252 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:37.652493 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.650292 31967 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:37.633109  5560 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.653967 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.656672 32044 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:37.614838  5580 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:19:37.659354 31966 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:37.662986 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.611958  5574 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.666638 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.666553 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.666203 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.668016 32042 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.665936 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.672101 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.672318 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.649885  5362 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:37.629359  5500 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.674583 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.675918 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.676870 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:37.677325  5575 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:37.677757  5575 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:37.678028 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.678876  5575 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.679661 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.680047 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:37.680429  5575 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:37.680612 32043 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.683509  5499 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:37.683986 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2706) took 1552 ms. Trace:
I20260709 12:19:37.684223  5577 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:19:37.684743 32045 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:37.685007 31967 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:37.685277  5341 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.685772 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.686971 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2440) took 2015 ms. Trace:
I20260709 12:19:37.687079 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.687477 31967 rpcz_store.cc:276] 0709 12:19:35.671489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.685878 (+2014389us) service_pool.cc:224] Handling call
0709 12:19:37.686276 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.686557 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.686589 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.686657 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:37.686692 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.686751 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.686890 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.688378 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.689091 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:37.689338 31967 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:37.690119 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2441) took 1988 ms. Trace:
I20260709 12:19:37.690605 31967 rpcz_store.cc:276] 0709 12:19:35.702218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.688558 (+1986340us) service_pool.cc:224] Handling call
0709 12:19:37.689037 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:37.689831 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.690064 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.691929 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.692955 32114 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:37.692807 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2631) took 1824 ms. Trace:
W20260709 12:19:37.695142  5597 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:19:37.697690 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.697324 31967 rpcz_store.cc:276] 0709 12:19:35.868064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.691420 (+1823356us) service_pool.cc:224] Handling call
0709 12:19:37.691955 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.692355 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.692387 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.692457 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:37.692546 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.692617 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.692747 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.700120 31967 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: [16.10-17.11]
I20260709 12:19:37.700745 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2641) took 1814 ms. Trace:
I20260709 12:19:37.701134 31967 rpcz_store.cc:276] 0709 12:19:35.886348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.699365 (+1813017us) service_pool.cc:224] Handling call
0709 12:19:37.699795 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:37.700504 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.700699 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.675264  5520 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:37.704970  5084 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.705571  5520 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.705827  5084 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.706875 32118 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 12:19:37.675449  5417 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.708889  5417 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.709213  5084 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.710940  5417 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.711524  5417 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.711635  5448 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.702749 31967 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:37.712105  5417 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.712211  5448 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.712793 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2642) took 1818 ms. Trace:
I20260709 12:19:37.713243 31967 rpcz_store.cc:276] 0709 12:19:35.894721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.702007 (+1807286us) service_pool.cc:224] Handling call
0709 12:19:37.702353 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.712460 (+ 10107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.712717 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:37.684470 32119 rpcz_store.cc:276] 0709 12:19:36.136178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.142452 (+1006274us) service_pool.cc:224] Handling call
0709 12:19:37.683762 (+541310us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b54002b5cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:37.683891 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181068672}
I20260709 12:19:37.720877 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.670450 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:37.727259 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.690040 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.700690 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46882 (request call id 28) took 1854 ms. Trace:
I20260709 12:19:37.728601 31965 rpcz_store.cc:276] 0709 12:19:35.867677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.377541 (+1509864us) service_pool.cc:224] Handling call
0709 12:19:37.700508 (+322967us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400312754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1c9853 00007f229119760a 00007f228a71d7d4 00007f228a71d762 00007f228a71d691 00007f228a71d60c 00007f228a71c8c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:37.700609 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372814720}
I20260709 12:19:37.737210 32044 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.739374 31972 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:37.741045 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.744992 32045 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:37.745697  5524 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.751521 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2643) took 1854 ms. Trace:
I20260709 12:19:37.754619 32040 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.762729 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.684026  5499 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.759873  5448 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.767424 32042 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
W20260709 12:19:37.768919 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 114274us
I20260709 12:19:37.744499  5084 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.782845 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.787781  5269 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.812471  5269 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:37.791337 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 141175us
I20260709 12:19:37.782903 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.815109 32040 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:37.791899  5389 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:37.735823  5417 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.814987 32121 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:37.794768 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:37.796437  5569 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.797580  5480 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.800527 32041 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:37.803707 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.777890  5576 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.809700 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:37.810343  5084 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.811776  5520 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.851477 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:37.817431 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54624.3us
I20260709 12:19:37.818272 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 738) took 2776 ms. Trace:
I20260709 12:19:37.806792  5532 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:37.827939  5596 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:19:37.832813  5574 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.805312  5499 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:37.834242  5500 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.834892 31965 raft_consensus.cc:2410] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:19:37.845854 31964 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 13.
I20260709 12:19:37.809351 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.814002  5524 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.857182 32039 rpcz_store.cc:276] 0709 12:19:35.041670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.042153 (+   483us) service_pool.cc:224] Handling call
0709 12:19:35.061429 (+ 19276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.472003 (+2410574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.814504 (+342501us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54001941d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:37.818194 (+  3690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5475993984}
I20260709 12:19:37.876818 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:37.883873 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.884553 32040 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:19:37.849047  5399 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:37.887866 32038 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:37.893502 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 83945.7us
I20260709 12:19:37.893366  5524 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.896322  5580 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:37.881021  5417 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.897284 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.897529  5417 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.899959  5417 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:37.900427 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44288us
I20260709 12:19:37.895848  5524 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:37.882346 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:37.910621 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:37.911201 31965 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.914120 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.914361 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.912820 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:37.850446  5569 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.900460  5226 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:37.920100 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:37.886647  5389 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:37.891359  5399 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.927033  5001 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:37.854350  5502 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:19:37.927428  5527 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:37.930300 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.906636  5335 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:37.948498 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:37.931119 32114 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:37.906193  5362 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:19:37.950950 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:37.922775  5226 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.951911 32112 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.928411  5399 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.952745  5226 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:37.957931  5389 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.929734  5527 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:37.963132  5527 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.964289  5527 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.964426  5560 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:37.964921  5498 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:37.927488  5001 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:37.966598  5001 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.970664 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.972033 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.972677 31972 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:37.973698 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.974539 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.975136  5445 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.976315  5590 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:37.976943  5590 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:37.949218 32042 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:37.979197 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.979387 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.980158 32042 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.980607 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:37.980916 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.981356 32039 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:37.975190 32040 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:37.981686 32042 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:37.913400  5580 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 12:19:37.983654 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.982266 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:37.984102 32041 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:37.979288 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:37.942595  5252 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.990727 31968 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:37.989966 32039 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:37.992367 31965 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:37.996011 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2647) took 1877 ms. Trace:
I20260709 12:19:37.999069 31966 rpcz_store.cc:276] 0709 12:19:36.107189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.743093 (+1635904us) service_pool.cc:224] Handling call
0709 12:19:37.783535 (+ 40442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:37.812074 (+ 28539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.812107 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:37.812662 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:19:37.944423 (+131761us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:37.944473 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.944550 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:37.984533 (+ 39983us) raft_consensus.cc:1685] finished
0709 12:19:37.984551 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.984682 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19076,"raft.queue_time_us":30063,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4200,"thread_start_us":131231,"threads_started":1,"wal-append.queue_time_us":168568}
I20260709 12:19:38.002595  5497 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.002777 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.002949 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.011137 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:38.002140  5575 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:19:38.016079  5480 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.016993  5226 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.019722 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.020066  5425 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.020291 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.021464 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:38.022200 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.022284  5417 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:38.023073  5417 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.017371 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:38.024365 32116 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2700) took 1948 ms (client timeout 2139 ms). Trace:
W20260709 12:19:38.024835 32116 rpcz_store.cc:269] 0709 12:19:36.075550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.238429 (+162879us) service_pool.cc:224] Handling call
0709 12:19:38.024161 (+1785732us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54004394d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:38.024311 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544638336}
I20260709 12:19:38.025812 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.004289  5560 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:38.026523  5571 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.027781 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:38.028380 31964 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.029651 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.022964  5310 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.033789 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.034797 31972 raft_consensus.cc:2410] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:19:38.034929 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.035185  5226 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:38.034981 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.037144 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 720) took 2367 ms. Trace:
I20260709 12:19:38.037606  5524 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:38.063956 31969 rpcz_store.cc:276] 0709 12:19:35.669932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.373006 (+1703074us) service_pool.cc:224] Handling call
0709 12:19:37.690453 (+317447us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b5800003698. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaeee 000000000036eb39 00007f229111e1e0 00007f22911e70c4 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:19:37.740343 (+ 49890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:37.751093 (+ 10750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.751125 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:37.751692 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:19:37.901885 (+150193us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:37.901922 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.901992 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.036854 (+134862us) raft_consensus.cc:1685] finished
0709 12:19:38.036871 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.037069 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9776,"raft.queue_time_us":9945,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":31116,"spinlock_wait_cycles":8566144,"thread_start_us":149808,"threads_started":1,"wal-append.queue_time_us":258519}
I20260709 12:19:38.039249 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.042795  5576 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.042953 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.044927  5590 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.028577  5252 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.052117  5084 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.053735 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.054672 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.063433 31965 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:38.067719 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.085157 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.085947 31966 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:38.086966 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.087651 31966 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:38.093421 31964 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:38.090829 31968 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:38.062115  5496 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.100060 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.066782  5571 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:38.112237  5500 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:38.094832  5582 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:38.117590 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.033303  5417 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.124994 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:38.128013 31970 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.075189  5524 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.122820 32041 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:38.095175  5310 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:38.134496 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 218882us
W20260709 12:19:38.137477 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 117875us
W20260709 12:19:38.136847 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:38.141306 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.142652 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2650) took 1330 ms. Trace:
I20260709 12:19:38.143157 31968 rpcz_store.cc:276] 0709 12:19:36.812336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.097885 (+1285549us) service_pool.cc:224] Handling call
0709 12:19:38.134765 (+ 36880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.142158 (+  7393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.142200 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:38.142262 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:38.142350 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.142427 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.142581 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:38.151551 31969 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:38.126760  5582 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.140585 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.135596  5524 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.183969  5577 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:38.184500 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.184942 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.185654 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:38.188236 31969 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:38.196089 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2713) took 1435 ms. Trace:
I20260709 12:19:38.197698  5576 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.241693 32120 rpcz_store.cc:276] 0709 12:19:36.760580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.067068 (+306488us) service_pool.cc:224] Handling call
0709 12:19:38.195860 (+1128792us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540010efd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:38.196020 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501179136}
I20260709 12:19:38.208134  5252 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:37.751986 31967 rpcz_store.cc:276] 0709 12:19:35.897198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.714219 (+1817021us) service_pool.cc:224] Handling call
0709 12:19:37.717393 (+  3174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.751072 (+ 33679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.751109 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.751177 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:37.751232 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.751307 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.751438 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2826}
W20260709 12:19:38.215978 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2639) took 2339 ms (client timeout 2077 ms). Trace:
I20260709 12:19:38.217470 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:38.220021 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 989634us
I20260709 12:19:38.221900  5269 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.227012  5399 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.229275  5518 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.228950  5001 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.132309  5571 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.163136 32114 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
W20260709 12:19:38.235229 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 128914us
I20260709 12:19:38.131620  5499 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.236222 32116 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:38.237320 32657 raft_consensus.cc:2410] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:19:38.245579 31970 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:38.258886 31972 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 12:19:38.246294 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 83428.1us
I20260709 12:19:38.248282 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.260394 32659 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
W20260709 12:19:38.249282 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.261390 31970 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:38.137737  5310 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:38.265393 31967 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:38.262836 32121 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:38.265693 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:38.238575 32117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2709) took 1819 ms (client timeout 2077 ms). Trace:
W20260709 12:19:38.254283 31963 rpcz_store.cc:269] 0709 12:19:35.876147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.008186 (+1132039us) service_pool.cc:224] Handling call
0709 12:19:38.215735 (+1207549us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54001ceb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:38.215866 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25154816}
I20260709 12:19:38.205976  5569 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:38.250690 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 66302.7us
I20260709 12:19:38.264056 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.273478 32659 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:19:38.267369 32117 rpcz_store.cc:269] 0709 12:19:36.419206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.792076 (+372870us) service_pool.cc:224] Handling call
0709 12:19:38.238293 (+1446217us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:38.238497 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690467584}
I20260709 12:19:38.275183 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.278437 32119 raft_consensus.cc:2410] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 12:19:38.278852 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.275475 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:38.276556 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.280946 32043 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:38.281608 32044 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:38.282815 32121 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:38.284665 32656 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:38.234213  5582 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.288722 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.270750  5310 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:38.293479 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.296379 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.297309 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.298259 32119 raft_consensus.cc:2410] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:38.291702 32117 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:38.298156 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.302618 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.299218 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.305310 32114 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:38.300889 31968 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:38.320533  5498 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.321079  5499 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.321277  5616 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:38.321728  5616 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:38.322136  5616 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.326503 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.328100 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.329437 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.331013 31968 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:38.330858  5436 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:38.331745  5527 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:38.332187  5527 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:38.333217  5527 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.334852  5527 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:38.339360  5399 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.341236 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.341945 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:38.344224  5520 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.306265  5252 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.348021 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.283931  5389 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:19:38.261037  5518 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.372623 32117 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:38.372810 32047 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:38.373935  5617 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.359064  5001 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:38.349449  5520 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.299860  5497 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:19:38.365880  5524 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.386143 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.386397  5524 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.366969 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.387279  5165 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:38.388027 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.388917 31935 leader_election.cc:400] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19.
I20260709 12:19:38.389288 31935 leader_election.cc:403] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:19:38.389993  5527 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.391260 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.392642 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.393447  5441 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19.
I20260709 12:19:38.393955 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.394588  5616 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.308297  5576 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.377192 32653 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:38.377364 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2448) took 1093 ms. Trace:
I20260709 12:19:38.377780  5222 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:38.378451 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.379417 32658 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:38.352836  5499 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.381546 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.389153  5522 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.396821 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.401872 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:38.368363  5475 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.368621  5582 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.402747  5524 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.440292 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.440696 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.441844 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.442101 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:38.442787 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.443110  5518 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.403818  5527 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.413749 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:38.454391 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.411993 31971 rpcz_store.cc:276] 0709 12:19:37.285918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.109774 (+823856us) service_pool.cc:224] Handling call
0709 12:19:38.110143 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:38.110633 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.110668 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.111223 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.282745 (+171522us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:38.282787 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.282856 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.377077 (+ 94221us) raft_consensus.cc:1685] finished
0709 12:19:38.377110 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.377273 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2476,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":171115,"threads_started":1,"wal-append.queue_time_us":264121}
I20260709 12:19:38.419127  5576 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.421065 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.355258  5498 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:38.430624  5001 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:38.342484  5480 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 12:19:38.434232 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:38.435874 32656 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:38.437485  5341 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:38.438090 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.471946 32658 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:38.438419 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.440941 32651 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
W20260709 12:19:38.356232 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53651.5us
I20260709 12:19:38.456179 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.431075  5441 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:38.475427 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.477137 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.455586  5475 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:38.467592  5522 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.457221  5498 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:38.506090 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:38.468221  5368 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:38.473610  5001 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.502103 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:38.509054 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 102557us
I20260709 12:19:38.513129 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:38.488624  5222 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:38.475404  5341 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:38.523489 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.526064 32046 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:38.524271 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.476011  5441 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:38.481827  5576 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:38.527701 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 106715us
I20260709 12:19:38.482597  5587 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:38.533674 32039 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
W20260709 12:19:38.533763 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93239us
I20260709 12:19:38.535033 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:38.536720 31968 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:38.529120 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:38.503180  5475 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.470402  5425 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:19:38.508762  5518 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.505856  5499 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:38.470196  5524 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 12:19:38.554188 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.520484  5498 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:38.525408  5001 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.519548  5368 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:38.586511  5368 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.543929 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:38.560819  5252 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.597159  5582 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.598996  5571 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.599623  5571 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:38.599946  5571 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.600255  5571 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:38.534566  5576 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:38.533098  5436 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:38.561229 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.562865 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.553117  5437 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:38.610925 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48168.4us
I20260709 12:19:38.564604  5618 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:19:38.565366  5499 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:38.566484  5616 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.547223  5475 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.547284  5502 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:38.581120  5269 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:38.627737 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.583129  5341 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.535025  5441 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.560377  5617 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:19:38.512714  5522 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.611649 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:38.618512 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.615734 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:38.595319 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51486.5us
W20260709 12:19:38.620599 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 66583.3us
I20260709 12:19:38.656669 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.658634 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:38.649662  5368 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:38.664999 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53428.5us
I20260709 12:19:38.663045  5527 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.664076 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:38.672348 32118 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.677182 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:38.680291 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.674099 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.683168 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:38.686928 32113 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:38.624954 32046 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:38.607776  5576 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.629304 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:38.645462 32115 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:38.646099 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:38.614215  5437 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.690743 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.691573 32039 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:38.690537 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60048.6us
I20260709 12:19:38.692540 32112 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:38.645462  5617 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:38.650946  5341 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:38.651675 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.652824  5310 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.655891  5502 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:38.704902 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.705065 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:38.707011 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.675261  5252 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:38.741833 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:38.711015 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.677084  5616 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:19:38.714426 32047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2438) took 3240 ms (client timeout 3370 ms). Trace:
I20260709 12:19:38.743721 32044 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:38.744825 32038 raft_consensus.cc:2368] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:38.749015 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:38.751187 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77336.6us
W20260709 12:19:38.746650 32047 rpcz_store.cc:269] 0709 12:19:35.454381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.604821 (+150440us) service_pool.cc:224] Handling call
0709 12:19:38.674908 (+3070087us) spinlock_profiling.cc:243] Waited 2.77 s on lock 0x7b54000348d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:38.695173 (+ 20265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6362455296}
I20260709 12:19:38.757254 31965 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:38.708819  5437 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:38.709626  5617 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:38.767758  5310 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.769176 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2447) took 1567 ms. Trace:
I20260709 12:19:38.769588 31964 rpcz_store.cc:276] 0709 12:19:37.202716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.121309 (+918593us) service_pool.cc:224] Handling call
0709 12:19:38.123411 (+  2102us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:38.138824 (+ 15413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.138873 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:38.162688 (+ 23815us) log.cc:844] Serialized 52 byte log entry
0709 12:19:38.512964 (+350276us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:38.532012 (+ 19048us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.532233 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.532311 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.768959 (+236648us) raft_consensus.cc:1685] finished
0709 12:19:38.768981 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.769119 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18965,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":6877,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":21417,"thread_start_us":230413,"threads_started":1,"wal-append.queue_time_us":484207}
I20260709 12:19:38.742101  5335 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:38.746274  5527 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:38.691486  5587 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:19:38.782747  5528 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:38.787949  5436 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:38.788376  5436 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:38.789391  5436 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.777386  5335 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.777892  5527 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.757467  5252 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:38.791862 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.830538 32042 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:38.791801  5502 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.778172  5499 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.824988 32039 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:38.832209 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.845180 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.833845 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:38.838217  5368 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.841056  5222 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.830077  5335 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:38.820340  5582 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.830986  5527 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:38.863271 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.864087 32039 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:38.864880 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.865294  5522 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:38.866456  5522 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:38.867381  5522 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:38.867837  5522 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:38.868659  5631 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.132s	user 0.003s	sys 0.000s
I20260709 12:19:38.868765  5522 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:38.911469 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 235591us
I20260709 12:19:38.920583 32118 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:38.926465 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2658) took 1022 ms. Trace:
I20260709 12:19:38.927232 31970 rpcz_store.cc:276] 0709 12:19:37.903797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.291854 (+388057us) service_pool.cc:224] Handling call
0709 12:19:38.307855 (+ 16001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.344159 (+ 36304us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:38.660209 (+316050us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.926107 (+265898us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:38.926160 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:38.926201 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.926268 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.926408 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5446,"prepare.queue_time_us":71778,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"thread_start_us":486509,"threads_started":2}
I20260709 12:19:38.869855  5617 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.932516 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.933363 32047 raft_consensus.cc:2368] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:38.935289  5254 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:38.935885  5254 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.937304  5571 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.921947  5480 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:38.938633  5480 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:38.939518  5480 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.939898  5335 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:38.940796  5480 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.871614  5582 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:38.943637  5252 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:38.946871  5630 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:19:38.949244 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:38.950619 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.951769 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:38.953450  5633 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:19:38.957983 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.942062  5335 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:38.981915 32042 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:38.958621  5165 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:38.946803 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:38.965287 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2648) took 2842 ms. Trace:
I20260709 12:19:38.984534 31966 rpcz_store.cc:276] 0709 12:19:36.123006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.088361 (+1965355us) service_pool.cc:224] Handling call
0709 12:19:38.088793 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:38.089655 (+   862us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:38.304370 (+214715us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.618639 (+314269us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.619412 (+   773us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.626028 (+  6616us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.626166 (+   138us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:38.630561 (+  4395us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.634481 (+  3920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.634563 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.829864 (+195301us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b18003562c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:38.952806 (+122942us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:38.965014 (+ 12208us) raft_consensus.cc:1685] finished
0709 12:19:38.965055 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.965217 (+   162us) inbound_call.cc:177] Queueing success response
0709 12:19:38.981551 (+ 16334us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18003562c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
Metrics: {"mutex_wait_us":61781,"prepare.queue_time_us":342044,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":7739,"raft.queue_time_us":9580,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":28350,"spinlock_wait_cycles":34051072,"thread_start_us":470999,"threads_started":2,"wal-append.queue_time_us":458396}
I20260709 12:19:38.984079 32047 raft_consensus.cc:2376] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17. Re-sending same reply.
I20260709 12:19:38.991118 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:38.992177 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 340550us
I20260709 12:19:38.993121 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:38.993541 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.014947 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:39.012306  5524 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:39.020364 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:39.021144 32047 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.021879 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:39.022653 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
W20260709 12:19:39.030479 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 353494us
I20260709 12:19:39.030920 32039 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:39.032424 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2649) took 2237 ms. Trace:
W20260709 12:19:39.036077 32116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.010s	sys 0.007s
W20260709 12:19:39.036412 32116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.010s	sys 0.007s
I20260709 12:19:39.036258 31965 rpcz_store.cc:276] 0709 12:19:36.794823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.094303 (+1299480us) service_pool.cc:224] Handling call
0709 12:19:38.136598 (+ 42295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.757838 (+621240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.032124 (+274286us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54000b6654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:39.032351 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426720128}
I20260709 12:19:38.983387  5165 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:39.034126 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:39.014146  5587 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:39.038997 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.039477 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:39.040182 32039 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:39.027539 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:39.035132 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.048746 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:39.029943  5425 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:39.054419 32042 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:39.032398  5084 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:39.057523 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 435134us
I20260709 12:19:39.045392  5569 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:39.075516 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:39.050287  5368 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.079568 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:39.054296  5616 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:39.072392 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.056077  5524 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:39.083171 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.086266 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:39.086885 32656 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:39.086990 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:39.055562  5587 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:39.098426 31966 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:39.066160  5582 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.099861 32042 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:39.054940  5475 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.103374  5475 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.104118  5475 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.104630  5475 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.078240  5335 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:19:39.108779  5335 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:39.109163  5335 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:39.109572  5335 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:39.110833  5640 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:19:39.111023  5362 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:39.112066  5362 raft_consensus.cc:2764] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:39.086261  5524 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.113642  5524 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:39.073135  5425 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:39.131541  5425 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.134732  5569 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:39.135473  5569 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:39.119475  5571 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.109176  5575 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:39.123111  5001 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.007s
I20260709 12:19:39.124958  5475 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.127121  5441 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.074785  5576 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.129142  5254 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.129379  5335 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.144467 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:39.140295  5165 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:39.149148 31963 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.009s	sys 0.029s
W20260709 12:19:39.160120 31963 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.029s
I20260709 12:19:39.162391 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.157008 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.162699 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.163095 32121 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.158351 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:39.148237  5575 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.170655 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.171268 32118 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:39.176939 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 408729us
I20260709 12:19:39.177770 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:39.180264 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.183598 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.186013 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:39.190846  5498 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.562s	user 0.016s	sys 0.001s
I20260709 12:19:39.152954  5001 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.171015  5575 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.217152 32118 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:39.217736 32656 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
W20260709 12:19:39.278146 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 444344us
W20260709 12:19:39.278708 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.018s
I20260709 12:19:39.277384 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:39.280448 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.000s	sys 0.018s
W20260709 12:19:39.216359 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.616s (SENT) [suppressed 25 similar messages]
I20260709 12:19:39.272819  5165 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:39.306370 32659 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:39.154873  5587 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.309069 32660 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:39.196084  5479 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
W20260709 12:19:39.309762 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.309780 32656 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
W20260709 12:19:39.311281 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 224337us
I20260709 12:19:39.305281  5416 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.284933  5254 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.285522  5654 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:39.275043  5498 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.390162  5222 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:39.397588 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:39.399313 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.401612 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.397523  5222 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:39.376415  5341 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.409222 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:39.418438 32043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.009s	sys 0.029s
W20260709 12:19:39.422168 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.396497  5654 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:39.456599 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47497us
W20260709 12:19:39.382472 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 62301.8us
I20260709 12:19:39.313688 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.407377  5252 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:39.461712 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 245355us
I20260709 12:19:39.408138  5647 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.391813  5532 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:39.388465  5254 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.412298  5518 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:39.419134  5576 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:39.423146  5359 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:39.456640  5527 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
I20260709 12:19:39.418200  5655 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:39.475956  5638 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:19:39.415673  5341 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.507609 32047 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:39.474205  5441 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:39.544580 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 87056.4us
I20260709 12:19:39.484699  5663 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:39.485108  5518 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.485150  5335 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.483497  5527 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:39.549031  5644 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
W20260709 12:19:39.549157  5437 consensus_meta.cc:287] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.779s	user 0.010s	sys 0.005s
I20260709 12:19:39.559134 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 914) took 1525 ms. Trace:
W20260709 12:19:39.565310 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 258858us
I20260709 12:19:39.560406  5663 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:39.556681  5362 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:19:39.483681  5222 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.563570 32116 rpcz_store.cc:276] 0709 12:19:38.033553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.293798 (+260245us) service_pool.cc:224] Handling call
0709 12:19:38.302876 (+  9078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:38.355871 (+ 52995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.355918 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.444745 (+ 88827us) log.cc:844] Serialized 26 byte log entry
0709 12:19:39.036845 (+592100us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:39.036884 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.039358 (+  2474us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.558937 (+519579us) raft_consensus.cc:1685] finished
0709 12:19:39.558959 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.559081 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41897,"raft.queue_time_us":62232,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":156,"thread_start_us":511971,"threads_started":1,"wal-append.queue_time_us":692585}
I20260709 12:19:39.577843 32013 leader_election.cc:400] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:39.580045 32013 leader_election.cc:403] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:39.549015  5659 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:39.579690 32044 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 12:19:39.581506  5657 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.102s	user 0.001s	sys 0.001s
I20260709 12:19:39.582024 32039 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:19:39.582474 32039 pending_rounds.cc:85] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 7
I20260709 12:19:39.582698 32039 pending_rounds.cc:107] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:19:39.582907 32039 raft_consensus.cc:2889] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:39.583830  5518 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.584972  5341 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.575253  5437 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.586946  5527 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:39.588012  5660 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:19:39.588840  5437 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.590983 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47552 (request call id 1000) took 1142 ms. Trace:
I20260709 12:19:39.591926 32121 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
W20260709 12:19:39.590260  5642 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:19:39.591413 32660 rpcz_store.cc:276] 0709 12:19:38.436065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.437965 (+  1900us) service_pool.cc:224] Handling call
0709 12:19:39.578367 (+1140402us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54004c13d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:39.578518 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915427200}
W20260709 12:19:39.596351  5662 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 12:19:39.597144  5641 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:19:39.597338  5222 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:39.600759 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2657) took 1711 ms. Trace:
I20260709 12:19:39.601111  5571 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:39.601001 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 442715us
I20260709 12:19:39.601636  5659 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:39.601877 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 424150us
I20260709 12:19:39.618981 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:39.602361  5084 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.603087  5480 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:39.488767  5655 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.560829  5652 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:39.610054 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.576380  5252 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:39.641520 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.616317 31963 rpcz_store.cc:276] 0709 12:19:37.889085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.282244 (+393159us) service_pool.cc:224] Handling call
0709 12:19:38.282539 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.297034 (+ 14495us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:19:38.523966 (+226932us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.161078 (+637112us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.599682 (+438604us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b180043fa20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:19:39.600108 (+   426us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.600293 (+   185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:39.600339 (+    46us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:19:39.600391 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.600472 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.600694 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428645,"prepare.queue_time_us":122326,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":439514,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":2997,"spinlock_wait_cycles":15900672,"thread_start_us":759402,"threads_started":2,"wal-append.queue_time_us":775391}
W20260709 12:19:39.622445  5656 log.cc:927] Time spent T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:19:39.604995  5498 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:39.626753 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:39.627007  5665 log.cc:927] Time spent T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:19:39.627595 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.662410 31966 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:39.628631  5368 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.628410  5663 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.629132  5441 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:39.630038  5524 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.005s
I20260709 12:19:39.630601 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:39.628340 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.605378  5532 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
W20260709 12:19:39.654345 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 76764us
I20260709 12:19:39.656635 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:39.660432  5502 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:39.665019 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 84951us
I20260709 12:19:39.657816  5497 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.640288  5480 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:39.661103  5644 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:39.682348 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 739) took 1183 ms. Trace:
I20260709 12:19:39.682726 31968 rpcz_store.cc:276] 0709 12:19:38.498527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.552022 (+ 53495us) service_pool.cc:224] Handling call
0709 12:19:38.552579 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:38.554647 (+  2068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.554678 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.555395 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.989239 (+433844us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:39.028746 (+ 39507us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.030670 (+  1924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.030745 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.108137 (+ 77392us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b1800352540. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:39.682038 (+573901us) raft_consensus.cc:1685] finished
0709 12:19:39.682145 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.682273 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3002,"prepare.queue_time_us":43079,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":7014,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":25590,"spinlock_wait_cycles":6505344,"thread_start_us":358362,"threads_started":1,"wal-append.queue_time_us":544910}
I20260709 12:19:39.685377  5480 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.686872  5480 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:39.687908 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:39.690550  5429 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.691596  5429 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:39.689932 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:39.696512  5084 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:39.697072  5084 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:39.697644  5084 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.654915  5571 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:39.702355  5659 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.703025  5571 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.653538  5425 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.704653  5425 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.706739  5425 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.707441  5575 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.709717  5575 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:39.710529 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:39.711109  5252 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.711635  5425 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:39.711813  5497 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.713173 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:39.713944  5497 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:39.661664  5310 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.670223  5368 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.681075 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 771) took 1184 ms. Trace:
W20260709 12:19:39.719367 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.714618 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:39.676275  5441 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.579278  5654 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:39.721027 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46165.9us
I20260709 12:19:39.724893 32084 leader_election.cc:400] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:39.725246 32084 leader_election.cc:403] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:39.726245 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.726653 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.729045 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.729621 32118 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.718223 32041 rpcz_store.cc:276] 0709 12:19:38.497034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.497193 (+   159us) service_pool.cc:224] Handling call
0709 12:19:38.512751 (+ 15558us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:38.556258 (+ 43507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.556297 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:38.564137 (+  7840us) log.cc:844] Serialized 52 byte log entry
0709 12:19:38.911224 (+347087us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:38.969450 (+ 58226us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.969694 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.969788 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.463559 (+493771us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b18004972c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:39.633180 (+169621us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b18004972c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:39.680822 (+ 47642us) raft_consensus.cc:1685] finished
0709 12:19:39.680847 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.680998 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":57975,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":30069,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":22552,"spinlock_wait_cycles":46465664,"thread_start_us":277872,"threads_started":1,"wal-append.queue_time_us":400006}
I20260709 12:19:39.721617 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:39.739578 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.740170 32116 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:39.742723  5648 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:39.744769  5084 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.745070  5650 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:39.757459 32118 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:39.761469  5335 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:39.763567  5368 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.750608  5649 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.763958  5084 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.764592  5649 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.713940  5425 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:39.739464  5569 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.002s
I20260709 12:19:39.675192  5524 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.735080  5527 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.724895  5582 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:39.735248  5429 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:39.786748  5502 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:39.781488  5310 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:39.807627 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 83526.7us
I20260709 12:19:39.808733 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:39.809553 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.813587 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:39.812232 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90698.9us
I20260709 12:19:39.814706 31968 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:39.816466 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.817471 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.829555 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.828997 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.830505 31970 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:39.817831 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:39.831359 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.785053  5569 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.834474 31964 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:39.832304 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:39.838192 32121 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:39.820551  5310 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:39.839843 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.839983 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.841747 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.844751 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.848047  5445 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.864074 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:39.851147  5441 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.852308  5524 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.908849 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.909513 32119 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.853101  5436 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.854075  5649 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.820099  5425 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:39.859757  5569 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.914747  5425 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:39.860762 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.915699  5681 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:39.916260  5681 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.916594 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.917624 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.861541 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.918471  5681 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.919816 31964 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:39.844815 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.815421  5527 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.866400 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.814568  5648 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:39.866827 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.929948 32117 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:39.800743  5582 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.904006 31971 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:19:39.848356 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.921378 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.934723 32114 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:39.905521 31968 raft_consensus.cc:2376] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19. Re-sending same reply.
I20260709 12:19:39.836006  5429 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:39.936467 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.940065 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:39.921231 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.942003 31965 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:39.928388 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:39.943248 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.944110 31964 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:39.938004 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.837349  5335 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.949815 31972 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:39.952863 32658 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:39.959280 31970 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.9-21.9]
I20260709 12:19:39.961424 32112 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:39.964473 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.965405 31964 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.918982  5681 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.954391  5429 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.968034 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:39.903854  5437 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:39.989789  5437 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:39.991147  5437 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.969974 32120 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:39.957538 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:39.963552 31968 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:39.975132  5681 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.975585 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:39.915701  5425 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.975167  5429 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.997620 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.000339 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.002903 32653 raft_consensus.cc:2410] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:19:40.003662  5524 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:40.004127  5524 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.004484  5524 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:39.928814  5479 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:40.005898  5479 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.976553 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.006260  5479 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.006568  5681 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.007232 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.007696 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.008292 32038 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:40.008333 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.009001 32047 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:40.009232 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.009829 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:39.995464  5648 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.979496 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:39.810827  5502 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:39.994602  5527 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:39.980310 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 145555us
I20260709 12:19:39.982558  5582 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:39.986331 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:39.986606  5001 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:39.967471  5445 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:39.988375 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.037900 32039 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:39.978163 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:39.999979 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:40.038594 32659 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.000797 32651 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:40.001833 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:39.978768 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.010782 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.040302 32657 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:40.010295 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.015475 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:40.018649 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.043845 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.044844 32116 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:40.045770 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.018930 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.019263 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.012851 32046 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:40.020941 32040 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.019219 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.020493 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.019129 31968 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:40.054824 32653 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:40.023242 31967 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:40.054924  5528 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:40.017587 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.055622 32120 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:40.031837 32119 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:40.056835 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.039448 31969 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:40.037011 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:40.079038  5429 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.082448  5429 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:40.086890  5679 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:19:40.082482  5310 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.070678 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:40.068517  5496 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.056s	sys 0.080s
I20260709 12:19:40.101819  5425 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.072690 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:40.076267 31972 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:40.103034 31969 raft_consensus.cc:2368] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:40.100165 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:40.101953  5496 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.057s	sys 0.081s
I20260709 12:19:40.072120 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.103502 32119 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.109448 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.108108 32112 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:40.110423 32117 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:40.117705 32119 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:40.118726 31972 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:40.119434 31972 raft_consensus.cc:606] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:40.119901 31972 raft_consensus.cc:614] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.072882  5582 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
W20260709 12:19:40.138819 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 115034us
I20260709 12:19:40.136135 32040 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:40.139753 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:40.136343 32112 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.137353 31969 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:19:40.113276  5429 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:40.145648 31964 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:40.145718 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.149044 31964 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:40.149282 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:40.150743 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45315us
I20260709 12:19:40.158064 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.152063 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.104194  5479 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.154192  5527 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.157143  5663 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:40.172199 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.158584  5532 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.173946  5651 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:40.175171 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.165098  5522 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:19:40.182961 32658 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:40.183758  5479 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:40.168325  5582 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:19:40.186148 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:40.172951  5648 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.187886  5527 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:40.188495 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48813.6us
I20260709 12:19:40.195853 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.197520 31972 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.198576 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.206960 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.194587  5651 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.200383  5359 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:40.213685  5651 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.214263  5254 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.214936  5582 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:40.215334  5582 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:40.219086 31964 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:40.219738 31964 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:40.212972 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.220180 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.220685 31964 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.212258  5522 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:40.217034  5359 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:40.240075  5359 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.222690  5520 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:40.237751 32118 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:40.238068 31964 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.238137 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.242515  5648 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:40.205737  5084 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.217304  5436 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:40.255553 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.259125 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.259865  5310 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:40.260368 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.261160 32112 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:40.247402  5691 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.252s	user 0.004s	sys 0.000s
I20260709 12:19:40.211196  5527 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.246417  5569 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:40.296957 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.298286 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.299095 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.272297 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.272792  5165 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:40.277519 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.278226 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.315059 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.316286 32047 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:40.317317 32047 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:40.279045  5520 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:40.319731 31964 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:40.279430  5524 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.280862  5617 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:40.281682  5697 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:19:40.281853 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.359417 32040 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:40.319109  5528 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.257885  5436 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.360275  5436 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:40.255642  5445 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.282550 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.361917 32043 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.284597  5527 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.286330 31965 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:40.286679 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.363775 32112 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
W20260709 12:19:40.294611 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 96081.1us
I20260709 12:19:40.302546 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.313040 31972 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:40.265588  5310 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:40.367857  5310 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:40.325237  5698 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:19:40.319036  5165 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.318666  5429 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:40.318591  5648 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:40.354717  5582 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.356712  5520 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.318959  5681 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:40.382676  5165 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:40.378338 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75834.9us
I20260709 12:19:40.381783 32659 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:40.366938 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:40.388505 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.392343 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.394387 32047 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:40.396883 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:40.398032 32047 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:19:40.399257 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.399552 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.399029 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.400211 32040 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.387780  5648 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:40.387609  5681 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.392277  5528 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.386233  5429 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:40.403384 32084 leader_election.cc:400] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:40.412698 32084 leader_election.cc:403] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:40.386843  5524 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:40.412988 31972 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:40.397271  5569 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:40.396812  5527 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:40.403337  5582 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.405793  5681 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.402530  5651 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.423475  5335 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:40.430058  5480 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:40.430509  5480 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:40.431318  5480 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 17.17, Last appended by leader: 17, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:40.432297 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.432499  5480 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.434938 32040 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:40.436232 32115 raft_consensus.cc:1217] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Deduplicated request from leader. Original: 14.8->[16.9-18.10]   Dedup: 16.9->[18.10-18.10]
I20260709 12:19:40.436736 32115 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 12:19:40.440388 32112 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:40.427280  5648 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:40.443842  5690 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.442s	user 0.000s	sys 0.003s
I20260709 12:19:40.424386  5527 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.449035  5527 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:40.449930  5576 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.450515  5576 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:40.452023 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.452761  5576 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.455143 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:40.456043 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.461791 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.465754 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.467093 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.467939 32658 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:40.468827 32040 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:40.471192 32657 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:40.423458  5429 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.473536 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:40.475039  5655 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.475375 32656 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 13. (term mismatch)
I20260709 12:19:40.483618 32656 pending_rounds.cc:85] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 12
I20260709 12:19:40.476488 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:40.476724 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73471.6us
I20260709 12:19:40.458350  5335 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.484671 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:40.477874  5527 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:40.479779 31972 raft_consensus.cc:606] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:40.480029  5417 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:40.483922 32656 pending_rounds.cc:107] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 12.13
I20260709 12:19:40.472232  5703 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:19:40.487402 32656 raft_consensus.cc:2889] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:40.487187  5527 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:40.498695 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.474257  5359 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:19:40.504417 31966 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:19:40.506675 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.508155 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.493844  5417 consensus_queue.cc:1243] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:19:40.508790 32115 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 11.
I20260709 12:19:40.495878  5252 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:40.488407  5335 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.502018  5527 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.517833 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.511283  5655 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.518680  5518 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:40.526793 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:40.508392  5359 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:40.527272 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.528049 32119 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:40.525915  5518 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:40.528710  5518 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:40.529117 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.529870 32112 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:40.531026  5655 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.533874  5655 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:40.539300 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:40.541353 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.519979  5617 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.544018 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:40.545748 32115 raft_consensus.cc:1240] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:40.546756 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:40.548830  5368 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:40.549217  5368 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:40.550034  5368 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.551127  5368 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:40.524075  5520 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:40.555291 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.557044 32121 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.560345  5416 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:40.563746 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.566176 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.567204 31963 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:40.567416  5684 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:40.568118  5684 consensus_queue.cc:1243] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:19:40.568261 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:40.568527 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:40.569098 31964 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:40.569495 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:40.569876 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:40.570493  5502 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:40.570643 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:40.571045  5502 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:40.571636 31969 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:40.572029  5502 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.572757 31969 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.573887 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:40.574542 31966 raft_consensus.cc:2376] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18. Re-sending same reply.
I20260709 12:19:40.542660  5425 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.575345 31964 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.577486 32118 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:40.577693 32115 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:40.576740  5651 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:40.584201 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57472.3us
I20260709 12:19:40.556576  5587 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:40.558717  5252 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.589814 32121 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:40.586555 31964 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:40.575445  5520 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.597236  5436 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.605099 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.605976 32042 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:40.606639  5673 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.607146  5673 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.605032  5527 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.605301  5436 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.609853 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:40.612654 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.614513 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.616984  5527 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.619774 32121 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:40.589069  5587 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.583212  5479 consensus_queue.cc:1059] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:19:40.585772  5425 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.626768 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.629820 32115 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.630824 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.635663 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.625180  5569 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:40.624293  5587 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:40.629367  5520 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:40.642742  5702 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:40.643855  5654 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:40.665354  5654 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.665956 32042 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:40.666713  5654 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:40.649137 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 83960.4us
I20260709 12:19:40.636040  5576 raft_consensus.cc:3055] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:40.653110 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.659741  5425 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.660473 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.671378 32045 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
W20260709 12:19:40.671643 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 104632us
I20260709 12:19:40.660885  5335 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:40.661661 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.673175 32121 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:40.664029 31963 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:40.663832  5165 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.677378 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.623886  5681 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.686664 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.620857  5518 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.670130  5663 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:40.641794  5252 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:40.644711  5569 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.703466  5569 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.669715  5576 raft_consensus.cc:740] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:40.677364 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.680384 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.714843 32121 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:40.715076  5436 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.716444  5648 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:40.716854  5648 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:40.681354 31964 pending_rounds.cc:85] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 14
I20260709 12:19:40.717696  5648 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.718367 31964 pending_rounds.cc:107] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:19:40.718696 31964 raft_consensus.cc:2889] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:40.718943  5648 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.681970 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:40.665730  5702 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:40.729638  5702 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:40.683949 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.647538 32115 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:40.694038  5527 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:40.732331  5527 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.732792  5702 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.733999  5702 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.678222  5001 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:19:40.692754 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.694546 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.738184 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.738804 32119 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.668151  5520 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.751286  5254 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:40.751734  5254 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:40.752882  5520 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:40.752723  5254 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.707520  5368 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:40.754607  5575 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:19:40.755764 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.756511  5673 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:40.756839  5436 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:40.757591  5527 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.755780  5252 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:19:40.683095  5425 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.709724 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:40.713536 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:40.676339 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.723174 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:40.731745  5681 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:40.759507 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 88255.7us
I20260709 12:19:40.761212 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.754042  5254 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.698405  5165 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.762984 32112 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.763411 32121 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:19:40.770061 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56642.6us
I20260709 12:19:40.767673 32119 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:40.771378 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.772786 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.799288 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.800449 32659 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:40.800385 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:40.802661 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:40.813226 32659 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:19:40.815265 32651 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:40.813651 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.816773 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:40.817238 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.821524 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:40.783562  5527 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.822644 32657 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.783903  5520 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:40.803573  5437 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:40.820992 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:40.782604  5436 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:40.825613 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.827941 32653 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:40.850114 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.804242  5359 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.814968  5445 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.807404  5084 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:40.811951  5480 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:40.824795  5310 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:40.822266  5659 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.782749  5425 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.814905  5663 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:40.821188  5681 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:40.852278  5684 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.818608  5417 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:19:40.856113  5528 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:40.827162  5520 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.814870  5673 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:19:40.852842  5437 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.879235  5445 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:40.857076  5084 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.851995  5441 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:19:40.893929 32120 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:40.851402  5527 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.881579  5684 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.861621  5310 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:40.867617 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.888629  5528 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:40.912995 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 96294.6us
I20260709 12:19:40.893720  5520 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:40.914582 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.897699  5437 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.879396  5651 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:40.913959  5084 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:40.932493 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:40.880676  5681 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:40.863684  5252 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:19:40.877799  5480 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.921003  5524 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.911860  5445 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:40.852619  5576 consensus_queue.cc:260] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.934603 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:40.923379  5528 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.827754  5654 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.964282 32120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:19:40.964877 32120 raft_consensus.cc:606] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:40.965231 32120 raft_consensus.cc:614] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.917608 32121 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:40.938052 32046 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:40.856482  5359 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.935537  5681 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.948127  5524 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.913980  5659 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:40.953161  5480 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:40.954057  5310 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.934079  5651 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:40.955806 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:40.942410  5663 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:40.960079  5704 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.470s	user 0.005s	sys 0.003s
I20260709 12:19:40.942873  5445 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.932804  5436 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.970662 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:40.986876 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.995854 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:40.987903 32112 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:40.992391 32118 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:40.972570  5576 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.997583 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:41.003677  5659 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.004582  5368 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:41.006047  5524 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:40.992810  5479 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:19:40.984248  5416 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:19:40.979362  5441 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:19:41.033144 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:41.033885  5654 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:41.006419  5436 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.007335  5417 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:19:41.049778 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.041543 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:41.007584  5429 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:41.006057  5368 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:41.059237 32657 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:41.060586 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:41.062873 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 154351us
I20260709 12:19:41.063683  5436 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.065284  5436 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:41.065873  5436 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:41.066938  5436 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:41.073717 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 163419us
W20260709 12:19:41.075388 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.076879 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 260756us
I20260709 12:19:41.049511  5648 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:19:41.081445  5520 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.085238 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:41.062587  5524 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:41.038141  5528 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.097944  5731 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:19:41.105381 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 174815us
I20260709 12:19:41.120963 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:41.126160 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:41.126389 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.126816 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.075678  5429 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:41.135459 32121 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:41.141522 32042 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:41.135905 32112 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:41.136174 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:41.137856 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:41.144104 32045 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:41.149128 32043 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 12:19:41.149513 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:41.150496 32042 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:41.152431 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:41.107205  5518 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.158156 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:41.162305 32043 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:41.158365 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:41.102171  5582 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 12:19:41.155019 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0475334410002688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 12:19:41.127657  5368 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:41.122717  5659 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:41.202777 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.200223 32040 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:41.134447  5743 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:41.205189 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 163701us
I20260709 12:19:41.118815  5084 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.154150  5335 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:41.216162 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.162909  5684 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.211719 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:41.217401 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.223229 32121 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:41.221956 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:41.201910  5518 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:41.226207 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:41.230796 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:41.219285  5736 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:19:41.170240  5569 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.229641  5518 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.210525  5368 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.229503  5743 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:41.243498  5651 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.252321  5359 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.220432  5659 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.226960  5575 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
W20260709 12:19:41.328912 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 93467.6us
I20260709 12:19:41.329679  5741 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:41.330436 32657 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:41.315193  5518 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:41.347457  5528 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.352128  5527 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.251607  5429 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.234280  5335 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.356611 32118 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:19:41.372303  5730 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:19:41.319090 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.390457  5743 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.537137 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:41.514443  5733 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.269s	user 0.000s	sys 0.004s
I20260709 12:19:41.479518 32038 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:41.489501 31968 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:41.488677 31964 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 12:19:41.526407  5715 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.490140 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:41.479326  5734 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.493453 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:41.494129  5310 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:41.495584 31965 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:41.496552 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:41.497985 31966 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:19:41.460809  5441 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:41.487329  5741 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.499011 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:41.559118 31966 pending_rounds.cc:85] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 13
I20260709 12:19:41.559407 31966 pending_rounds.cc:107] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:19:41.559523 31967 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:41.500471 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:41.341995  5569 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.533306 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:41.561086 32659 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.557138 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:41.563563 31968 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:41.564760 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64416.2us
I20260709 12:19:41.565778 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:41.567271 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:41.522439 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.347535  5520 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.541276 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:41.513947  5732 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.186s	user 0.000s	sys 0.004s
I20260709 12:19:41.559686 31966 raft_consensus.cc:2889] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:41.482450  5742 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:19:41.574481 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 422125us
I20260709 12:19:41.525931  5710 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:41.582597 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:41.478158 32045 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:19:41.613683 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.615185 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:41.626613 32120 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.624531  5359 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:41.635182  5359 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.625131  5710 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.642956  5359 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:41.615384  5681 consensus_meta.cc:287] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.001s
I20260709 12:19:41.562448  5741 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.535851  5527 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:19:41.648950 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:41.650763  5359 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:41.654508  5436 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:41.562732  5715 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.682530 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:41.670444 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:41.614176 32659 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:41.678655  5436 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.670078  5084 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.585656  5310 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.575855  5734 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.589254  5651 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:41.583758  5742 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:41.586123  5480 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:41.608120  5746 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.581772  5655 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:41.691751 32658 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:41.585289  5441 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.681097  5335 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.646031  5681 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.742394 31971 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:41.710404  5417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.042s	sys 0.082s
I20260709 12:19:41.690196  5436 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:41.713874 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 43512.8us
I20260709 12:19:41.715893  5359 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:41.738415  5722 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:19:41.744254  5417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.042s	sys 0.083s
W20260709 12:19:41.766743  5576 consensus_meta.cc:287] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.765s	user 0.012s	sys 0.002s
W20260709 12:19:41.768724  5587 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.098s	user 0.010s	sys 0.004s
W20260709 12:19:41.778718 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.778612 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:41.773062  5254 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.970s	user 0.006s	sys 0.011s
W20260709 12:19:41.773418  5445 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.761s	user 0.016s	sys 0.000s
I20260709 12:19:41.794116 32046 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:41.794214  5445 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.795760 32041 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:41.719990  5746 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:41.782150 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.050s	sys 0.074s
W20260709 12:19:41.800155 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.051s	sys 0.074s
W20260709 12:19:41.770622  5437 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.002s
I20260709 12:19:41.803799  5441 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.652092 32120 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:41.805481  5310 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.767721  5417 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:41.814392 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.822423 32119 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:41.816689 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:41.823490 32118 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.818823 31971 raft_consensus.cc:2410] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 12:19:41.782616  5582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.004s	sys 0.018s
I20260709 12:19:41.772626  5659 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.830663 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:41.819438 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:41.833698 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.798123  5587 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:41.819866 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 597924us
I20260709 12:19:41.829131 32112 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:41.898710 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.957947 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:41.838344 32115 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:41.860175 32117 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:41.959857 32117 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:41.834916 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.961704 32113 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:41.940357 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:41.949559 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 367054us
I20260709 12:19:41.965312 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:41.841574 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:41.967473 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.968115 32119 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:41.969931 32118 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:41.917279  5310 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.777616  5742 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.988047 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.973713 32114 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:41.975406 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:41.989881 32038 raft_consensus.cc:2410] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:41.975790 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:41.991168 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:41.977367 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:41.992202 32040 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:41.972997 32653 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 14: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:41.995482 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:41.997339 32660 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:41.855755  5750 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:19:41.836599  5582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.005s	sys 0.019s
I20260709 12:19:42.000545 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.013953 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:41.870079  5527 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:42.001444 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.030933  5742 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:42.031876 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.032455 32040 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.994684  5417 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:41.820735  5437 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.036368 31969 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:42.037721  5437 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.994289  5310 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.039315  5746 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:42.040036  5733 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:19:42.040655  5310 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:41.902701  5480 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.853951  5441 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:42.011493 32115 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:41.905459  5734 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.000456  5359 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:42.010187 32042 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:19:42.000267  5569 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:42.016215 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.016649  5713 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:41.831393  5429 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.240s
I20260709 12:19:42.034737  5527 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:41.910640  5681 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.067080 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:41.999097 32117 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.014276 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.074592 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.012430 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:42.077941 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 112702us
I20260709 12:19:42.030601  5425 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:41.911352  5753 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.193s
I20260709 12:19:42.086918 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.088719 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:42.089485 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 157917us
I20260709 12:19:42.085165 32117 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:42.088382 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.091059 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.085494 32114 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:42.092756 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.097409  5359 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:42.100889  5681 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.101190  5527 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.101459  5681 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.102558  5587 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.092326 32042 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
W20260709 12:19:42.105733 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 91534.4us
W20260709 12:19:42.088433  5518 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.001s
I20260709 12:19:42.075133  5480 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.111124 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.084916 32040 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:19:42.117966 32623 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.020s	sys 0.034s
W20260709 12:19:42.118842 32623 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.020s	sys 0.034s
W20260709 12:19:42.119640 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 633312us
W20260709 12:19:42.137878 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:42.158434 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69805us
I20260709 12:19:42.159338 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.161345 32042 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:42.163985 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55674.5us
I20260709 12:19:42.164754 32038 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:42.089087  5750 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:42.076666  5441 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.166612  5254 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.172927 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:42.174947  5741 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 12:19:42.105126  5713 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.176035  5681 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.176970  5761 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:42.177249 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.178339 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:42.181865 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:42.098467  5359 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.331910 32120 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:42.219031 32047 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:42.219329 32117 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 12:19:42.334430 32047 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:42.228964 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:42.334619 32117 pending_rounds.cc:85] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 9
I20260709 12:19:42.238840  5760 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.220238  5569 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
W20260709 12:19:42.330658  5763 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 12:19:42.335120 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 106294us
I20260709 12:19:42.224749  5335 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:42.335111 32117 pending_rounds.cc:107] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 12:19:42.336697 32117 raft_consensus.cc:2889] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:42.348260  5527 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:42.348639  5441 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.349033  5527 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.333035  5741 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.352124 31963 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.352558  5753 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:42.353035  5741 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:42.355255 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 181011us
I20260709 12:19:42.356904 32010 leader_election.cc:400] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:42.357323 32010 leader_election.cc:403] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:42.382501 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:42.383606  5254 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:42.384099  5254 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:42.384505  5254 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:42.384790  5527 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.386477  5713 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:42.386698  5252 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.387084  5713 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.387220  5252 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.157663  5518 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.389210  5713 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:42.390440  5582 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16.
I20260709 12:19:42.390856  5518 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.393119 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.398810 32040 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:19:42.406280 32040 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:42.406862 32040 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:42.391302  5760 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.407931 32040 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.410207 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.412741 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.405054  5518 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:42.421947  5518 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:42.421535 32038 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.422538  5518 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:42.424206  5751 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:42.425930 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.406116  5742 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:42.410615 32040 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.426728  5751 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:42.426980  5781 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.427498  5781 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.429566  5781 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:42.426745  5742 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:42.353281  5753 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.430835  5742 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.432922  5753 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.411754  5582 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:19:42.443259  5759 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:42.443904  5751 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:42.445323  5742 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.446163 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.447520  5742 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.448189 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:42.448635  5760 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.389238  5252 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:42.222050  5750 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.443297 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:42.489629 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 330436us
W20260709 12:19:42.490994 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47789.1us
W20260709 12:19:42.491385 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 295278us
I20260709 12:19:42.491725 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:42.493062 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.499743 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.502130 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.504374 32117 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:42.490200  5441 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.501696  5417 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.503127  5429 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:42.507925 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:42.530661 31964 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.514792 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:42.534204 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.508236  5582 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:42.509079  5779 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:19:42.509917  5772 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:42.538822  5359 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:42.539000 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.538578  5518 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.539599 32115 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:42.539398  5359 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.540510 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.541544  5518 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.517385  5441 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.542459  5429 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.544363  5441 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.517539 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.545876  5429 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.518369 31967 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:42.547582 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.548190 31967 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:42.549105 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.549680 31967 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:42.518285 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.551424 31972 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:42.499617  5781 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:19:42.552289 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.553020 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:42.554175 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.527132 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.522876 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:42.541620 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:42.606889 32115 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.530551  5778 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.515981 32038 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:42.501885  5252 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:19:42.608831 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.599985 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:42.519623 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 171885us
I20260709 12:19:42.505888 32040 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:19:42.598102  5518 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.541596  5359 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:42.612905 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 59970.3us
I20260709 12:19:42.599138  5759 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:42.615011 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 92207.6us
I20260709 12:19:42.603904  5723 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:42.525311  5368 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.616657 31972 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.617009 32040 pending_rounds.cc:85] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 9
I20260709 12:19:42.603897 31964 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:42.523842  5569 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:19:42.600737  5429 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.537621  5772 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.521126  5741 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:19:42.627184 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.604254  5582 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:42.633410 32040 pending_rounds.cc:107] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:42.609841  5760 raft_consensus.cc:2764] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:19:42.634276 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.634831 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2481) took 1136 ms. Trace:
I20260709 12:19:42.627856  5784 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:42.637312  5779 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:42.613530  5751 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.710916 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:42.627751  5518 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.735879 31969 rpcz_store.cc:276] 0709 12:19:41.498569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.498686 (+   117us) service_pool.cc:224] Handling call
0709 12:19:41.498935 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.036753 (+537818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.634515 (+597762us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400313154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:42.634772 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235549952}
I20260709 12:19:42.529776  5417 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.739691 31972 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:42.767653 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.768781 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.769462 31972 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.770296 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:42.745499 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:42.770890 32115 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.771184 31964 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.771658 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.614001  5778 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.772751 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.773736  5778 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.770339 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:42.775528  5789 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:19:42.776489 31972 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:42.776835 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2737) took 1548 ms. Trace:
W20260709 12:19:42.778340  5775 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:19:42.777266 32121 rpcz_store.cc:276] 0709 12:19:41.228139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.228320 (+   181us) service_pool.cc:224] Handling call
0709 12:19:41.228713 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:41.229130 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:41.229159 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:42.473402 (+1244243us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:42.500626 (+ 27224us) log.cc:844] Serialized 26 byte log entry
0709 12:19:42.589331 (+ 88705us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:42.589393 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.589522 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.776541 (+187019us) raft_consensus.cc:1685] finished
0709 12:19:42.776563 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.776773 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13140,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7436,"spinlock_wait_cycles":2595584,"thread_start_us":74603,"threads_started":1,"wal-append.queue_time_us":102381}
I20260709 12:19:42.777729 31972 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:42.783561  5727 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.563s	user 0.004s	sys 0.011s
I20260709 12:19:42.784286 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2479) took 1288 ms. Trace:
I20260709 12:19:42.784813 31971 rpcz_store.cc:276] 0709 12:19:41.496176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.496295 (+   119us) service_pool.cc:224] Handling call
0709 12:19:42.784023 (+1287728us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540034e254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:42.784217 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421612416}
W20260709 12:19:42.683908  5776 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:19:42.732661 32040 raft_consensus.cc:2889] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:42.769109 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.787163 31969 raft_consensus.cc:2410] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:19:42.640661  5759 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.794036 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2665) took 1841 ms. Trace:
I20260709 12:19:42.791514 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.795075  5759 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.744307  5779 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:42.795789  5779 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:42.796619 32660 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 892) took 2119 ms (client timeout 1568 ms). Trace:
W20260709 12:19:42.796998 32660 rpcz_store.cc:269] 0709 12:19:40.676928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.677091 (+   163us) service_pool.cc:224] Handling call
0709 12:19:42.796415 (+2119324us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540063e554. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:42.796525 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2949169920}
I20260709 12:19:42.748776  5784 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.793017 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:42.802621  5787 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:19:42.803222 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.794117  5445 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.794567 32039 rpcz_store.cc:276] 0709 12:19:40.952745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.953922 (+  1177us) service_pool.cc:224] Handling call
0709 12:19:40.954280 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.991359 (+ 37079us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:41.250016 (+258657us) log.cc:844] Serialized 14 byte log entry
0709 12:19:41.950164 (+700148us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:41.962293 (+ 12129us) log.cc:844] Serialized 26 byte log entry
0709 12:19:41.962498 (+   205us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:41.962543 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:41.962633 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.792009 (+829376us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b1800287440. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:42.793742 (+  1733us) raft_consensus.cc:1685] finished
0709 12:19:42.793842 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.793979 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65947,"prepare.queue_time_us":32667,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":35,"raft.queue_time_us":108857,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":34857,"spinlock_wait_cycles":4046592,"thread_start_us":812273,"threads_started":3,"wal-append.queue_time_us":405259}
I20260709 12:19:42.791761  5518 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.817504 31967 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.818015  5794 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:42.818679  5794 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.819414 31964 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:42.802052  5784 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.788926  5791 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:19:42.821623 32046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2502) took 1701 ms (client timeout 2177 ms). Trace:
I20260709 12:19:42.803984 32040 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.808166 32038 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:42.812311 32115 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:42.803984 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:42.831061 31963 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2482) took 1331 ms (client timeout 1504 ms). Trace:
W20260709 12:19:42.837951 31963 rpcz_store.cc:269] 0709 12:19:41.499311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.536875 (+ 37564us) service_pool.cc:224] Handling call
0709 12:19:42.830786 (+1293911us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b5400314a54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:42.830979 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872120576}
I20260709 12:19:42.835513  5445 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:42.839995 32044 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.847002  5417 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.847368  5772 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.848800  5715 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:42.849360  5715 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.735287  5429 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.851608  5429 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.843932 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:42.852960 31969 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.853762 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.853981 32044 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:42.854372 31969 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:42.855335 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.855998 31969 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:42.840265  5791 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.858366 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.859292 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.859408  5582 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.860783 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.851616 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:42.836656 32046 rpcz_store.cc:269] 0709 12:19:41.120458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.120622 (+   164us) service_pool.cc:224] Handling call
0709 12:19:42.821389 (+1700767us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54001225d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:42.821540 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1516010240}
I20260709 12:19:42.864048 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.864485 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.840161  5784 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.873548 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.874195 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:42.877857 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.878515 32039 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:42.879976 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.846078 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:42.881294 31967 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:42.862135 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.866225 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:42.891245 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 264127us
I20260709 12:19:42.859458 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.894460 32044 raft_consensus.cc:2393] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:42.895599 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.896391 32044 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:42.897500 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.892663 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:42.899950 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:42.902297 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:42.904258 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:42.905623 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.905730 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.907109 32120 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:42.908468 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.912647 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:42.910305 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.915284 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:42.877424  5722 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:42.875813  5703 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:42.919556 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.928309 32115 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:42.920346 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.914199 32044 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:42.899261  5718 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.837s	user 0.006s	sys 0.028s
I20260709 12:19:42.923466 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.925009 32040 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:42.927170 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:42.934684 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.929497 32653 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:42.932214 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:42.936358 32039 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:42.937136 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:42.932960 32656 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:42.941669 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:42.942564 32120 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:42.944336 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:42.845683  5445 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.948489  5445 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.858469  5254 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.950907  5254 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:42.898897  5715 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:42.954722  5715 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.957494  5715 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:42.926486  5359 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.959998 32046 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:42.962455  5335 raft_consensus.cc:2764] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:19:42.962848  5713 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:42.931725 32047 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:42.918067  5759 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:42.967521 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2750) took 2284 ms. Trace:
I20260709 12:19:42.968966 32116 rpcz_store.cc:276] 0709 12:19:40.679414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.691581 (+ 12167us) service_pool.cc:224] Handling call
0709 12:19:40.691881 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.692194 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.692221 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:40.708284 (+ 16063us) log.cc:844] Serialized 26 byte log entry
0709 12:19:40.879239 (+170955us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:40.879285 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.879350 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.640975 (+1761625us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:42.963098 (+322123us) raft_consensus.cc:1685] finished
0709 12:19:42.963121 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.963350 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10737,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":6479744,"thread_start_us":123590,"threads_started":1,"wal-append.queue_time_us":216650}
I20260709 12:19:42.929913  5751 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:42.968163 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:42.882647  5784 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:42.930991  5796 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:42.968451  5576 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:42.963308  5713 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:42.978538 32039 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:42.983728 32039 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:42.971228  5791 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:42.976590  5778 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:42.966818  5720 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:19:42.992355 32659 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:42.987986  5797 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:43.000715 32044 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:42.990602  5576 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:42.993913  5778 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.013408 32044 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:42.993763  5252 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:42.999547  5720 consensus_queue.cc:1243] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:43.000432  5777 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:43.012563  5778 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.042817 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2476) took 1554 ms. Trace:
I20260709 12:19:43.043921 31970 rpcz_store.cc:276] 0709 12:19:41.488248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.493214 (+  4966us) service_pool.cc:224] Handling call
0709 12:19:43.042569 (+1549355us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540034d854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:43.042727 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145058560}
I20260709 12:19:43.016744  5797 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:43.044293 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 125631us
W20260709 12:19:43.044363 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 127575us
W20260709 12:19:43.051232 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 138658us
W20260709 12:19:43.055532 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 120820us
I20260709 12:19:43.056865 31967 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:43.061322 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.063303 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:43.092213 31967 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.9-21.9]
I20260709 12:19:43.092778 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.051270  5252 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:43.051103  5777 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.110795  5797 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.120878  5252 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.121937  5777 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.094096  5703 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:43.060699  5794 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:43.060544  5576 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.127954  5702 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:43.128688  5702 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:43.114990  5784 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.018190  5713 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.118515  5779 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.132519 32042 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:43.135303 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.136168 32042 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:43.137372 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.138747  5779 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.145668  5663 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:43.146235  5663 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:43.147184  5663 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.149837  5663 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:43.150449 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2681) took 1168 ms. Trace:
I20260709 12:19:43.150868 32045 rpcz_store.cc:276] 0709 12:19:41.981820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.014761 (+ 32941us) service_pool.cc:224] Handling call
0709 12:19:43.150261 (+1135500us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400216e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:43.150397 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982214016}
I20260709 12:19:43.152020 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.154985 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.129724  5702 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.095808 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:43.158078 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62546.2us
I20260709 12:19:43.158269 32042 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:43.158860 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:43.158970 32113 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:43.159467 32045 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:19:43.169132 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:43.164295 32045 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:43.172122 32042 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:43.183049 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.164238  5702 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:43.194657 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 133058us
I20260709 12:19:43.202982  5781 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:43.163933  5778 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.204515 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2509) took 2058 ms. Trace:
I20260709 12:19:43.179829  5794 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:19:43.161526  5806 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:43.211925 32041 rpcz_store.cc:276] 0709 12:19:41.146350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.146615 (+   265us) service_pool.cc:224] Handling call
0709 12:19:41.147003 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:41.796141 (+649138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.204251 (+1408110us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54001937d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:43.204452 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491505536}
I20260709 12:19:43.210618  5781 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:43.210842 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.180351  5761 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.190764  5777 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:43.225911 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.216996  5751 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:43.220170  5416 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:43.190202  5742 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:43.221299 32047 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.227383 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.228312 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.211323  5703 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.234175  5703 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.236480  5703 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:43.236814 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 710455us
I20260709 12:19:43.237761 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:43.237957 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.238579 32041 raft_consensus.cc:2376] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20. Re-sending same reply.
I20260709 12:19:43.238478  5368 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:43.239984  5710 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.240196  5368 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.240895  5723 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:43.240758  5368 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.241526  5723 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.244237  5528 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.244745  5417 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.244952  5772 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:43.245437  5772 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:43.245539  5793 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:43.246153  5793 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:43.246552  5772 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.246973  5793 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.248193  5793 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.248799 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2750) took 1314 ms. Trace:
I20260709 12:19:43.249239 32114 rpcz_store.cc:276] 0709 12:19:41.934520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.066718 (+132198us) service_pool.cc:224] Handling call
0709 12:19:43.248637 (+1181919us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b540012a7d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:43.248743 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11097472}
I20260709 12:19:43.244782  5528 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.253798  5681 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.254297  5681 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.255407  5528 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.256265  5681 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.245373  5417 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.259732 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.228228 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.260866 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:43.266769 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.114190 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:43.270829 32113 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.271628 32047 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:43.272401 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2676) took 1293 ms. Trace:
I20260709 12:19:43.272733 32047 rpcz_store.cc:276] 0709 12:19:41.979443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.172126 (+192683us) service_pool.cc:224] Handling call
0709 12:19:42.218904 (+ 46778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.931283 (+712379us) spinlock_profiling.cc:243] Waited 8.34 ms on lock 0x7b4c003c2d40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228a49ba20 00007f228a4aca29 00007f228a472644 00007f228a499f2d 00007f228a499b93 00007f228c1ca18b 00007f228c1cacb5 00007f228c1c5f30 00007f228c1c77de 00007f228c1d1b49 00007f228c1d2416
0709 12:19:43.220782 (+289499us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b580049fdd0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c12b0aa 00007f228c1ad447 00007f228c1c5fe7 00007f228c1c77de 00007f228c1d1b49 00007f228c1d2416 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419 00007f228a70abd4
0709 12:19:43.272171 (+ 51389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.272351 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203914368}
I20260709 12:19:43.249879  5741 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:43.243855  5723 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:43.275535 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.279301 32011 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.284065 32113 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:43.276543 32045 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:43.275497 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:43.277895 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48781.1us
I20260709 12:19:43.233464  5798 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:19:43.290735  5480 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.291255 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.291639  5480 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:43.292343  5480 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.295315  5368 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.304656 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.305372 32121 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:43.274910  5715 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.278650  5720 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:43.282181  5729 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:19:43.232158  5575 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:19:43.312940 32121 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:19:43.233057  5742 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.282497  5779 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:43.342334 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.336428 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:43.299814  5797 raft_consensus.cc:2764] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:19:43.345340 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47218.1us
W20260709 12:19:43.345821 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.338565  5784 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:19:43.348136  5784 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:43.348497  5784 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:43.349570 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.349426  5784 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.351022 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.351970 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.352777 32653 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:43.310300  5715 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.358613 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:43.336503  5720 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.373908 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:43.376209 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64185.4us
I20260709 12:19:43.341511  5779 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:43.345234  5748 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:43.389819 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.391930 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.392645 32656 raft_consensus.cc:2368] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:43.339604  5576 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:19:43.359766  5587 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.361888  5715 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.399977 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.400626 32045 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:43.398166 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.361732  5714 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:43.372342  5759 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.378417  5720 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.378670  5784 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.386284  5252 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.417361 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.306627  5781 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.421169 31968 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.421602 31936 leader_election.cc:400] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:43.421973 31936 leader_election.cc:403] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:43.409607 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.380563  5582 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:19:43.485061 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:43.485677 32114 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.403497 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:43.411209 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.387651  5518 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:43.394219 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.413388 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.487655 32044 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:19:43.487890 32046 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:43.418735 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:43.488744 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 81167.2us
I20260709 12:19:43.373413  5791 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:19:43.395315  5809 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.417632 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:43.422772 32045 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:43.391515  5779 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:43.420315 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 160174us
I20260709 12:19:43.495822 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:43.497756 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 94380.2us
I20260709 12:19:43.497849 32045 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:43.423226  5252 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.418890  5441 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.419759  5368 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.466974  5417 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.473659  5713 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:43.385102  5798 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:19:43.424310  5648 raft_consensus.cc:3060] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.464027  5800 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:19:43.386792  5808 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:19:43.477536 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.479650 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.476210  5703 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.378328  5743 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:19:43.498667 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.480297 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.500744 31972 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:19:43.501050  5723 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.517485 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.499963 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:43.476933  5818 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:19:43.504908 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:43.504880 31968 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:43.505241 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:43.506047 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.506835 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.531287 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2488) took 1014 ms. Trace:
I20260709 12:19:43.512004 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.532068 31964 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:43.474900  5778 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.516397 31967 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:43.533771 32114 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:43.501617  5760 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:43.504499 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.512935  5441 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.410264  5714 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:43.526988 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46798.1us
I20260709 12:19:43.497977  5761 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.556277 32040 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:43.527545  5723 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:43.560528 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55362.4us
I20260709 12:19:43.534865  5750 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:43.406428  5587 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.531670 31965 rpcz_store.cc:276] 0709 12:19:42.517303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.518375 (+  1072us) service_pool.cc:224] Handling call
0709 12:19:42.529060 (+ 10685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.551581 (+ 22521us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:43.156263 (+604682us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b1800478ba0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:19:43.235486 (+ 79223us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b1800478ba0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22900a358f 00007f22890e2571 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd
0709 12:19:43.389402 (+153916us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.530938 (+141536us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:43.531002 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:43.531046 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.531118 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.531230 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26896,"prepare.queue_time_us":56161,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":37905,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":43747,"spinlock_wait_cycles":10102400,"thread_start_us":349349,"threads_started":2}
I20260709 12:19:43.533507 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.490736  5518 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.532238 32046 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:43.534899 32626 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:43.522394  5742 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.540968 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.543231 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.549464  5792 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:43.554459  5815 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:19:43.555343  5831 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:19:43.528774 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.533715  5480 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:43.580506 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:43.542872  5648 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
W20260709 12:19:43.596423 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 55516.4us
I20260709 12:19:43.598877 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2512) took 2450 ms. Trace:
W20260709 12:19:43.599287 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56189.7us
I20260709 12:19:43.612450 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:43.611541 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 92260.1us
W20260709 12:19:43.605116 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 70281.2us
I20260709 12:19:43.530923  5368 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.534127  5703 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.619750 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:43.622918 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.602528  5417 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.588922  5663 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.623230 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.612046 32043 rpcz_store.cc:276] 0709 12:19:41.148534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.167298 (+ 18764us) service_pool.cc:224] Handling call
0709 12:19:41.550678 (+383380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:41.555227 (+  4549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:41.555257 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:41.555702 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:19:41.949899 (+394197us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:41.949941 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:41.950008 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.060618 (+110610us) raft_consensus.cc:1685] finished
0709 12:19:42.060648 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.578906 (+1518258us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54000348d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:43.598805 (+ 19899us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26413,"raft.queue_time_us":13245,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4370,"spinlock_wait_cycles":868684032,"thread_start_us":393806,"threads_started":1,"wal-append.queue_time_us":404899,"wal-append.run_cpu_time_us":1686,"wal-append.run_wall_time_us":1065035}
I20260709 12:19:43.624189 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.637840  5828 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:43.638749  5828 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:43.639142  5828 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.639303  5827 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:43.639585  5828 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.641490  5748 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:43.593560  5518 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.589293  5750 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:43.503891  5809 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.599701  5792 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:43.620877  5796 raft_consensus.cc:2804] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:43.644896  5800 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:19:43.660043  5803 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:43.690184  5803 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:43.690608  5803 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.621073  5823 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:43.619071  5480 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.697332  5823 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:43.697458  5480 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:43.697846  5823 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:43.671470  5368 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:43.672178  5834 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:19:43.672616 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49443.6us
W20260709 12:19:43.698926  5822 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:19:43.699863  5575 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:19:43.700348 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.672734  5429 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:43.702497  5681 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:43.702626  5429 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:43.702957  5681 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.703424  5681 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.703482 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.626109  5715 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.673245  5703 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.706050 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:43.679184 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55073.2us
I20260709 12:19:43.703686  5429 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.643685  5663 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.679834  5772 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:43.681857 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.682298  5837 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:43.682638 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.684063  5809 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.657532  5750 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.659535  5582 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.711540 32116 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:43.711700 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.767100 32115 raft_consensus.cc:2368] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:43.717305  5663 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.673928  5792 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.713191 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:43.679314  5441 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.687057  5252 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.722172  5527 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:43.705523 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:43.775452 32114 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:43.665347  5797 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:43.772193 31938 connection.cc:256] RPC call timeout handler was delayed by -1.1702638330002628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:19:43.725051  5796 raft_consensus.cc:697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:43.780313 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 110295us
W20260709 12:19:43.781950 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 68856.3us
I20260709 12:19:43.782938 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.786171 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:43.788789 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.788477 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.793589 31970 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
W20260709 12:19:43.779304  5793 consensus_meta.cc:287] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.001s
I20260709 12:19:43.798614 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.799851 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.800870 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:43.728927  5837 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.832839 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:43.795820  5528 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.789784  5772 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:43.765718  5582 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.833448 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.785527  5797 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.786607  5417 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.849789 32657 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:43.851814 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.796909  5792 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:43.854858 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:43.789110  5779 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.860203 31966 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:43.835779  5441 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:43.858532 31965 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:43.874761  5582 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.877996  5720 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.877920 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:43.880973  5784 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:19:43.881127  5836 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:19:43.890070  5797 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:43.921816  5518 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.859809  5417 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.965891  5750 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.966384  5417 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:43.898087  5824 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 12:19:43.898902  5838 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:19:43.898620  5803 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.968602 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2702) took 1154 ms. Trace:
I20260709 12:19:43.969612  5803 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.900576 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:43.854372  5714 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.902292  5587 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.863550  5753 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.903191  5846 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.903430  5441 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:43.848014  5528 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.912396  5837 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.912213 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:43.946012 32121 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:43.948661  5720 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:43.949975  5794 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:43.870440  5842 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.950649  5828 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.951701  5779 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:43.860653  5796 consensus_queue.cc:237] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:43.950085 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:43.946396 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.051733 32114 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:43.892918  5663 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.961475  5823 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:43.884125  5723 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.060930 32116 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:43.899387 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.062219 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.063298 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.063652  5528 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.064277 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.065258 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.065754 31969 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:44.031435  5417 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:44.066584  5417 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.068598  5417 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.971907 31965 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:43.974609 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2703) took 1134 ms. Trace:
I20260709 12:19:44.069193 31963 rpcz_store.cc:276] 0709 12:19:42.840319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.846165 (+  5846us) service_pool.cc:224] Handling call
0709 12:19:42.846480 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.478393 (+631913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.478436 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.494791 (+ 16355us) log.cc:844] Serialized 26 byte log entry
0709 12:19:43.639002 (+144211us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b480036c0d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:43.719313 (+ 80311us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:43.719358 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.719550 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:43.970000 (+250450us) raft_consensus.cc:1685] finished
0709 12:19:43.970018 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.974549 (+  4531us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":395741,"raft.queue_time_us":403897,"raft.run_cpu_time_us":2715,"raft.run_wall_time_us":387842,"spinlock_wait_cycles":4373760,"thread_start_us":410296,"threads_started":2,"wal-append.queue_time_us":212280}
I20260709 12:19:44.070464  5587 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.968953 31971 rpcz_store.cc:276] 0709 12:19:42.814162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.814379 (+   217us) service_pool.cc:224] Handling call
0709 12:19:42.814759 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.243105 (+428346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.243141 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.243612 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 12:19:43.401372 (+157760us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:43.415186 (+ 13814us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.416238 (+  1052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.416310 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:43.968425 (+552115us) raft_consensus.cc:1685] finished
0709 12:19:43.968444 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.968551 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447445,"prepare.queue_time_us":19630,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":436528,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":10274,"thread_start_us":157354,"threads_started":1,"wal-append.queue_time_us":344061}
I20260709 12:19:44.037309  5441 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:43.974754 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.071429  5441 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.999536 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:44.014842 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.043439 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:44.073136 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.073282 32120 raft_consensus.cc:2410] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:19:44.074247  5723 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:44.075618 31969 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:44.076754 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:44.077905 31967 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:44.078877 32654 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:19:44.046108 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 245280us
I20260709 12:19:43.968065  5750 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.082947 31964 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:44.051879 32117 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:43.962162  5480 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.031096  5760 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:43.994036  5784 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:44.000093  5713 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:44.039232  5794 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:44.055503 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.144970 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.044251  5445 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.052857  5532 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.058511  5663 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.060220  5823 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:43.970472 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:44.079933 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:44.090464 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:44.105360 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:44.123047 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.123598 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.129539 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:44.084851 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.043681  5772 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:44.105898  5741 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.105477  5252 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.133997  5720 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.071060  5587 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:44.172518 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.203214  5832 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:44.202554 32654 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
W20260709 12:19:44.204211 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 233832us
I20260709 12:19:44.082587  5723 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:44.206005 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 121234us
I20260709 12:19:44.238472 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:44.145874  5784 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:44.236281 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 156454us
I20260709 12:19:44.241146  5575 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:19:44.236801 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 113281us
I20260709 12:19:44.245065 31938 leader_election.cc:400] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:44.243901 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:44.210603  5663 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:44.148324  5760 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.265086 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:44.151597  5445 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:44.208438  5681 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.004s
I20260709 12:19:44.274251 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.241559  5575 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.241955  5648 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:44.280680 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:44.211262  5741 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.232293  5252 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.151228  5713 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.292913  5847 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:44.271632  5797 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.233712  5720 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.241904  5854 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:44.296972 32045 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:19:44.154708  5532 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.278381  5368 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:44.263875  5793 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:19:44.270617  5823 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:44.280464  5575 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:44.308736 32114 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:44.269977  5846 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:19:44.271852  5848 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:19:44.301393  5847 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.282032  5681 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.299011  5518 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.313024 32118 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:44.304425  5528 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:44.271958  5794 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:44.323132  5849 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:19:44.351428 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70830.8us
I20260709 12:19:44.307113  5648 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:19:44.281932  5827 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:44.330782 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:44.352425 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:44.329538 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47995.8us
I20260709 12:19:44.380478 32653 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:44.380882 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.309795  5797 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.362063  5741 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:44.363797  5856 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:19:44.365635  5855 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:19:44.367710  5587 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.331075  5847 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.297010  5772 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.369443  5252 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.375572  5723 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.377050  5760 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.402187 32653 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:44.390445 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2514) took 1885 ms. Trace:
W20260709 12:19:44.395941 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:44.397979 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45647.3us
I20260709 12:19:44.421150 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.422401 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:44.420063 32040 rpcz_store.cc:276] 0709 12:19:42.505458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.963886 (+458428us) service_pool.cc:224] Handling call
0709 12:19:43.247808 (+283922us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800177c98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890aaa22 00007f2291fc38d9 00007f229111dee0 00007f22911e70ab 00007f22911e7131 00007f22911a8fc0 00007f2291193929 00007f22911933d2 00007f228a70abd4
0709 12:19:43.248004 (+   196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:43.556680 (+308676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.390191 (+833511us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54001946d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:44.390384 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710845184}
I20260709 12:19:44.390833 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 1020) took 1042 ms. Trace:
I20260709 12:19:44.333184  5480 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.342604  5828 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.432765 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.428007 32651 rpcz_store.cc:276] 0709 12:19:43.348167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.352049 (+  3882us) service_pool.cc:224] Handling call
0709 12:19:43.352330 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:43.362918 (+ 10588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.362957 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:43.363013 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:43.363063 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.390613 (+1027550us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b540061e754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:44.390663 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:44.390776 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23893376}
I20260709 12:19:44.445228  5582 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.446025  5582 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.449674  5582 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.451503  5703 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.452306  5703 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.452752  5582 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.453563  5582 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.455111  5703 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.456712  5582 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.451611  5518 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.458205 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.462056 32040 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
W20260709 12:19:44.367704  5852 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:19:44.397637  5741 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.398597  5715 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.398301  5847 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.395270  5809 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:44.488724 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.491541 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.494019 32047 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:44.386571  5445 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.383829  5797 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.346552  5417 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:19:44.494933 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.475299 32114 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:44.465764 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.369438  5681 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.497210 32041 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:44.550437 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.497749 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.492801 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:44.552035 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59312.4us
I20260709 12:19:44.466414 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.552986 32040 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.383993  5827 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:44.441260  5429 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:44.439401  5854 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:44.538671 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.537652 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:44.566917 31967 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:44.459196  5867 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.459286  5252 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:44.556453 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:44.458420  5823 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:19:44.578920  5837 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.579624  5837 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:44.580938  5860 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:19:44.581910 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:44.582628 32114 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:44.582919  5837 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.586455 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:44.587754 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.588829 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.558060 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.599905 32039 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:44.560321 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 127654us
I20260709 12:19:44.490860  5480 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:44.382301  5441 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.494756  5518 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.563162 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:44.603736 31965 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:44.589881 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.497838  5809 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:44.555847 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 296330us
I20260709 12:19:44.606861 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:44.616947 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.617718 32043 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:44.618098 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.619271 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.620922 32040 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:44.622872 31967 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:44.624320 32114 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:44.625597 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:44.627179 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.633461  5681 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:44.633983  5681 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.636025  5681 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.644559 31965 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:44.573226  5868 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:44.600582  5823 consensus_queue.cc:1243] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:44.649340  5868 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.686472  5868 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:44.683933  5532 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.704280  5713 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.706280  5720 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.719189 32038 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:19:44.575173  5252 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.650985 32047 raft_consensus.cc:2410] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:44.651556 31963 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:19:44.736181 32039 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:44.741060 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.618988  5480 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.747571 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:44.748515 31965 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:44.750633  5441 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.751480 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.637902  5854 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:44.756747 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.761452 31967 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:44.757339 32114 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:44.762712 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.763337  5518 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.764225 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:44.764503  5480 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.765202 32118 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:44.766185 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.743139  5809 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.770443  5797 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:19:44.759931 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.760501 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:44.648777  5827 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:44.638089  5648 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:44.647580  5800 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:44.778887  5827 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:44.778271 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2503) took 1263 ms. Trace:
W20260709 12:19:44.777583 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 203606us
I20260709 12:19:44.578325  5867 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.796068 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:44.786350 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:44.792089 31968 rpcz_store.cc:276] 0709 12:19:43.515088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.537611 (+ 22523us) service_pool.cc:224] Handling call
0709 12:19:43.538019 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.593427 (+ 55408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.593456 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.604315 (+ 10859us) log.cc:844] Serialized 26 byte log entry
0709 12:19:43.778363 (+174048us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:43.789172 (+ 10809us) log.cc:844] Serialized 14 byte log entry
0709 12:19:44.043480 (+254308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.753380 (+709900us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b5400315e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:44.753426 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:44.753470 (+    44us) raft_consensus.cc:1685] finished
0709 12:19:44.778026 (+ 24556us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:44.778205 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33890,"prepare.queue_time_us":10254,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":179100,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":21780,"spinlock_wait_cycles":119055616,"thread_start_us":383070,"threads_started":2,"wal-append.queue_time_us":231287}
I20260709 12:19:44.775287  5663 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.765064  5722 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:19:44.815655 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:44.817293 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:44.816589 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:44.783224  5648 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.786016  5800 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:44.797374  5827 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:44.787529  5480 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:44.825935  5648 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:44.824218  5518 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:44.819852  5877 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:44.831506  5877 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.833665  5877 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.835131  5722 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:44.835639  5722 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.836212 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:44.836316 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:44.836642 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:44.836865 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:19:44.836983 32657 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:44.837283 32653 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:44.837646 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.838773 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.839776 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.841094  5673 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:44.841594  5673 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:44.841787  5252 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.842473  5673 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.842893 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.843510  5781 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.843992 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.845036 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.845577 31972 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:44.838124  5720 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:44.846763  5847 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.824836  5480 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:44.848229  5480 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.855831  5648 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.860183 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.861137 31972 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:44.861042 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:44.861310  5778 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:44.862025  5870 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:44.861696  5845 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.862483  5870 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:44.862929  5870 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:44.862699  5845 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.820415  5663 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.864866  5845 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.865995  5782 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:44.866477  5782 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:44.867372  5782 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.868687  5782 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:44.869603 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 954) took 1469 ms. Trace:
I20260709 12:19:44.869982 32113 rpcz_store.cc:276] 0709 12:19:43.400569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.400726 (+   157us) service_pool.cc:224] Handling call
0709 12:19:43.400992 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:43.684565 (+283573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.684599 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:43.684645 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:43.684678 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.869408 (+1184730us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54001491d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:44.869453 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:44.869554 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651683200}
I20260709 12:19:44.871093  5648 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.871423 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.871992 32113 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:44.872917 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.875159 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 768) took 1018 ms. Trace:
I20260709 12:19:44.875630  5532 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:44.875531 31964 rpcz_store.cc:276] 0709 12:19:43.856990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.857103 (+   113us) service_pool.cc:224] Handling call
0709 12:19:43.857500 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.083352 (+225852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.874952 (+791600us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400002154. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:44.875106 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518039296}
I20260709 12:19:44.876128  5445 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:44.876752  5445 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.877187  5729 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:44.879905 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:44.880637  5729 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:44.880730  5748 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.881538  5445 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.881268  5748 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.882515  5335 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:44.883011 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.847101  5720 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.883664  5806 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:44.884480 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.885354  5720 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.885488 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:44.883037  5335 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.887322 31964 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
W20260709 12:19:44.888309  5575 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.548s	user 0.007s	sys 0.003s
W20260709 12:19:44.824065  5784 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.001s
I20260709 12:19:44.888803  5575 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.889010  5784 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:44.888571  5335 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.890285 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2792) took 1548 ms. Trace:
I20260709 12:19:44.890942  5575 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.890700 32121 rpcz_store.cc:276] 0709 12:19:43.342264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.342443 (+   179us) service_pool.cc:224] Handling call
0709 12:19:43.342748 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.038061 (+695313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.890069 (+852008us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400433fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:44.890225 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1349796736}
I20260709 12:19:44.892019 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.893476 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.894842 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.895390  5784 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.895449 32039 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 13.
I20260709 12:19:44.897152 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.897960 32040 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:19:44.876205  5532 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.900382  5532 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:44.901113 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.902627 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.903185 32044 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:44.903383  5809 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:44.886096 31971 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:44.904594  5722 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:44.905856  5663 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:44.906736  5663 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:44.907073 32045 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:44.907230  5663 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:44.909339  5663 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:44.914469 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.834262  5877 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:44.914079 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.915707 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.893482 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.919229 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:44.919902 31968 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:44.921218 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:44.922024 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.923277 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.874184 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:44.925683 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.973894 32044 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 23 because replica has last-logged OpId of term: 21 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 8.
I20260709 12:19:44.928891 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.943096 31965 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:44.975028 32121 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:44.977469 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.709s (SENT) [suppressed 12 similar messages]
I20260709 12:19:44.978796 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.978895 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.979442 32038 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.979981 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:44.980671 32041 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:44.980873 32046 pending_rounds.cc:85] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 8
I20260709 12:19:44.981149 32046 pending_rounds.cc:107] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:19:44.981422 32046 raft_consensus.cc:2889] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:44.982687 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63533.7us
I20260709 12:19:44.983654 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:44.985157 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:44.821023  5800 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.991386 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:44.992177 32044 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:44.996197 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:45.011819 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90665.5us
I20260709 12:19:45.012809 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:45.017436 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.019750 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.021123 32011 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:45.024636 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.025532 32115 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:45.023468 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:45.016185  5532 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:45.010866  5720 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:44.999295  5868 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.018718  5750 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:45.018443  5877 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:45.068536 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.112972 32115 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:45.022758  5808 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:44.975109  5848 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:19:45.104188 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.039589  5532 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.104486  5729 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:45.111294 32038 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:19:45.119640 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 303102us
I20260709 12:19:45.125304 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:45.128417 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:45.109616  5842 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:45.112479  5750 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.103615  5759 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:45.103345  5806 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.024041  5870 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.153447 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.131767  5441 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:45.131994  5794 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:45.139005 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.140925 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.142483 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.161793 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.162494 31971 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:45.036193  5751 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 12:19:45.148706 31972 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 20. Current term is 21. Ops: [18.15-20.16]
I20260709 12:19:45.130404  5877 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.154217 31964 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:45.143911  5532 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.103466  5254 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:19:45.155401 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.166886 31963 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:45.181030 32115 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:45.149675  5741 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.000s
I20260709 12:19:45.106590  5520 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:45.143580  5729 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.144915  5750 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:45.182672 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 205224us
I20260709 12:19:45.184907 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.191955 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.192840 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.199831 31971 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:19:45.195727 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.159981  5441 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:45.159188  5808 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:45.203681 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.205070 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.205909 31964 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:45.200928 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:45.209561 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:19:45.183753  5429 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.000s
I20260709 12:19:45.205029 31963 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:45.211575 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.213384 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.149120  5842 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.182107  5254 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.196183  5520 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.224283 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.225086 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.194814  5741 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.162186  5806 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.200702  5729 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:45.203274  5673 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:45.216532  5748 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:45.204195  5807 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.257931  5842 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:45.259904 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.023s	sys 0.028s
W20260709 12:19:45.260293 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.023s	sys 0.028s
I20260709 12:19:45.260341  5870 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.261857  5569 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:45.219329  5876 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:45.220351  5703 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.270332 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.270749  5703 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.271608 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.272449 32119 raft_consensus.cc:2410] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 23 because replica has last-logged OpId of term: 21 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 8.
I20260709 12:19:45.273016  5703 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.226214  5663 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:45.273702 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:45.273912  5663 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:45.274936  5663 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.224148  5254 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:45.275839 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:45.276696 32657 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:45.275856  5808 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 17.17, Last appended by leader: 17, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.230473  5520 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:45.280246  5867 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:45.214732  5832 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:45.231455 32112 raft_consensus.cc:2410] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 12:19:45.260592 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 1.01926e+06us
I20260709 12:19:45.289182 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:45.264978  5876 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:45.221129  5429 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:45.239734 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.298898 32121 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.241053  5837 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:45.282063 32041 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:45.236553  5835 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:19:45.235711 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:45.287473 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44681.5us
I20260709 12:19:45.290385 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.278648 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.214738  5883 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:45.291428 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 252870us
I20260709 12:19:45.305011 31969 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:45.211908  5827 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.274947 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:45.298277  5575 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:45.310906 32010 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.301529 31970 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
W20260709 12:19:45.303145 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67527.6us
I20260709 12:19:45.313068 32085 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.308645 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.317639 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:45.323068 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.324896 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.326236 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.329124 32038 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:45.332485  5879 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:45.334193  5827 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.334674  5702 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:45.334779  5798 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:45.335287 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.335883 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:45.295162  5832 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.336443 32041 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:45.337522  5703 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:45.337946  5703 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:45.338505  5703 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.298751  5441 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.342784 31967 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:45.343367  5441 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:45.343634  5417 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.344425  5417 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:45.344797  5417 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:45.345165  5417 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:45.347232  5781 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.334784  5575 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:45.323179  5518 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:19:45.348523  5575 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.304749  5835 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.335654  5793 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:19:45.351099  5870 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:45.351477  5870 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:45.352316  5870 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.353562  5870 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:45.353482  5520 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.284492  5673 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.356504 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:45.338006  5832 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.357546  5758 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:45.323678  5429 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.359535  5806 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.316781  5748 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:45.305234  5742 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:45.300213  5845 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.361507  5868 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:45.362346 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.363255 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:45.428579 32657 raft_consensus.cc:2410] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:19:45.364310  5741 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:45.364981 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:45.429816 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64921.4us
I20260709 12:19:45.430781 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.365571  5827 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:45.350157  5835 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:45.314895  5883 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.322989  5842 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
W20260709 12:19:45.376143 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68536us
I20260709 12:19:45.378067 32121 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:45.299674  5876 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.418865 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:45.419603 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 105226us
I20260709 12:19:45.449402 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:45.335700  5798 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:45.435608 32039 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:45.438769 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.458595 32121 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:45.443039  5417 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.445124  5703 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.421109  5720 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.417024  5648 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:45.426667  5809 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.004s
I20260709 12:19:45.425707 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.510521  5715 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:45.516739  5569 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.517017  5809 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.517539  5809 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.518586  5703 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.519587  5809 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:45.517053  5417 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.520332  5720 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.521986 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.522665 32047 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:45.451635 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:45.435279  5848 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.530411 32041 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:45.531895 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:45.506856 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49446.7us
I20260709 12:19:45.507989 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:45.534057 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.534488 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.473850 32044 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:45.465344 32117 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:19:45.432907  5429 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.477051 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.468389 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:45.424296  5758 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.450099 32653 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:19:45.523473 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71939.2us
I20260709 12:19:45.505857 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.347801  5781 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.446034  5750 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.419309  5792 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:19:45.522404  5791 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.538862  5335 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:45.536336 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.470700 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.575009 32040 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:45.551164 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.420644  5800 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.524905 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.551189 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.454947  5798 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:45.553341 32038 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:45.581718 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:45.433763  5845 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.557236 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1117) took 2198 ms. Trace:
I20260709 12:19:45.446660  5868 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:45.584656 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.596611 32117 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
W20260709 12:19:45.596654 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 165963us
I20260709 12:19:45.550766  5758 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:45.589159 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:45.597790 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.594133 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2779) took 1471 ms. Trace:
I20260709 12:19:45.556360  5742 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.598565 32112 rpcz_store.cc:276] 0709 12:19:44.122552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.122741 (+   189us) service_pool.cc:224] Handling call
0709 12:19:45.577541 (+1454800us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540013a954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:45.594037 (+ 16496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2366058752}
I20260709 12:19:45.573004  5720 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.603382 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.605600 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.607312 32118 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:45.591430 32660 rpcz_store.cc:276] 0709 12:19:43.358595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.358808 (+   213us) service_pool.cc:224] Handling call
0709 12:19:43.385881 (+ 27073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:43.590172 (+204291us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:43.713624 (+123452us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.802243 (+ 88619us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.802424 (+   181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:43.802466 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:43.802508 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.908879 (+1106371us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b180043e760. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1
0709 12:19:44.909435 (+   556us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b180043e760. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:45.556929 (+647494us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54003efb54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:45.556997 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.557166 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":167994,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":761,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5907,"spinlock_wait_cycles":565336576,"thread_start_us":180535,"threads_started":2,"wal-append.queue_time_us":172363,"wal-append.run_cpu_time_us":1657,"wal-append.run_wall_time_us":1001683}
I20260709 12:19:45.608327 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.536226  5648 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.605859 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:45.552825  5532 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.610546 32117 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:45.573554  5335 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:45.611940 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:45.615008 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:45.595891  5848 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:45.591374  5748 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.621224  5429 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.622462  5648 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.622781  5729 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:45.625891  5891 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:19:45.626271  5879 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.628607  5869 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:45.632334  5800 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:45.586216  5791 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.618619  5335 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:45.609536  5741 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.635056  5883 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:45.603195  5827 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:19:45.613866  5663 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:45.638751  5847 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:45.618876  5842 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:45.640163  5781 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.640532  5750 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.610121  5844 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:19:45.614250  5868 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:45.640993  5798 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.644375  5673 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.646126  5532 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.647619 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2806) took 1151 ms. Trace:
I20260709 12:19:45.724153 32115 rpcz_store.cc:276] 0709 12:19:44.496163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.336606 (+840443us) service_pool.cc:224] Handling call
0709 12:19:45.336888 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.523929 (+187041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:45.523974 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:45.524030 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:45.524074 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.647410 (+123336us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540043add4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:45.647451 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.647568 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429577984}
I20260709 12:19:45.725719 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.726332 32115 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.652323 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.728641 31968 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:45.654259 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.656039 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:45.656242 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44430.6us
I20260709 12:19:45.731225 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.612255  5792 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:45.734166  5792 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.734583  5792 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:45.734931  5741 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:45.658746 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.659073 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.659544 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:45.661427 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 127463us
I20260709 12:19:45.740454 32655 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:45.741756 31963 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:45.663403 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.665689 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.741770 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2776) took 1662 ms. Trace:
I20260709 12:19:45.742892 31967 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:45.666887 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.657557  5575 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:45.673839 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:45.744206 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 77420.5us
I20260709 12:19:45.671773  5907 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:45.638136  5254 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.678027  5663 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.676203  5842 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.673708 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:45.743146 32120 rpcz_store.cc:276] 0709 12:19:44.079741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.079910 (+   169us) service_pool.cc:224] Handling call
0709 12:19:44.080200 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.130917 (+ 50717us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:44.912210 (+781293us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b1800222640. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:19:45.209659 (+297449us) log.cc:844] Serialized 14 byte log entry
0709 12:19:45.457800 (+248141us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:45.458596 (+   796us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.458776 (+   180us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:45.458917 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.459015 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:45.741494 (+282479us) raft_consensus.cc:1685] finished
0709 12:19:45.741588 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.741714 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61920,"prepare.queue_time_us":781588,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":50504,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":19646,"spinlock_wait_cycles":9610496,"thread_start_us":544936,"threads_started":2,"wal-append.queue_time_us":376015}
I20260709 12:19:45.752022 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.752637 32120 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.745292 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.747015 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.667476  5879 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:45.757848 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 154577us
I20260709 12:19:45.759920 31969 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:45.762168 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.766774 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.763011 31963 raft_consensus.cc:2376] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20. Re-sending same reply.
I20260709 12:19:45.744268  5575 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:45.768939 31972 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:45.787499 32120 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:45.788262 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.789528 32120 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:45.781169 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.748024  5673 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:45.753057  5842 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:45.805771 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.828482 32121 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:45.820690 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2811) took 1219 ms. Trace:
I20260709 12:19:45.818842 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.777341 32115 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:45.767546  5781 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:45.867919 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:45.842293 32112 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:45.868768 32118 rpcz_store.cc:276] 0709 12:19:44.601751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.765892 (+164141us) service_pool.cc:224] Handling call
0709 12:19:45.820433 (+1054541us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54001234d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:45.820615 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1932501248}
I20260709 12:19:45.870096 31967 raft_consensus.cc:2376] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20. Re-sending same reply.
W20260709 12:19:45.873692 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 139993us
I20260709 12:19:45.877982 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:45.805302  5901 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:45.792555  5575 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:45.840092  5909 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:19:45.821458  5713 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:45.817193  5715 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:45.887197 32115 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:45.887814  5750 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.783180  5417 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 12:19:45.888937 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.779323  5748 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 12:19:45.900701 32046 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:45.886386 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.824148  5335 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.768536  5911 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:45.894631 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:45.895045 32112 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:45.812614  5663 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.912545 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:45.880571  5741 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:19:45.912858  5832 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.913887 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:45.914352 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.915030 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.895929 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.917371  5761 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:45.917766  5761 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:45.897472 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.792250  5714 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.098s
I20260709 12:19:45.918562  5761 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.919214  5714 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:45.919649  5714 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:45.919901  5761 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:45.920516  5714 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:45.921730  5714 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:45.923681  5791 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:45.924548  5520 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.915688 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:45.870604  5848 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:45.903064 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.890588 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.907025 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:45.909425 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:45.932037 32043 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:19:45.915849 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.897238 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:45.938889 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:45.939550 32041 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:45.941649 31964 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:45.942092 31963 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:19:45.873559  5876 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.942480 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.896116  5715 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:45.944021 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.945291 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.946046 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.946122 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.884446  5782 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:19:45.955724 31968 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:45.953956 31963 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:45.956925 31972 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 14: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:45.954747 31971 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:45.945483  5417 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:45.898602  5713 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.902338  5750 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:45.953001  5791 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.968621 32119 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:45.969350 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.975605 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:45.976305 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:19:45.978363 31968 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:45.980268 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2821) took 1058 ms. Trace:
I20260709 12:19:45.980921 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:45.981352  5925 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:45.980743 32113 rpcz_store.cc:276] 0709 12:19:44.922139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.513184 (+591045us) service_pool.cc:224] Handling call
0709 12:19:45.513617 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:45.669886 (+156269us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:45.669926 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:45.674376 (+  4450us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.968073 (+293697us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:45.968125 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.968228 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:45.980042 (+ 11814us) raft_consensus.cc:1685] finished
0709 12:19:45.980063 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.980199 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1141,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":5256,"thread_start_us":378663,"threads_started":2,"wal-append.queue_time_us":295333}
I20260709 12:19:45.984519  5929 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:45.985193  5929 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.985739  5929 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.958508  5729 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:45.985666 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:45.995136 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.996119 32038 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:45.952270  5913 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:19:45.998930 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.999540 31971 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.001057 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:45.967206  5417 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:46.006174  5835 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.551s	user 0.005s	sys 0.005s
I20260709 12:19:45.997303  5911 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.996806  5832 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:19:46.007663  5729 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:46.052919 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.000123  5758 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.054556 31966 raft_consensus.cc:2368] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22.
I20260709 12:19:46.006965  5848 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.067535 31971 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:46.073086 31971 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 17. Preceding OpId from leader: term: 19 index: 18. (index mismatch)
I20260709 12:19:46.074956 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:46.083150  5335 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.083355  5876 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.040021  5835 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.084976  5750 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.085875  5835 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.087278 32651 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:19:46.088364  5713 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.090342 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.090502  5720 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:46.092710 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:46.095826  5758 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.053463  5729 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.090979  5532 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:46.095997  5918 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:46.097970  5729 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.098798  5929 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.099661 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2805) took 1618 ms. Trace:
I20260709 12:19:46.100433  5792 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.101660  5929 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.105625  5845 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.100059 32119 rpcz_store.cc:276] 0709 12:19:44.481769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.272958 (+791189us) service_pool.cc:224] Handling call
0709 12:19:45.273260 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.974513 (+701253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.099431 (+124918us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b540012acd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:46.099598 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598700416}
I20260709 12:19:46.115836 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2816) took 1395 ms. Trace:
I20260709 12:19:46.116389 32114 rpcz_store.cc:276] 0709 12:19:44.720765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.343910 (+623145us) service_pool.cc:224] Handling call
0709 12:19:45.344202 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.438046 (+ 93844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:45.438076 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:45.438119 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:45.438155 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.115529 (+677374us) spinlock_profiling.cc:243] Waited 93.6 ms on lock 0x7b540012bbd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:46.115594 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.115758 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215377280}
I20260709 12:19:46.121773 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:46.119995  5918 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:46.125029 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:46.129772 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:46.125957  5729 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:46.131151  5729 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:46.126773  5648 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:46.131713  5648 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.132093  5648 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.138398  5729 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.139690  5761 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:46.140002  5729 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.140205  5761 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.140662  5761 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.141629 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.143976  5806 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:46.145514  5806 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:46.118786  5842 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.149061  5842 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.149689  5851 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:46.149994  5796 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:46.150193 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.150486  5796 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:46.151064  5837 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:46.151425  5796 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.151961  5751 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:46.152809  5796 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.153344  5532 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:46.153738  5532 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:46.124732  5792 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.155385  5532 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.156749  5532 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.158947 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.161547 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.161787 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.160183  5751 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:46.162304 32042 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 25: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:46.162418 32038 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.160148  5715 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.162631  5751 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:46.164196  5879 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.121893  5575 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.164853  5751 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.166440  5575 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.167289 32112 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:46.170251 32038 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:19:46.158020  5417 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.171748 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:46.174485  5714 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:46.175275  5714 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:46.151464  5837 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.176440  5837 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:46.176625  5714 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:46.178414 31970 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2506) took 1641 ms (client timeout 1760 ms). Trace:
W20260709 12:19:46.200546 31970 rpcz_store.cc:269] 0709 12:19:44.537091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.537288 (+   197us) service_pool.cc:224] Handling call
0709 12:19:46.178230 (+1640942us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400455bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:46.178356 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554485120}
I20260709 12:19:46.170832  5827 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:19:46.202050  5827 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:46.202549  5827 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:46.203275  5827 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.204432  5827 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:46.168489  5907 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:19:46.182907  5897 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.606s	user 0.003s	sys 0.000s
W20260709 12:19:46.169974  5723 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.018s	sys 0.031s
I20260709 12:19:46.186654  5796 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:46.187711  5648 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:46.188500 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.195971 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.225395 32042 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:46.197019  5760 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:19:46.197839  5837 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.199378  5761 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.199699 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.200035 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:46.247236 31967 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:46.174001  5715 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.226688 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:46.227733 32653 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:46.220552  5417 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:19:46.252141  5417 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:46.253065  5761 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.232750 32659 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:46.229785  5848 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:46.252496  5417 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:46.220686  5723 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.187s	user 0.018s	sys 0.031s
I20260709 12:19:46.250897 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.267238 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.267825 31967 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.268858  5751 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.269008  5527 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:46.270304  5527 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:46.270771  5527 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:46.271824  5527 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.272713 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.273414 32038 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 12.
I20260709 12:19:46.273627  5527 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.276278 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:46.277105  5782 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:46.277491  5782 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.277823  5782 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:46.278563  5648 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.279636  5798 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.242632 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.283289 32660 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:19:46.269308  5751 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.262286 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.284266 32040 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.255458  5806 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:46.229516  5796 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.290050 32653 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 17. Preceding OpId from leader: term: 19 index: 18. (index mismatch)
I20260709 12:19:46.293540 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.295909 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.297780 32013 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.299486 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:46.300794 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:46.301256 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:46.301640 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.303205 31967 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:46.303325 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.303897 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.285722  5751 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.261833  5854 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:46.305725 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.306583 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:46.307353 32654 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 14.
I20260709 12:19:46.303974 32653 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:46.305788  5720 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.309500  5520 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.309670  5720 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:46.310331  5520 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:46.310406  5720 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.310822  5520 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.313831  5532 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:19:46.307068  5806 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:46.316404  5806 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:46.316816  5806 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.318707  5720 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:46.319465  5898 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.790s	user 0.002s	sys 0.001s
I20260709 12:19:46.321271 32040 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:46.320052  5796 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.264467  5848 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.324131  5720 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.324716  5848 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.304508 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.339360 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.320052 32653 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:46.341439 32659 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:19:46.340924 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.350983 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.304102  5761 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:46.366132  5837 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.330020  5782 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.385175 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.391578 31971 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:46.341948  5750 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:19:46.385710 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 74929.3us
I20260709 12:19:46.356753  5729 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:46.348613  5582 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:46.355870  5445 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:46.404175 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:46.405754 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 53467.3us
I20260709 12:19:46.403817  5445 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:46.399709  5854 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:46.401039  5729 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.414443  5445 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:46.406770  5582 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:46.420553  5254 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.454131 31968 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:46.461238 32660 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:19:46.429020  5729 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.465282 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:46.467681 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:46.473397  5796 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:46.478499  5847 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:46.479095  5847 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:46.479960  5847 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.481034  5782 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.434206  5648 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:46.483999 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.454119 31971 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:19:46.432020  5582 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:46.455574  5869 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:46.490404  5750 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:46.428486  5854 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:46.534272 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60989us
W20260709 12:19:46.536276 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 127750us
I20260709 12:19:46.539479 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.540200 32045 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:46.544689  5854 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.558409 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:46.559357 32654 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:46.561550 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.569245  5924 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:46.569869  5924 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:46.570780  5924 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:46.575304 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 756502us
I20260709 12:19:46.576602 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.578485  5673 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:46.579113  5673 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:46.580897  5673 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:46.581276  5673 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.581586  5673 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.582489  5844 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:46.587930 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:46.589566  5335 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:46.590436  5335 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:19:46.590814  5335 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:46.591611  5335 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.592087  5520 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.592789  5335 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:46.592984 32660 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:46.590428  5809 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:46.594018  5809 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:46.594403  5809 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:46.595170 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.596046  5723 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.596426  5723 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:46.597194  5723 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:46.600919 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.601382  5417 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.602370  5868 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:46.602980 32112 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 12:19:46.604076  5876 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.604569  5876 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.606386  5713 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:46.607144  5713 consensus_queue.cc:1243] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer ec7f26ea46944ea498ad253b1b1819b0 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:19:46.607620  5713 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.607937  5713 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:46.565135  5848 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:46.608702  5713 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.609390  5848 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:46.609957  5713 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.619042  5750 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.537137  5869 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:46.623381  5869 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.624706  5869 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.630942  5876 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.635175 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:46.635834  5924 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:46.637355 32040 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:46.638298  5532 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:46.639070  5532 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:46.609887  5848 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.645823  5417 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:46.503854  5648 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.652786  5648 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.659046 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.659741 32120 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:46.661648 32120 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 12:19:46.662034 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.663508 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.673254 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.675650 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.676455 32116 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:46.678197 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.677481 31865 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 16, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 16 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:46.678841 32120 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:46.680254 32114 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:46.679821 31865 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 19, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:46.598120  5723 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.682147 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.681743 31865 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 15, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 15 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:46.682879 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 104823us
I20260709 12:19:46.683976 31865 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:46.684829 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.711539 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.685359  5844 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:46.690973 32040 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:46.724236 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 324422us
I20260709 12:19:46.692819 32044 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:46.725201 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.726019 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.698385 32045 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:46.713706 31865 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 18, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 18 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:19:46.728344 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 168312us
W20260709 12:19:46.723165  5944 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:19:46.725260 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.695262 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.729487 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.732127 32011 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.732270 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.735404 31865 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 20, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 20 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:46.737605 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:46.739955 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.740445 32114 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:46.743330 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.673681  5445 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.749517 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.751657 32114 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 11.
I20260709 12:19:46.741256 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:46.753774 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.755035 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.755795 32045 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:46.758558 31971 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:19:46.760279 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.762115 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.762838 32660 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:46.763659 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.761466 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.766639 31964 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 23: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:46.767151 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:46.768342 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.769629 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:46.770136 32039 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:46.770433 31970 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:46.640889  5532 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.725934  5877 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.776154 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.773715 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.777029 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.772085 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:46.777648 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.777783 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.778393 32047 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:46.777405 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.779233 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.779599 31964 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:46.782205 31970 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:46.784376 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.779930 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.785526 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.785671 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.775785  5569 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.787258  5569 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.787642  5569 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.788472 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:46.791087 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:46.791322 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:46.791978 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.794848 32121 raft_consensus.cc:1275] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:46.795182 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.789825 32623 leader_election.cc:400] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22.
I20260709 12:19:46.796248 32623 leader_election.cc:403] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:46.761523  5748 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.746860  5761 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:46.808681 32112 raft_consensus.cc:2410] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:19:46.809711 31969 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:19:46.810088 31969 pending_rounds.cc:85] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 8
I20260709 12:19:46.810329 31969 pending_rounds.cc:107] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:19:46.810755 31969 raft_consensus.cc:2889] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:46.812418 31964 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:46.817828  5809 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.819417  5794 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22.
I20260709 12:19:46.820283  5809 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:46.820858 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:46.798736 32040 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:46.761278  5720 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:46.745775  5784 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.822641  5784 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:46.823279  5784 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.823604 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:46.823379  5582 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.824687 32042 raft_consensus.cc:2410] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:19:46.825266 31868 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 231) took 39718 ms (client timeout 15000 ms). Trace:
I20260709 12:19:46.826078  5729 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.832448 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:46.833324 32116 raft_consensus.cc:2410] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:46.799499 31970 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:19:46.740185  5846 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
W20260709 12:19:46.835460  5751 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.000s
I20260709 12:19:46.835947  5751 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:46.837529 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 63901.9us
I20260709 12:19:46.696379  5806 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.797708  5883 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:46.802253  5782 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:46.863021 32047 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:46.863266  5729 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.863539  5947 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:46.864148 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:46.864830 32046 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:46.865183  5582 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.867453 32041 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:19:46.797998 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:46.836019  5569 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.729230  5907 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:19:46.873456  5569 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:46.875799 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 80799.3us
I20260709 12:19:46.876541 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.877624 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:46.879082  5713 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:46.880151  5908 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.880528  5908 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:46.880882  5908 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.883013 32042 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:46.883813  5445 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.863269  5782 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:46.884389  5782 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:46.885226  5782 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.889082  5876 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.889482  5876 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:46.826658 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:46.839401 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:46.839973 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 74159.9us
I20260709 12:19:46.840158 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:46.782226 31865 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 15, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 15 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:46.843109 32038 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:19:46.843945 32120 raft_consensus.cc:2410] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:19:46.800107 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:46.825652 31868 rpcz_store.cc:269] 0709 12:19:07.106664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.110384 (+  3720us) service_pool.cc:224] Handling call
0709 12:19:46.825045 (+39714661us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:35.312504 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:35.394743 (+ 82239us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:35.394779 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:19:35.394797 (+    18us) tablet.cc:662] Decoding operations
0709 12:19:35.424209 (+ 29412us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:35.424271 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:35.424298 (+    27us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:19:35.425434 (+  1136us) tablet.cc:701] Row locks acquired
0709 12:19:35.425465 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:19:35.425609 (+   144us) write_op.cc:270] Start()
0709 12:19:35.425730 (+   121us) write_op.cc:276] Timestamp: P: 1783599575425559 usec, L: 0
0709 12:19:35.425752 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:19:35.461328 (+ 35576us) log.cc:844] Serialized 6292 byte log entry
0709 12:19:36.707245 (+1245917us) op_driver.cc:464] REPLICATION: finished
0709 12:19:37.162882 (+455637us) write_op.cc:301] APPLY: starting
0709 12:19:37.163074 (+   192us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:37.958623 (+795549us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b1800136ae0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bf0e0ec 00007f228bf10af7 00007f228bf11fb0 00007f228fc261bb 00007f228fc26532 00007f228fc6db93 00007f228fb1ade3 00007f228fb1a6ab 00007f228fb1be77
0709 12:19:38.369907 (+411284us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:38.369953 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:38.801570 (+431617us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:38.931900 (+130330us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:19:38.931978 (+    78us) write_op.cc:312] APPLY: finished
0709 12:19:38.932553 (+   575us) log.cc:844] Serialized 148 byte log entry
0709 12:19:39.284723 (+352170us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:39.407851 (+123128us) write_op.cc:454] Released schema lock
0709 12:19:39.423368 (+ 15517us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:39.552166 (+128798us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:39.556147 (+  3981us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2718197,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":455253,"apply.run_cpu_time_us":67544,"apply.run_wall_time_us":2373362,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70943,"num_ops":16,"prepare.queue_time_us":145379,"prepare.run_cpu_time_us":40475,"prepare.run_wall_time_us":650719,"raft.queue_time_us":296165,"raft.run_cpu_time_us":14560,"raft.run_wall_time_us":462738,"replication_time_us":1281287,"spinlock_wait_cycles":15376256,"thread_start_us":1364511,"threads_started":5,"wal-append.queue_time_us":1059308,"wal-append.run_cpu_time_us":39637,"wal-append.run_wall_time_us":2829902}]]}
W20260709 12:19:46.894353 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 93737.6us
I20260709 12:19:46.790961  5760 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:46.857095 32039 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:19:46.851943  5720 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.860109  5520 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.862042 32044 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:46.802690  5946 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:46.802505  5798 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:19:46.901752  5868 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.922880  5868 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:46.821028 32119 raft_consensus.cc:1240] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:46.869787 32040 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:46.905519 32121 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 12:19:46.925177 32121 pending_rounds.cc:85] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 14
I20260709 12:19:46.925431 32121 pending_rounds.cc:107] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:19:46.925658 32121 raft_consensus.cc:2889] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:46.898525 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48062.1us
I20260709 12:19:46.838187  5784 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:46.900516 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61166.9us
I20260709 12:19:46.903645  5751 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.932622  5784 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:46.933051  5806 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:46.933346  5949 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:46.934078  5949 consensus_queue.cc:1243] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:46.926297  5760 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:46.899444  5648 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:46.902863 32113 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:46.919958  5254 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:46.912729 32045 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:46.802917  5870 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:46.797319  5895 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.847899  5441 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.901262  5945 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.926883 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:46.960189 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:46.961041  5823 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:46.892375  5935 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.930517 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:46.940227  5946 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:46.901266 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61186.6us
I20260709 12:19:46.906961 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:46.890290  5876 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.920226  5782 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:46.975250 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68353.9us
I20260709 12:19:46.976079 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:46.980432 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.981451 32120 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:46.984602 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.985615 32118 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:46.986217 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.908020 31865 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 15, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 15 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:46.988076 31865 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:46.951769  5870 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:46.952353  5895 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:46.954536  5441 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:46.901906  5796 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:19:46.993201 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:46.990039 31865 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 16, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 16 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:47.020292  5910 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:19:47.026919  5441 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:47.027515  5844 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:46.923759  5868 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.028065  5844 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:46.998282 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:47.006340  5876 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.999300 32115 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:47.030117  5848 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.013648 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:47.032334  5848 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.032600 32120 raft_consensus.cc:1217] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Deduplicated request from leader. Original: 14.8->[16.9-20.11]   Dedup: 16.9->[18.10-20.11]
I20260709 12:19:47.033121 32120 pending_rounds.cc:85] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 9
I20260709 12:19:47.033372 32120 pending_rounds.cc:107] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:47.033617 32120 raft_consensus.cc:2889] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:47.033680  5761 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:47.017544 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:47.034103 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:47.034215  5761 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.018658 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:47.019497 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 43510.1us
I20260709 12:19:47.020382  5784 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:47.035742  5784 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:47.036154  5784 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:47.036823  5761 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.028947  5868 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:47.013279 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:47.038738  5946 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.024838 31865 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:47.041534 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.042761 31865 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 17, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.044297 31865 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.052464  5895 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.052568 32112 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:47.029060  5844 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.045614 31865 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:47.054859 31865 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 10 to 15, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 15 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.052711  5796 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:19:47.049831  5784 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.059878  5796 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.060528  5796 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.061793  5784 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:47.063151  5844 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:47.056905 31865 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:46.963312  5823 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:47.071447 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58286.2us
I20260709 12:19:47.070789 31865 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:47.007769  5948 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:19:46.943876  5648 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.016388  5846 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:46.950693  5254 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.022668  5949 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:47.076207 31865 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:47.082825 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.083738 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:47.014132  5877 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:47.085803 32112 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:47.022609  5924 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:46.935250  5760 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.021096  5794 consensus_queue.cc:1048] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:19:47.095535 31865 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:47.078047  5907 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:47.097483 32121 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:47.077311  5809 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
W20260709 12:19:47.077834 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66199.4us
I20260709 12:19:47.023130  5723 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:19:47.078575  5823 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:47.103281 31865 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 8 to 15, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 15 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.096694 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:47.015708  5935 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.115576 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:47.117412 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:47.113826  5847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.035s	sys 0.067s
I20260709 12:19:47.119282 31865 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 7 to 17, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.088228  5417 consensus_queue.cc:1059] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:19:47.111116  5948 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.143788 31865 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 17, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.117641  5722 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:47.149673 31865 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:47.116658  5715 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.151453 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:47.118799  5945 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:47.116441  5520 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:19:47.190958  5254 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.122385  5895 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:47.192154  5895 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:47.193085  5895 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.194697  5846 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:47.118971  5877 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:47.195261  5846 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.197283  5846 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.120268  5837 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.198019  5837 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.200223  5837 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:47.201182  5720 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:47.208576  5254 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:47.209023  5254 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:47.211417  5908 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.212641 32116 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:47.213064  5729 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:47.213490  5729 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:47.213657  5908 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:47.214507 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:47.214303  5729 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.215123 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:47.215674  5729 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:47.219801 32116 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:47.155411 32038 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:47.102764  5949 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:47.142797  5847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.036s	sys 0.067s
I20260709 12:19:47.222488 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:47.145438 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:47.161620 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.226338 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:47.226876 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.162168 32654 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:47.167466 32115 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:47.169394 32121 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:47.170746 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.153857  5948 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:47.173482  5953 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:19:47.179232  5673 consensus_meta.cc:287] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.003s
I20260709 12:19:47.179600  5715 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:47.242154 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 96790.2us
I20260709 12:19:47.243131 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.181869 32113 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:47.176363  5868 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:47.184755  5794 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.185997  5648 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.187505  5935 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:47.188275  5796 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:47.111564  5870 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.220799 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.221992 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.248131 31964 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:47.248515 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:47.230803 32118 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:47.236593 32121 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:47.244387 32038 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:47.245417 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:47.209929  5254 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.251813 32045 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:47.241257  5798 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:47.255681  5794 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.252018  5673 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.247135  5945 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:19:47.252843  5784 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.253222  5846 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:19:47.253440  5723 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:47.309458  5907 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:47.297703  5715 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:47.311025  5869 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.000s
I20260709 12:19:47.245311  5782 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:47.233067  5520 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:19:47.258083 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:47.320713  5948 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:47.304949  5953 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:47.304267  5949 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:47.328598 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.371039 32045 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:47.371127 31964 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:47.335001 32117 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:19:47.342701 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.350548 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:47.372634 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:47.376077 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.378435 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:47.379597 32044 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:47.366858  5782 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.257493  5254 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:47.398550  5751 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:47.398918  5750 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.399641  5750 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.407727  5751 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:47.408269  5751 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:47.409265  5751 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.410909  5751 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:47.363812 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:47.326692  5798 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.323112  5532 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:47.335409  5809 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:47.362986  5723 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.438874 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.439333  5723 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:47.439656 31965 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:47.358132  5796 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.393966 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:47.416890 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:47.442355 32044 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:47.442072  5796 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.370195  5335 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.443255  5335 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.444521  5796 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:47.445086 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.435395 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.445686 31970 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:47.437045 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.446470 31972 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:47.437994 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:47.448377 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54511us
I20260709 12:19:47.448202 31965 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:47.438504  5809 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.387218  5935 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:47.437892  5532 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.365561  5784 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.457581 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.484545 32113 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:47.484723 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.485536 31969 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:47.485728  5532 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.450915 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:47.486874  5796 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:47.351725 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:47.472867 31972 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:47.396476  5782 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:47.415374  5750 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:47.466130 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.487787  5877 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:47.489225  5877 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:47.489454 31968 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:47.489665  5877 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:47.487409  5796 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.488445  5520 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:47.445353  5335 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.417716  5794 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:47.453024 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:47.418006  5913 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:47.452217 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.436053  5868 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.435544  5798 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:47.487905 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 136290us
I20260709 12:19:47.498795 32044 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:47.470451  5808 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:47.495893  5796 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:47.509275 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.509491 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.520800 31969 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:47.517180  5868 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:47.521981  5955 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:19:47.522786 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.482821  5935 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.513309 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:47.516114 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.493943  5908 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.514083 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.499593 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.525699 32117 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:47.524027 32654 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:47.527144 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:47.529349 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.530292 31963 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:47.510226  5673 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:47.527230 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:47.529743 32117 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:47.526216 31968 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:47.500115  5913 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:47.523252 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:47.523842 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.535463 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:47.517756  5808 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.539462 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.542872 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:47.526091  5935 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:47.548162 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:47.528528  5335 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.536232  5945 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:47.569765 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:47.523171  5907 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:19:47.531669  5908 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:47.569880  5808 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:47.536098  5854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.017s	sys 0.026s
W20260709 12:19:47.538635  5925 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.026s	sys 0.027s
I20260709 12:19:47.583171  5837 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:47.557129 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:47.599043 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:47.569561  5910 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:47.575590  5729 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:47.603111 32659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.051s	sys 0.106s
I20260709 12:19:47.575893  5945 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:47.621058 32659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.053s	sys 0.108s
W20260709 12:19:47.605125 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48112.9us
I20260709 12:19:47.621853 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:47.582922  5335 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.628767 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:47.603623  5925 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.026s	sys 0.027s
W20260709 12:19:47.602792  5854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.018s	sys 0.026s
I20260709 12:19:47.621093  5910 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:47.630100  5823 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 12:19:47.615614  5868 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:47.625586  5417 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:47.626348  5945 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:47.589094  5913 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:47.646777  5417 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:47.798878 31865 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.018s	sys 0.013s
W20260709 12:19:47.980234 31865 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.019s	sys 0.013s
I20260709 12:19:47.551996 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:47.555135 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:47.559059 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:47.851038 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.635432  5809 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:48.262619 32117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.006s	sys 0.016s
W20260709 12:19:48.263064 32117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.006s	sys 0.016s
W20260709 12:19:48.264178 32116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.010s	sys 0.013s
W20260709 12:19:48.264583 32116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.010s	sys 0.013s
I20260709 12:19:48.265758  5844 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.266703  5844 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.269845  5844 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:48.240892 32121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.004s	sys 0.016s
W20260709 12:19:48.270514 32121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.005s	sys 0.017s
I20260709 12:19:48.272235  5908 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.272660  5870 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.274997  5720 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:48.275415  5720 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:48.276908  5254 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.859s	user 0.016s	sys 0.011s
W20260709 12:19:48.277745  5960 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.674s	user 0.006s	sys 0.006s
W20260709 12:19:48.278899  5925 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.007s	sys 0.036s
W20260709 12:19:48.279521  5925 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.007s	sys 0.037s
I20260709 12:19:48.280462  5925 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:48.281746  5963 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:19:47.606273  5837 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.282936  5837 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.285849 32659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 939) took 1532 ms. Trace:
I20260709 12:19:48.286604  5673 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:47.630008  5876 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:48.297452  5784 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:47.627662  5729 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.301556  5729 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.303020  5848 raft_consensus.cc:3055] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:48.303556  5848 raft_consensus.cc:740] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:48.304891  5848 consensus_queue.cc:260] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.306768 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.309482  5869 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:48.310020  5869 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:48.177733 31964 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:48.178457  5520 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.178683  5715 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:47.647639  5910 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.197811  5966 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:48.182544  5907 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.257371  5761 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.258247 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.263998  5965 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:47.691233  5823 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.281471  5925 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.254338 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.009s	sys 0.023s
I20260709 12:19:48.286403 32659 rpcz_store.cc:276] 0709 12:19:46.753389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.763364 (+  9975us) service_pool.cc:224] Handling call
0709 12:19:46.790708 (+ 27344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:46.800760 (+ 10052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.800790 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:46.801234 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:19:47.621587 (+820353us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:19:47.621625 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.621685 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.285141 (+663456us) raft_consensus.cc:1685] finished
0709 12:19:48.285197 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.285536 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12034,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":820002,"threads_started":1,"wal-append.queue_time_us":1481147}
I20260709 12:19:48.176884  5335 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.273751  5870 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.327440 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.362918 31968 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:48.366983 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.009s	sys 0.024s
I20260709 12:19:48.331214  5713 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:48.327736  5729 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:48.276913  5720 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.379786  5761 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.382602 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 753736us
I20260709 12:19:48.384290 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:48.386389  5970 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.387269  5970 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.390383  5883 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:48.391266  5532 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.392010  5532 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:48.401340  5980 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.143s	user 0.001s	sys 0.002s
I20260709 12:19:48.410868  5827 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:48.411688 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 975) took 1560 ms. Trace:
I20260709 12:19:48.412245 31966 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:48.414057  5907 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.416689  5907 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.417763 32113 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:48.419075 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.419284 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.420063 32118 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:48.420259 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 982) took 1080 ms. Trace:
I20260709 12:19:48.421173  5848 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:48.360558  5844 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:48.429458  5870 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.429811  5844 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.430848 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 878784us
I20260709 12:19:48.431591  5729 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.432737  5844 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.434508  5869 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.435614  5254 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:48.435985 32117 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:48.435889  5869 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:48.441471 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 59221.7us
I20260709 12:19:48.434088  5729 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.451331  5827 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:48.456763  5827 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.457929  5723 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.018s	user 0.010s	sys 0.014s
I20260709 12:19:48.412206 32114 rpcz_store.cc:276] 0709 12:19:46.762018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.762149 (+   131us) service_pool.cc:224] Handling call
0709 12:19:46.762462 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:46.771993 (+  9531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.772023 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:46.772443 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 12:19:46.899803 (+127360us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b1800036a00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:19:47.361079 (+461276us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:47.850047 (+488968us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.850695 (+   648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.850895 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.193996 (+343101us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:48.290499 (+ 96503us) raft_consensus.cc:1685] finished
0709 12:19:48.290556 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.321865 (+ 31309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25354,"prepare.queue_time_us":329297,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":270357,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":12349,"spinlock_wait_cycles":22121088,"thread_start_us":932688,"threads_started":2,"wal-append.queue_time_us":797302}
I20260709 12:19:48.458463  5723 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.460246 32114 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:48.460635 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.461104  5723 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.461373 32659 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:48.461843  5761 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.462221 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2850) took 1461 ms. Trace:
I20260709 12:19:48.462682 32120 rpcz_store.cc:276] 0709 12:19:47.000851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.032081 (+ 31230us) service_pool.cc:224] Handling call
0709 12:19:47.032411 (+   330us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:47.072790 (+ 40379us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:47.087224 (+ 14434us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.555073 (+467849us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:47.555742 (+   669us) log.cc:844] Serialized 52 byte log entry
0709 12:19:47.555903 (+   161us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:47.873625 (+317722us) spinlock_profiling.cc:243] Waited 9.86 ms on lock 0x7b18003d06c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf 00007f228c1e9038
0709 12:19:48.291778 (+418153us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.373065 (+ 81287us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.403183 (+ 30118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.458443 (+ 55260us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b18003d06c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:48.461809 (+  3366us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.461858 (+    49us) raft_consensus.cc:1685] finished
0709 12:19:48.461871 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.462050 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31678,"prepare.queue_time_us":166903,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":4831,"raft.queue_time_us":48998,"raft.run_cpu_time_us":540,"raft.run_wall_time_us":42681,"spinlock_wait_cycles":26560640,"thread_start_us":811586,"threads_started":2,"wal-append.queue_time_us":1070205}
I20260709 12:19:48.465550 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.466493 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:48.467428 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 83481.5us
I20260709 12:19:48.467697  5784 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:48.468374 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:48.470399  5876 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:48.453298  5417 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.471097  5965 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.471969  5417 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:48.474998  5798 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.951s	user 0.010s	sys 0.014s
I20260709 12:19:48.475564  5798 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.477562 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:48.477902  5798 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:48.478554 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46777.9us
I20260709 12:19:48.470780 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2818) took 1146 ms. Trace:
I20260709 12:19:48.405889  5532 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:48.454671 32119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.009s	sys 0.024s
W20260709 12:19:48.490087 32119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.009s	sys 0.025s
I20260709 12:19:48.376430  5520 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.458477  5854 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:48.491144  5877 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.992s	user 0.008s	sys 0.008s
I20260709 12:19:48.491683  5854 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.492949 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.493211  5520 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.494578  5846 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.495250  5823 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.495375  5846 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.496179  5823 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.498840  5969 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:48.499495  5782 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.006s	user 0.019s	sys 0.006s
I20260709 12:19:48.499614  5969 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.500111  5782 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.501915  5808 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.895s	user 0.006s	sys 0.011s
I20260709 12:19:48.405324  5966 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.470173 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:48.481413 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:48.510428  5970 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.504547 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:48.513090  5994 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.507575 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.506480  5907 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.513767  5994 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.514748  5907 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.503561 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 835) took 1562 ms. Trace:
I20260709 12:19:48.504875  5527 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:19:48.508878 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:48.507999 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.517961 32655 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:48.516695 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.520874 31972 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:48.522289  5907 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.523106 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.491707  5877 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.524181 32658 raft_consensus.cc:2368] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:48.503643  5913 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.481925  5729 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:19:48.508816  5966 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.518580  5870 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.479732 32115 rpcz_store.cc:276] 0709 12:19:47.324655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.326984 (+  2329us) service_pool.cc:224] Handling call
0709 12:19:47.366111 (+ 39127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.373011 (+  6900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.373040 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:47.373487 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 12:19:48.420701 (+1047214us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:48.420774 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.420870 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.456346 (+ 35476us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b1800481360. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:48.470485 (+ 14139us) raft_consensus.cc:1685] finished
0709 12:19:48.470520 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.470725 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6486,"raft.queue_time_us":19686,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":10406,"spinlock_wait_cycles":4862336,"thread_start_us":961284,"threads_started":1,"wal-append.queue_time_us":960602}
I20260709 12:19:48.515743 32040 rpcz_store.cc:276] 0709 12:19:46.941380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.942066 (+   686us) service_pool.cc:224] Handling call
0709 12:19:46.942506 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:46.959667 (+ 17161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.959696 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:46.965141 (+  5445us) log.cc:844] Serialized 52 byte log entry
0709 12:19:47.551839 (+586698us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:47.558917 (+  7078us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.560301 (+  1384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.560395 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.280728 (+720333us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:48.503043 (+222315us) raft_consensus.cc:1685] finished
0709 12:19:48.503159 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.503359 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16789,"prepare.queue_time_us":6416,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":21159,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":35640,"spinlock_wait_cycles":48557312,"thread_start_us":497195,"threads_started":1,"wal-append.queue_time_us":687686}
I20260709 12:19:48.494220  5854 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.496970  5823 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.502214  5969 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.420943 32121 rpcz_store.cc:276] 0709 12:19:47.340307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.343407 (+  3100us) service_pool.cc:224] Handling call
0709 12:19:47.343820 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.360415 (+ 16595us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:47.544509 (+184094us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.271177 (+726668us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.272067 (+   890us) log.cc:844] Serialized 26 byte log entry
0709 12:19:48.272282 (+   215us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:48.272317 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.272413 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.419925 (+147512us) raft_consensus.cc:1685] finished
0709 12:19:48.419958 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.420179 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38768,"prepare.queue_time_us":278965,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":35823,"raft.queue_time_us":18510,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":5992,"thread_start_us":903591,"threads_started":2,"wal-append.queue_time_us":828535}
I20260709 12:19:48.502399  5808 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.508149 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:48.527818 31964 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:48.537842 32653 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 12:19:48.543128 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 61905.5us
I20260709 12:19:48.543399 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.552697 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:48.592423 32658 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:48.551823 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.505381 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.569283 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2861) took 1064 ms. Trace:
I20260709 12:19:48.528051 31968 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.540496 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2838) took 1562 ms. Trace:
I20260709 12:19:48.552552 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:48.549109 32115 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:48.565421 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 983) took 1224 ms. Trace:
I20260709 12:19:48.526269 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.582767 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:48.658444 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 150388us
I20260709 12:19:48.575137 32117 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:48.562727  5945 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.874s	user 0.014s	sys 0.010s
W20260709 12:19:48.537511  5751 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 1.126s	user 0.013s	sys 0.014s
W20260709 12:19:48.525478  5935 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.937s	user 0.015s	sys 0.012s
I20260709 12:19:48.568395 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2561) took 1010 ms. Trace:
W20260709 12:19:48.528206  5796 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.998s	user 0.013s	sys 0.011s
I20260709 12:19:48.668506 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.549283  5720 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:48.498234  5846 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.670405  5335 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:48.525799  5877 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.554472 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.553169  5970 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:48.668013 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:19:48.576803  5823 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.675433 32113 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:48.531648  5527 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:48.535650  5925 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.564720  5723 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.584347  5994 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.680208 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.677214  5335 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:48.593796 32116 rpcz_store.cc:276] 0709 12:19:47.505151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.532435 (+ 27284us) service_pool.cc:224] Handling call
0709 12:19:47.532800 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.535182 (+  2382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.535211 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:47.535645 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:19:48.265394 (+729749us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:48.265477 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.265606 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.569084 (+303478us) raft_consensus.cc:1685] finished
0709 12:19:48.569107 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.569231 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3008,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":9022,"thread_start_us":729090,"threads_started":1,"wal-append.queue_time_us":1031592}
I20260709 12:19:48.542274  5910 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:48.683347 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 130984us
I20260709 12:19:48.685038 32114 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:48.679473 31969 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:48.676935 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.554975  5520 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.123s
I20260709 12:19:48.666258  5965 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.692171 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:48.530395  5961 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:48.544876  5870 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.690917 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.594794 32119 rpcz_store.cc:276] 0709 12:19:46.977793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.979271 (+  1478us) service_pool.cc:224] Handling call
0709 12:19:46.979673 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.979924 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:19:47.024413 (+ 44489us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.204928 (+180515us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b18002e3f00. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:19:48.286671 (+1081743us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:48.539829 (+253158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:48.539949 (+   120us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:48.540078 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.540221 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.540424 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44259,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":59812,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2100,"spinlock_wait_cycles":36274304,"thread_start_us":1285344,"threads_started":2,"wal-append.queue_time_us":1246586}
I20260709 12:19:48.677634  5417 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:48.694914 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:48.696177  5910 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.685595  5883 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.691639  5532 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.530628  5441 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.150s
I20260709 12:19:48.663895 32112 rpcz_store.cc:276] 0709 12:19:47.340727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.341535 (+   808us) service_pool.cc:224] Handling call
0709 12:19:47.341958 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.359757 (+ 17799us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:47.850191 (+490434us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.455309 (+605118us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.505708 (+ 50399us) log.cc:844] Serialized 26 byte log entry
0709 12:19:48.505846 (+   138us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:48.506533 (+   687us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.509742 (+  3209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.564860 (+ 55118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.564926 (+    66us) raft_consensus.cc:1685] finished
0709 12:19:48.564948 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.565188 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105089,"prepare.queue_time_us":259123,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":106209,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":6042,"thread_start_us":622423,"threads_started":2,"wal-append.queue_time_us":335218}
I20260709 12:19:48.698829 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.699038 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:48.678033  5808 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.706566 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.681273  5784 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:48.530941  5648 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.157s
I20260709 12:19:48.683233  5527 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:48.704540 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.685938  5335 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:48.709964 32121 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:48.711589 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44100.8us
I20260709 12:19:48.705116 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:48.718374 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 188090us
I20260709 12:19:48.719753 32044 raft_consensus.cc:2393] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:48.719336  5966 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:48.722268 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:48.723179 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 253155us
I20260709 12:19:48.722942  5532 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.725538  5532 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:48.726958 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54165.8us
I20260709 12:19:48.727183  5648 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:48.702527  5876 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.745961  5844 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:48.746666  5876 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.747645  5844 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:48.748137  5844 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:48.749058  5844 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.698333  5794 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.687906  5713 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.750494  5794 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.750622  5713 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.751469 32117 raft_consensus.cc:2393] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:48.752852 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:48.716506 32120 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:48.707679  5417 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:48.753773 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:48.753943 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.752876  5794 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.754724 32120 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:19:48.756057  5987 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:19:48.756660  5987 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.757112  5987 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:19:48.757262 32120 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:48.713014 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:48.733034 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.759660 32120 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 23 index: 10. (index mismatch)
I20260709 12:19:48.759893 32655 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:48.760618 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.733275  5877 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:48.762094 32114 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.734972  5527 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.735601 32658 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:48.728739  5883 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.764693  5870 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.722026  5996 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:48.734036  5989 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:48.739109 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.740098 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.740571  5913 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.741035  5961 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:48.697696  5837 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.742599 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.743692  5715 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:48.790154 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.743492 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:48.743585 32011 leader_election.cc:400] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
I20260709 12:19:48.750128  5844 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.752998  5713 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.677441  5945 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.753707  5417 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:48.731745 31968 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:48.764746  5883 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.736284  5723 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.823463  5945 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.765316  5994 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.684582  5796 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.771466  5848 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:48.766578 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.673408 31970 rpcz_store.cc:276] 0709 12:19:47.557483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.557620 (+   137us) service_pool.cc:224] Handling call
0709 12:19:47.557995 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.593696 (+ 35701us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:48.294255 (+700559us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.565818 (+271563us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.566430 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:19:48.567783 (+  1353us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:48.567838 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.567961 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.568024 (+    63us) raft_consensus.cc:1685] finished
0709 12:19:48.568043 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.568222 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92303,"prepare.queue_time_us":604164,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":81,"raft.queue_time_us":120358,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":3241,"thread_start_us":570414,"threads_started":2,"wal-append.queue_time_us":225630}
I20260709 12:19:48.783188 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.730170 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.680785  5935 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.784533 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.787703 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:48.786613 32656 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:48.744727  5910 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:48.792098 32011 leader_election.cc:403] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:48.714396 31969 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:48.786203  5989 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:48.791579 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48156.8us
I20260709 12:19:48.791507 32659 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:48.750099  5876 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.799503  5895 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:48.801373  5335 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.852885  5895 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:48.853842  5895 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.854417  5335 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.802507  5961 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.855162  5895 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.802891  5782 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:48.803242 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57613.3us
I20260709 12:19:48.860149  5532 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.784489  5996 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.861304  5996 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:48.862848 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:48.807783  5837 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.863750 32657 raft_consensus.cc:2368] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:48.808223  5913 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.865660  5913 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.809152  5970 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:48.866211  5970 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:48.866786 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:48.867215  5970 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.868024 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:48.869678 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:48.870029 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.870532 31965 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:19:48.802918 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:48.815486 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:48.872418 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57010.7us
I20260709 12:19:48.873359 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:48.849331 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 106098us
W20260709 12:19:48.816028 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65000.3us
I20260709 12:19:48.875773  5760 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:48.876142  5760 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.876461  5760 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.876709  5945 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:48.831251  5723 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:48.879860  5848 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:48.880302  5848 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.880697  5848 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.837566  5994 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.882472 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:48.885136 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.885550 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:48.885839 32115 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:48.886868 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:48.887118 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:48.887712 32045 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:48.889861 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:48.890623 32113 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.863793 32044 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:48.895149 32045 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:48.895711 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:48.896778  5994 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.897966  5782 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:48.898407  5782 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:48.898782  5782 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.899744  5527 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.902253  5854 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:48.902690  5854 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:48.903693  5854 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.904961  5854 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.908213 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:48.909628  5945 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:48.910471  5961 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:48.912568 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.913295 32115 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:48.914577 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:48.914669 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:48.901121  5987 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:48.917560  5970 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:48.836997 32114 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:48.844712 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:48.838792 32119 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:48.846061 32652 raft_consensus.cc:2368] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:48.841572  5870 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:48.927866  5870 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:48.928414  5870 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:48.929206 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84597.5us
I20260709 12:19:48.849284  5796 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.927834  5417 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:48.851418  5935 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:48.852023 32658 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.923965 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:48.915182 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:48.924883 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:48.930776  5796 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:48.849392  5989 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:48.934594 32653 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:48.940236 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:48.942811 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49079us
W20260709 12:19:48.954178 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.961704 32011 leader_election.cc:400] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:48.962069 32011 leader_election.cc:403] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:48.964150 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.963901 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.942804  5520 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:48.961236 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:48.953063  5723 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:48.968576 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:48.942653  5994 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:48.975237 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:48.967039 31964 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:48.971289 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:48.975291 32011 leader_election.cc:400] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:48.973654 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.946879  5796 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:48.972239 31965 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.976861 32011 leader_election.cc:403] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:48.976903 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:48.978688 31969 raft_consensus.cc:2410] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:48.960180  5913 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:48.983201 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:48.969924  5520 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.994935 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.980620  5723 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:48.999348 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.002689 32113 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:48.980669  5648 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:49.013083 31966 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:49.014212 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.014655 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.018390 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.018498 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:48.984211  5994 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:49.021436 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:48.986706  5254 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.022444 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:48.983457  5869 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:49.003484  5520 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.027092  5837 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.007175  5969 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.017208  5794 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:49.010566  5648 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.044392 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.037508 31970 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.033475 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.031086 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.036638 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:49.035444  5935 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:49.055719 31969 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:49.056604  5949 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:49.037842  5987 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:49.058928 31968 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:49.064448 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.027778  5808 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.051038  5648 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.020539  5996 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.075488 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.076057 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:49.083436 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47355.3us
I20260709 12:19:49.124121  5994 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.126480  5924 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:19:49.126885 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.134131 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.132776 31966 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
W20260709 12:19:49.135807 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51476.5us
I20260709 12:19:49.127182  5947 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:49.127576 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.127743 31967 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 23 index: 10. (index mismatch)
W20260709 12:19:49.128074 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102681us
I20260709 12:19:49.128446  5907 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:49.126543  5441 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:49.057554  5796 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.075377 32038 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:19:49.082845 31971 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:19:49.159518 32044 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:49.132192  5989 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.143497 32658 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:49.155643 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:49.157397 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.063050  5869 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:49.158623 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.144688  5723 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.152338 31970 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:49.136788 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.071331  5913 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:49.061412  5969 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.157441  5751 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.167376 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.169845  5989 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.170571 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:49.170790 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.162047  5846 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.172852 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.175458 32113 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
W20260709 12:19:49.197495 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60768.8us
I20260709 12:19:49.176956 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.178354 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.182685 32045 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:49.163677  5987 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:49.063570  5949 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.184194 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.158361 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.185631 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.192945 32118 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:49.075824  5808 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:49.175527 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 116115us
I20260709 12:19:49.162640  5417 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.158046  5947 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:49.209827 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:49.206713  5913 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.210502 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.211009 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.210875  5947 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.211992  5913 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.206369 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.212349 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.213078 32039 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:49.213865 32043 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.214462  5823 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:49.199986 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.215580 32047 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.215687 32116 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:49.206445 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.203249 31967 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:49.205174 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.212991 32046 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:49.218080 31970 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.199951 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.223498 31967 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:19:49.223848 31967 pending_rounds.cc:85] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:19:49.224833 31966 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:49.225114 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.228086 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.232838  5808 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.233505  5760 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.234618 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.235391 32113 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:49.236019  5969 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.237614  5848 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.238936 31966 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:19:49.224108 31967 pending_rounds.cc:107] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:19:49.241664 31965 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:49.242422 32121 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:49.243163  5794 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:49.260654 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.263645 31967 raft_consensus.cc:2889] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:49.263546 31971 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:49.234447  6011 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:49.268144 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:49.268993 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.216790  5910 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.263311 31966 pending_rounds.cc:85] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 9
I20260709 12:19:49.271750 31966 pending_rounds.cc:107] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:19:49.272168 31966 raft_consensus.cc:2889] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:49.273556 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.216836  5823 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:19:49.270576  5975 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:19:49.219441  5907 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.276158  5760 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.283949 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:49.297365 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.281738 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.191004  5441 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:49.299535 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.268765  5989 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:49.279876 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.269644  6011 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.303771 31972 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:49.303715 32658 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:49.304378 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:49.309737 32653 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:19:49.314743 31970 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:49.316460 32653 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:19:49.207043  5846 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.318620 32047 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:49.246840  5870 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.283247  5823 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:49.326279 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.222563  5751 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.301573  5969 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:49.325747 31969 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:49.332401 31968 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
W20260709 12:19:49.334856  5976 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:49.335218  5869 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:49.337057 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65880.5us
I20260709 12:19:49.338025 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.338459  5876 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.339298  5876 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.339462 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:49.345026 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.320289  5794 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:49.321267  5996 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.323038  5844 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:19:49.347016 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 128308us
I20260709 12:19:49.305024  5910 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.320803  5913 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.357707 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.322553  5989 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:49.354455 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:49.317739  6011 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.353732 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.357192 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.356923  5969 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.365334 32119 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:19:49.338181  6007 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.215s	user 0.004s	sys 0.000s
I20260709 12:19:49.358299 32114 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.359315 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.359454 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:49.362051 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.363425 32042 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:49.338164  5441 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:49.396514 32041 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:49.363679 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:49.400233  5907 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.363893 32116 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:49.371790  5648 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.401916  5254 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:49.402648 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:49.403239 32659 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.403287  5648 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.373226 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.359117  5520 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.358666  5794 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:49.378521  5823 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.379299  5848 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:49.379721  5870 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:49.380190  5532 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.006s
I20260709 12:19:49.358875  5335 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.380721  5913 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.381361  5751 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:49.387563 32046 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:49.389870 32118 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:49.423633 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.424225 32116 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.402662 31963 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:49.349642  6020 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:49.410754 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.414052 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.427659 32047 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:49.414628 32045 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:49.415104  6016 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:49.417093 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.419116 32113 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:49.420054 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.382027  5844 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:49.426872 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.425220 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:49.437381 32117 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:49.437331 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.439126 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.408380  5782 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.427416  5532 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17.
W20260709 12:19:49.441934  5854 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.008s
I20260709 12:19:49.452849 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.457103 32659 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:19:49.462546 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.457404 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:49.447124  5996 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:49.471782 32116 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
W20260709 12:19:49.473896 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53936.4us
I20260709 12:19:49.475606  5844 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.456912  5520 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:49.483289  5910 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:49.464442  5883 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.457918  6020 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.499887  6020 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.471601 32118 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:49.509323 32040 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:49.517091 32114 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:19:49.521166 31971 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:49.524935 32047 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:49.522073  5948 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:49.539450  5948 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.539904  5948 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.541234  5782 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.464258  6016 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:49.540889 32013 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:49.544675  6022 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:19:49.528416 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63863.4us
W20260709 12:19:49.547811 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71594.4us
I20260709 12:19:49.528203  5996 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:49.548676 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.549580  5848 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:49.549257  5996 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.550091  5848 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:49.550869 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.551092  5848 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.552855 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.553917 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.554864 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.544803 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:49.542532  6016 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.555658 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.556918  5876 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:49.557741  5876 raft_consensus.cc:2764] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:19:49.558481  5876 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:49.558557 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:49.559567  5854 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.560040  5854 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.560384  5854 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.561856 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.563380 32047 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:19:49.562711 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:49.573436 32047 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:19:49.575601 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.572956 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:49.585830 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.585359 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:49.592495 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:49.559015  5876 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:49.594238  5808 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:19:49.593338 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45126.8us
I20260709 12:19:49.514628  5989 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.597316 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.531050  5823 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:49.609138 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 64686.9us
I20260709 12:19:49.611315 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:49.618378  6024 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:49.618836  5969 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.619266  6032 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:49.619758  6032 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:49.593228  6022 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:49.620128  6032 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:49.620832  6022 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.621708 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.589449  5913 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:49.603524  5837 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 12:19:49.627487 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.628152 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.628825 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.602895  5846 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:49.633437 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.634723 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:49.637203 32653 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:49.609499  5782 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:49.635543 32041 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:49.653877 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81002.1us
I20260709 12:19:49.622929  5870 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:49.660360 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.617805  6011 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.611151  5760 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:49.518708  5794 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.663849 32047 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:49.670557 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:49.672466 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.677098 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.678102 32047 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:49.682785 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.659168  5532 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:49.662137  5837 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:49.637264  6024 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:49.688233 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.689884 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:49.720364 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.721276 32043 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:49.690897  6031 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:49.692607  6046 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:49.695497  6045 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:19:49.698416  5846 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:49.699669  5876 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.700709 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:49.702013  6026 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.229s	user 0.000s	sys 0.003s
I20260709 12:19:49.705250 32113 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:49.705286 31964 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:49.684402 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:49.710793 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.720542 32652 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:49.713403  5532 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:49.730012 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52116.2us
I20260709 12:19:49.671672  5782 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:49.753206 32651 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:49.740100 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 164444us
I20260709 12:19:49.740870 32657 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:49.756958 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.758052 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:49.759852 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:49.761180 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.752395 32041 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:49.748749  5949 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:49.741744  6045 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:49.770982 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.775856 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.776739  5895 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:49.777112  5895 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.777470  5895 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.780395  6022 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:49.762879  5532 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:49.760991  6032 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.788630 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.769629  5837 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.755125  5760 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.784313  5913 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:49.802204 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:49.804613 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:49.812049 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46448.8us
I20260709 12:19:49.812620 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.813172 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:49.814986 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 101952us
I20260709 12:19:49.815491 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:49.787182  5949 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.803153  6024 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:49.831717  6048 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:19:49.829970  5969 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.866608 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:49.867398 31969 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.832494  5782 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.833570  5837 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.830197  5848 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.836900  6045 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:49.841296  6018 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:49.878458  5948 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:49.879531  6023 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.415s	user 0.000s	sys 0.002s
I20260709 12:19:49.841995 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:49.843058  6038 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:19:49.844167 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.845983 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.883989 32121 raft_consensus.cc:2410] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:19:49.822486 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.787277  5723 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:49.894730 32042 raft_consensus.cc:2368] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:19:49.866520 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:49.895556 32114 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:49.869268 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53853.5us
I20260709 12:19:49.897645 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:49.811815  5846 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.899008 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 844) took 1175 ms. Trace:
I20260709 12:19:49.899443 32040 rpcz_store.cc:276] 0709 12:19:48.713797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.885241 (+171444us) service_pool.cc:224] Handling call
0709 12:19:49.888992 (+1003751us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540032a854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:49.889136 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351499520}
I20260709 12:19:49.867142  5969 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:49.901680 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 79355.8us
I20260709 12:19:49.902490 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:49.871495  6020 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.816161  5823 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:49.831063  6011 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:49.816357  5961 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.875375  5869 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.865355  6022 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
W20260709 12:19:49.888140 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.868109  5335 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:49.811530  6032 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:49.834291  6042 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:49.900631 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:49.888608  5527 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:49.901710 31936 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:49.880687  5987 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:49.927839  6039 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:19:49.929567  6004 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.014s	sys 0.047s
I20260709 12:19:49.911379  5961 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:49.928762  6041 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:19:49.906721  5996 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:49.927325  5648 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:49.878495  6018 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:49.928478  6020 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.908756  6040 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:49.909467  5846 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.904381 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:49.944643 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:49.900111  5723 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:49.911399  5854 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.970459  6032 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:49.992368 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 148294us
W20260709 12:19:49.982265  6004 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.014s	sys 0.052s
I20260709 12:19:49.908365  5969 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:49.999266 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54682.2us
I20260709 12:19:49.980266  5848 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.980469  5254 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:49.944463  5869 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:49.983634  5961 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.986220  5996 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:49.993726  6040 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.010246 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:50.011752 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 110130us
I20260709 12:19:50.001394  5723 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:50.022432 31971 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:49.989270  5987 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.003719  5441 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:50.022178  5876 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:50.031466  6031 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:50.028736  5254 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.053508 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.054175 31965 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.054956 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.008451  5648 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:50.057544  5648 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.023829  6040 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.068228  5895 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.078303  5876 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:50.021571  5996 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:50.047983 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.053062  6031 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.002903  5527 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.004241  5948 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.097492 31965 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:50.104815 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.097661 31963 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:50.011272  5848 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.100226  6031 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:50.105883 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62448.6us
I20260709 12:19:50.081661  5837 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:50.143436 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 121301us
W20260709 12:19:50.143628 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 239293us
I20260709 12:19:50.146587 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:50.058197  5648 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.150643 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:50.155436 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.156190 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.159614  6060 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:50.160086  5913 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:50.160467  6060 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:50.160660  5913 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.161250  6060 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:50.161650  6060 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:50.161983  6060 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.164834  5760 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.167452  5854 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.168133  5854 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:50.169438  5844 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:50.170135  5844 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.174942  5846 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.175450 32657 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:50.175724  5532 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.176728  5648 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.177496 32045 raft_consensus.cc:2368] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:50.177851  5532 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.179445  6020 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.179286  6060 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.182060  6060 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.183636  5883 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:50.184304  5883 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.153834  5823 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:50.100593  6018 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.196774 31969 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
W20260709 12:19:50.199322 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 167488us
I20260709 12:19:50.189808  5883 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.198840  5907 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:50.202360  5848 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.199133  5823 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.205487  5823 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.206568  5913 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.208608  5913 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.209363  6040 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.210752 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:50.211432 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:50.212179  5532 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:50.213969  6040 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.214214  5532 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:50.215318  5527 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.219055 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:50.145774  5876 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.221457 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.237304  6031 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.245324 32113 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:50.227877  5948 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:50.259688 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103534us
I20260709 12:19:50.215787  5532 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:19:50.260569 31969 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:50.261068 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.263513 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.231748  5254 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.227722  5837 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:50.228520  5729 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:19:50.272316 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.273087 31969 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:50.271451 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.247604  6025 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:50.276003 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.274814 31971 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:50.274699  5913 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:50.277684  6025 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:50.284054  6025 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.263792  6058 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:50.266830  5527 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:50.292713 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.296998 31971 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:50.297816 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:50.298560 32658 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.302466  5913 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:50.305883  6031 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.295185  6040 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:19:50.273089  5760 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 12:19:50.317890  6057 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 12:19:50.302958  6058 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.323072  6018 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.323858  5869 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.340260 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.343071 32121 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:50.343206 31969 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:19:50.343622 31969 pending_rounds.cc:85] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 14
I20260709 12:19:50.343869 31969 pending_rounds.cc:107] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:19:50.343796  5996 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.344337 31969 raft_consensus.cc:2889] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:50.338143 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.345522 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.345991  5996 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.369637 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.370509 32046 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:50.340314 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.340736 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.341114 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.300773  5723 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.274698  6063 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:19:50.376859 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.377580 32117 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:50.377714 31935 leader_election.cc:400] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:19:50.378068 31935 leader_election.cc:403] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:50.379371 31935 leader_election.cc:400] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:50.379635 31935 leader_election.cc:403] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:50.343991 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.335117 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.338450 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.346285 31963 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:50.346414  5870 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:50.334038 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.405983 32113 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:50.350409  6058 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.351781  5987 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:50.356493 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 107833us
I20260709 12:19:50.367300 32658 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:50.433013 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.435268 32120 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:50.439086 32118 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:50.440464 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.440508 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:50.441107 32041 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.440708 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.441653 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.442220 32043 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:50.442261 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.443125 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.445436 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.445997 32113 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.438251 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.440327  5823 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:50.449620 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:50.456919 32116 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:50.457037 32657 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:50.460191 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:50.436110 32084 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.576s (SENT) [suppressed 6 similar messages]
I20260709 12:19:50.460502 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.461534 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:50.463317 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.465010 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.466115 32115 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:50.465982 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.466987 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.479286 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.376641  5869 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.440513  6065 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.463930 32042 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:50.480805 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.375748  6044 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.482930 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.490688 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.491325 32115 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:50.483351 32654 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:50.492627 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.483844 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.493507 31968 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:50.484759 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.486413 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.455489  5907 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:50.479931 31965 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.496649 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.497574  6044 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.499404  6061 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:50.520125 32040 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:50.493628  5969 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.482579  5848 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:50.502215 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.534729 31970 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 15: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:50.523718 31965 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:50.502591 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.502913 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.503211 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.503811 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.512341 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:50.516912 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52226.8us
I20260709 12:19:50.536687 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:50.517964 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.540292 32046 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:50.518595 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.532889 32113 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:50.499084  5907 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.543280 31968 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:50.487528  6024 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:50.518281  6040 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:19:50.548766  5948 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:50.517171  6044 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.584640 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.549821 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.535105  5848 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.568869 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:50.490499  5869 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.535846  5782 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:50.568213 32041 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:50.585868 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.588333 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:50.585402 32117 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 16.
I20260709 12:19:50.513149  5961 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:50.589426  6052 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:50.604849 31969 raft_consensus.cc:2410] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
W20260709 12:19:50.605027  6059 log.cc:927] Time spent T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.541s	user 0.010s	sys 0.000s
I20260709 12:19:50.609535 32115 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:50.613457 32115 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:50.623009 32047 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:50.624706 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:50.627106 32115 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:50.544858  6064 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:50.648452 32041 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:50.648892 32119 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:50.598323  6043 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.622s	user 0.012s	sys 0.005s
I20260709 12:19:50.570211  5907 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:50.649802 32044 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:50.583531  5948 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:50.611924  5848 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:50.649740 31963 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:50.654800 32115 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:50.657894  5870 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:50.648820  6049 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.660s	user 0.003s	sys 0.000s
I20260709 12:19:50.664069 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:50.665319 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 229215us
I20260709 12:19:50.668516 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.669241  6044 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:50.669307 31966 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:50.670220  6044 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.653756  6046 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.672729 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.602751  6024 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.664825  5870 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.682780  6042 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:50.674747  5895 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.685211 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.629400  5782 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.692358  6044 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.711290 31971 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.652804  6064 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.701354 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.727103 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.734850 31971 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:50.718377  6040 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:50.724988  6045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.016s	sys 0.045s
I20260709 12:19:50.705920  5876 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:19:50.747931 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.764106 31965 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:50.764292 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.693616  5987 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:19:50.722362  6061 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.735428 32118 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:50.726866  5869 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.767863 32041 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:19:50.732239  6042 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:50.742794  6031 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:50.744750  6040 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:50.704164  6046 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.781898  6031 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.775358 32038 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:50.743865  5723 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.738183  6066 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:50.772397 32047 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:50.794979  6042 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.787690  5876 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:50.748806  6044 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:19:50.796531 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.865895 32041 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:50.767680  5969 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.864663 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:50.760954  6045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.016s	sys 0.048s
W20260709 12:19:50.798381  6025 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.004s
I20260709 12:19:50.878214 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:50.797510  6040 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.885216 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.882716 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:50.745889  5948 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:50.877619 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 143036us
I20260709 12:19:50.860860  5876 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:50.831331  5870 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.902005 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.825711  6064 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:50.915268 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.901407 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:50.861788  6066 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:50.786352  6061 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.904022 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:50.886718  5869 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:50.943567  6070 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.420s	user 0.000s	sys 0.001s
I20260709 12:19:50.945966  6045 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.946625  6045 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:50.906550 32047 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:50.950716 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:50.899933  6016 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:50.951494  6016 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:50.951668 31965 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:19:50.919406  6066 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:50.952612  6016 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.953508 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:50.906251 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:50.955340 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49166.1us
I20260709 12:19:50.956044 32013 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:50.956844 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:50.957613  6063 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:50.958298  6063 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:50.897444  6025 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.958803  6063 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:50.959475 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.959664  6063 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:50.960216 32042 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.960361  6025 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.961061  6025 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:50.961395  6025 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.961755  6025 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:50.951709  6061 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.963009  5729 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:50.964211  5949 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:50.964560  5949 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:50.965390  5949 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:50.966459 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49823.7us
I20260709 12:19:50.967085 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:50.967231  5996 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:50.968405 31965 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:50.949172  6045 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:50.863808  6031 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:50.931929  5961 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:50.932171  5948 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.919667  5969 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:50.916590  5913 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:50.935745  6064 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:50.909489  6014 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:50.936690 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2847) took 1095 ms. Trace:
I20260709 12:19:51.017100 32116 rpcz_store.cc:276] 0709 12:19:49.841319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.842494 (+  1175us) service_pool.cc:224] Handling call
0709 12:19:49.842788 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:50.457392 (+614604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:50.936430 (+479038us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54002b5cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:50.936616 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411823360}
I20260709 12:19:50.938769 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:50.939062  5823 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:50.939790  5876 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.022811  5876 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:51.006049  5895 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:19:51.023360 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.006253  5869 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:51.024271 32046 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:51.015441  6064 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.025210  6064 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.025451 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:50.827363  6024 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:50.982762 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67599.9us
I20260709 12:19:51.026965 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:51.028409  5441 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:51.029024 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:50.983222 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 81870.2us
W20260709 12:19:51.031545  6065 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 12:19:51.031999  6065 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:19:51.032708  6065 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:50.981432  5989 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:51.012169 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.012970 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.010684 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.020936 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:50.990208  6062 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:51.029739  5869 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:50.931800  6046 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.006781 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:51.037674 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.018603 31971 raft_consensus.cc:2376] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19. Re-sending same reply.
I20260709 12:19:51.038223 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:51.038523 31972 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:51.062218 31967 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
W20260709 12:19:51.062384 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55732.5us
I20260709 12:19:51.065547 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:51.070785  5441 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.071208  6046 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:51.071373  5441 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.072446 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.072525 31965 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:51.073576  5441 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.073870 31971 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:51.074064  5527 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:51.074623  5527 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.076565 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.079800 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:51.105466 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.113520 31967 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:51.042830  6062 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:51.128729 31963 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:51.066445  5907 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.137367  6064 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.079638  5869 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:51.138967  5782 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.079387  6024 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:51.142470  5848 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.143802 32042 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:51.116276  5961 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:51.151399 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.034951  5913 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.081010  5808 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:51.122104  6072 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:51.155685  6066 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.156636  6062 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.157310 32046 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:51.159495  5798 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:51.160243  5527 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.164362  6040 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:51.167665 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 91163.5us
I20260709 12:19:51.180045 32038 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:51.306087 32038 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:51.191483 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.307077 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:51.200119 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.308084 32112 raft_consensus.cc:2368] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:51.204200 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.189950  6065 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:51.308805  6065 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.308976 32116 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:51.309199  6065 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:51.311563 32114 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:19:51.312821 31968 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:51.256389 32042 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:19:51.258973 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 276510us
W20260709 12:19:51.268029 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 300857us
I20260709 12:19:51.315083 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:51.316375 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.317405 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:51.273665 32044 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:51.319489  5870 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:51.320757  5844 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.326781  5913 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.331709 32652 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:51.193145  5823 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.286512 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.287523 32047 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:51.338229 32040 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:51.338837  5907 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.339450  5907 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.339880  5907 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.340451  6065 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.341967  5907 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.332800 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.333669 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.334570 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.335412 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.343050 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.398284 32043 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:51.397830 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.398967 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.399199  5729 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:51.368160  6031 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.400853 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.401611  5729 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.343091  6065 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.288550 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.350461  5798 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.405011  5798 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:51.351416  6064 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.405287 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:51.277539  5869 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.352499  5723 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:51.408497 31967 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:19:51.408715  5723 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:51.409768  5723 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.410378  5996 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:51.411058  5723 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:51.411082  5996 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:51.355824 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.412554  5996 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.417625 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.314178  5848 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.360441  5876 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:51.360330 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44024.6us
I20260709 12:19:51.422485 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:51.424540 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:51.365746 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.424986 31866 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 22 to 23, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 23 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.426770  5844 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:51.317960  6025 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.366544  5949 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:51.370105 32046 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:51.337272  5823 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.375149  6040 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.377117  5527 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:19:51.390287 31968 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:51.391710 31972 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:51.391772 31966 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:51.392604 31965 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:19:51.394691 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.396165  5760 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.344202  6066 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.402942  5729 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:51.427057 31866 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 17 to 18, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 18 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.460523 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.461223 31965 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.461274  5823 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.428077  5895 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:51.462564  5760 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.464272 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.465420  5760 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.467561 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.468310 31963 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:51.469468 32045 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2777) took 1106 ms. Trace:
I20260709 12:19:51.427822  5987 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:51.470726  5987 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:51.471136  5987 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:51.469982 32045 rpcz_store.cc:276] 0709 12:19:50.363271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:50.374976 (+ 11705us) service_pool.cc:224] Handling call
0709 12:19:50.456249 (+ 81273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:50.493271 (+ 37022us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:50.682498 (+189227us) log.cc:844] Serialized 14 byte log entry
0709 12:19:51.118638 (+436140us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:51.178612 (+ 59974us) log.cc:844] Serialized 26 byte log entry
0709 12:19:51.178750 (+   138us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:51.179027 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:19:51.179347 (+   320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.179449 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:51.357470 (+178021us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b180030fb80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:51.469212 (+111742us) raft_consensus.cc:1685] finished
0709 12:19:51.469242 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:51.469397 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31206,"prepare.queue_time_us":24834,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":26315,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":23475,"spinlock_wait_cycles":22019840,"thread_start_us":552636,"threads_started":2,"wal-append.queue_time_us":759490}
I20260709 12:19:51.479697 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.480465 32041 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.483261 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.427564 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.433156 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.487166 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.487697 31963 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:51.488114 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60660.6us
I20260709 12:19:51.488502 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:51.491825 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 170710us
I20260709 12:19:51.493697 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.497802 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.498986 32115 raft_consensus.cc:2368] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
W20260709 12:19:51.501365 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.502876 31966 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:51.506093 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.490248 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.508425 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.509235 31968 raft_consensus.cc:2368] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20.
I20260709 12:19:51.440984 32047 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:51.510643 32116 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:51.514837 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.515422 32039 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:51.491271 31866 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 17 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:51.436090 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:51.517382 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 392429us
I20260709 12:19:51.517606 32043 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:51.469835  5844 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:51.521253 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:51.521678 32045 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:51.518893 32112 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:19:51.474656  6031 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:51.534801 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.427630  6046 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.117s
I20260709 12:19:51.427402  6091 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:19:51.486078  5848 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.551545 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.539390 31965 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:51.506296  5989 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:51.558100 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.552708 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.506031  5895 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:51.558665 32117 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:51.556958 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.554087 31866 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.559405 31966 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:51.559993 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.559424 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.560458 32113 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.516042  6083 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:51.506611  6064 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:51.562049 32041 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:51.574162 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.575744 32084 leader_election.cc:400] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20.
I20260709 12:19:51.576987 32084 leader_election.cc:403] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:19:51.580962 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.581578 32112 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:51.585927 32113 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:51.541800  6040 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:51.588516 31938 leader_election.cc:400] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:51.588927 31938 leader_election.cc:403] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:19:51.593662 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:51.529881  6102 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.595185  5823 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:19:51.554613  6058 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:51.599159  6058 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:51.599560  5913 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:51.600028  5913 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:51.601040  5913 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.602919  6044 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:19:51.603847  6090 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20.
I20260709 12:19:51.608422 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:51.576510  6025 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.549115  6014 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:19:51.606389 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:19:51.551559  5848 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.551891  6063 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:19:51.631428 32118 raft_consensus.cc:606] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:51.632978 31866 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 13 to 17, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 17 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:51.633797 31966 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:19:51.613826 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.592406  6040 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.599587  6058 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:51.586723  5782 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:19:51.620951  5527 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.595642  5907 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:19:51.582798  6064 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.627146  5913 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:51.654701 31970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.021s	sys 0.066s
I20260709 12:19:51.610922  5798 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:19:51.596903  6089 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 12:19:51.672700 31970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.021s	sys 0.066s
I20260709 12:19:51.644313  5441 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:51.645218  6040 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:51.644337  6061 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:51.640564  6066 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:51.684835 31969 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.015s	sys 0.007s
W20260709 12:19:51.687801 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.023s	sys 0.033s
W20260709 12:19:51.719419 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.023s	sys 0.034s
W20260709 12:19:51.719911 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 684085us
W20260709 12:19:51.720068 31969 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.015s	sys 0.007s
I20260709 12:19:51.674062  6065 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.756628 31866 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 13, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 13 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:51.745435  5969 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.758046  5969 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.759070 32654 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:19:51.760397  5969 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.739389  6061 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:51.763203  6104 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:19:51.764572  6061 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.750643  6065 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.771819  6065 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.773910 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.774117 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.774847 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.779049 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.780607 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:51.781757  6065 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:51.777994 32039 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
W20260709 12:19:51.782858 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.784607  6062 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.785691 31967 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:51.787405  6064 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.739058  6066 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.789244  5848 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.791754 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.740883  5961 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:51.794304 32120 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:51.794866  5961 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.790110  6066 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:51.677107  5844 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.809854 32117 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:51.738500  5441 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:51.811232  5441 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.813045 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.814136  5848 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:51.814342  5907 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:51.814881  5848 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.815838 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:51.814888  5907 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.816901  5907 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:51.817898 31865 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 246) took 13163 ms (client timeout 15000 ms). Trace:
I20260709 12:19:51.819868  6062 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.821017 31866 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 17 to 18, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 18 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.816418 31967 raft_consensus.cc:606] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:51.822593 31967 raft_consensus.cc:614] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:51.822981 31866 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:51.826946 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.827731 32118 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:51.829173 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:19:51.829689 32118 raft_consensus.cc:606] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:51.830008 32118 raft_consensus.cc:614] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:51.831214 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:51.835155  5987 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.837595  5987 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.839391 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:51.840065 31967 raft_consensus.cc:606] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:51.840425 31967 raft_consensus.cc:614] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:51.841821  6089 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.842430  6089 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.843866 31967 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:19:51.844460 31967 raft_consensus.cc:606] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:51.844803 31967 raft_consensus.cc:614] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:51.844883  6089 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.846073 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:51.846484  6042 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.847194  6042 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.847954 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:51.849869  6042 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:51.855127 31866 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 13 to 18, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 18 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:19:51.856814 31938 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.858352 32112 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
W20260709 12:19:51.861518 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 285012us
W20260709 12:19:51.850714 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 332664us
I20260709 12:19:51.850515  6083 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:51.818470  5848 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:51.852301 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.872287 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:51.873457  5848 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:51.874011  5848 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:51.852909 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 333008us
I20260709 12:19:51.859943 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:51.863683 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 320098us
I20260709 12:19:51.864568  6062 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:51.896889 32013 leader_election.cc:400] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
I20260709 12:19:51.897379 32013 leader_election.cc:403] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:19:51.899149 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:51.862972 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 288910us
I20260709 12:19:51.903390 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:51.903297 31866 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 13, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 13 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.862915  5823 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:51.910943 31970 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:19:51.902148 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.910533 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:51.862705  5989 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:51.914728 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.912672 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:51.920295 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.922870 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:51.916958 31866 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:19:51.818303 31865 rpcz_store.cc:269] 0709 12:19:38.654491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.704363 (+ 49872us) service_pool.cc:224] Handling call
0709 12:19:51.817782 (+13113419us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:47.718808 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:48.287177 (+568369us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:48.287243 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:19:48.287281 (+    38us) tablet.cc:662] Decoding operations
0709 12:19:48.456237 (+168956us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:48.456342 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:48.456390 (+    48us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:19:48.550638 (+ 94248us) tablet.cc:701] Row locks acquired
0709 12:19:48.550700 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:19:48.550982 (+   282us) write_op.cc:270] Start()
0709 12:19:48.551187 (+   205us) write_op.cc:276] Timestamp: P: 1783599588550879 usec, L: 0
0709 12:19:48.551243 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 12:19:48.552814 (+  1571us) log.cc:844] Serialized 8946 byte log entry
0709 12:19:49.013280 (+460466us) op_driver.cc:464] REPLICATION: finished
0709 12:19:49.749228 (+735948us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f22890c50ac 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228fc05ef0
0709 12:19:49.943087 (+193859us) write_op.cc:301] APPLY: starting
0709 12:19:49.943205 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:50.236238 (+293033us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:50.236282 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:50.348128 (+111846us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:50.348597 (+   469us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 12:19:50.348662 (+    65us) write_op.cc:312] APPLY: finished
0709 12:19:50.349270 (+   608us) log.cc:844] Serialized 204 byte log entry
0709 12:19:50.635850 (+286580us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:50.656938 (+ 21088us) write_op.cc:454] Released schema lock
0709 12:19:50.667202 (+ 10264us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:50.692158 (+ 24956us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:50.692194 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":67948,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":929465,"apply.run_cpu_time_us":156774,"apply.run_wall_time_us":749021,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100175,"num_ops":23,"prepare.queue_time_us":497672,"prepare.run_cpu_time_us":46465,"prepare.run_wall_time_us":1008142,"raft.queue_time_us":184004,"raft.run_cpu_time_us":66746,"raft.run_wall_time_us":1020025,"replication_time_us":459205,"spinlock_wait_cycles":9966720,"thread_start_us":2243287,"threads_started":5,"wal-append.queue_time_us":556828,"wal-append.run_cpu_time_us":64065,"wal-append.run_wall_time_us":2231733}]]}
I20260709 12:19:51.814085  6058 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:51.906793  5527 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:51.929478  6024 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:19:51.926565 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.937167 32045 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:51.937543 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:51.932222 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.939803 32041 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:51.946756 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.952939 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.953423 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.953886 32041 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:51.922273  5723 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:51.924103  6022 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
I20260709 12:19:51.926544  6066 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:51.965248 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:51.934661  6025 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:51.972229 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2831) took 1060 ms. Trace:
I20260709 12:19:51.911909  6112 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:19:51.987442 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.988304 32041 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:51.917680  6062 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:51.978606 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:51.995288 32112 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:51.984167 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.996662 32042 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:51.980571 32657 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:51.985509 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:51.997457 32045 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.982278 31971 rpcz_store.cc:276] 0709 12:19:50.911541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.115720 (+204179us) service_pool.cc:224] Handling call
0709 12:19:51.116223 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.168139 (+ 51916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.168169 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:51.180478 (+ 12309us) log.cc:844] Serialized 26 byte log entry
0709 12:19:51.685388 (+504910us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:51.753099 (+ 67711us) log.cc:844] Serialized 14 byte log entry
0709 12:19:51.754903 (+  1804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.755005 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:51.971988 (+216983us) raft_consensus.cc:1685] finished
0709 12:19:51.972013 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:51.972159 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86265,"prepare.queue_time_us":66125,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":71307,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":3793,"thread_start_us":479440,"threads_started":1,"wal-append.queue_time_us":671287}
W20260709 12:19:52.001973 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 419480us
I20260709 12:19:51.924422  6113 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
W20260709 12:19:51.985996  5869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.015s	sys 0.035s
I20260709 12:19:52.016296 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.874974  5848 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:51.978999  6058 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:52.014941 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.015091 32040 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
W20260709 12:19:51.986964  6033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.017s
I20260709 12:19:52.017141 32042 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:19:52.022410 31971 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:51.955562  5527 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:52.063560 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46192.4us
I20260709 12:19:52.063920 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:52.064262 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44772.1us
I20260709 12:19:52.065305 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2822) took 1317 ms. Trace:
I20260709 12:19:52.008919 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:52.066236 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:51.955874  6040 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.005040  5723 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:52.016767  5869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.015s	sys 0.035s
W20260709 12:19:52.070647 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61816.6us
I20260709 12:19:52.070931 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:52.069278 32041 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:52.003615  5760 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:52.001456  6031 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.017671  5823 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.088999  6091 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:52.095338  5527 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.141947  5823 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.142911 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:52.067435  5989 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.098557  6062 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.150234 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:52.106660 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:52.151458 32112 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 15: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:52.065254  6033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.000s	sys 0.018s
I20260709 12:19:52.018231  6066 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:52.064615  6090 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:52.139652 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:51.874981  5907 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.166795 31970 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:52.144063 31971 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:19:52.063040  6061 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.144816 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.172052 32041 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:52.023343  6089 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.135859 32045 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
W20260709 12:19:52.143182 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.175001 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.154161 32116 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:52.154006 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.154732 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:52.157281 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:52.179003 32042 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:52.063335  5969 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.069880  6022 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:19:52.163125  5723 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.164196  6066 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.198025  5969 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.200438  5969 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:52.180348 32118 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:52.011478  6063 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 12:19:52.185525 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:52.196390 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:52.165251  6090 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:52.167614  6108 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:19:52.214696 32040 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 12:19:52.224097 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 294743us
I20260709 12:19:52.215777 32045 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:52.168047  5844 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:52.197930  6125 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.210893 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:52.221329 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:52.165329  6091 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.201485  6064 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:52.243029 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:52.219870  6033 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:52.168213  6031 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.223420  6090 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:52.250186 32042 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:52.166931  5760 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.284523  5760 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:52.252964  5907 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:52.245450  6042 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:52.261651 31970 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:52.265545 32120 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:19:52.249200  6045 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:52.287115  5961 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.297925 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:52.242722  6064 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.231930  6099 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:52.269558  5869 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.191766  6061 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.278134  6133 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:52.231451  5844 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.241907  6025 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:19:52.291734 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2865) took 1065 ms. Trace:
I20260709 12:19:52.266969  6139 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.270535  6033 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:52.245239  6083 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:19:52.311406 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:52.191461  6089 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.297778  6045 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.245195  6125 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.301899  5823 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:52.293582  5876 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
W20260709 12:19:52.364656 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 473407us
W20260709 12:19:52.365051 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 87728.5us
I20260709 12:19:52.293934  5907 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:52.359349 32121 rpcz_store.cc:276] 0709 12:19:51.225843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.226017 (+   174us) service_pool.cc:224] Handling call
0709 12:19:51.314973 (+ 88956us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:51.358845 (+ 43872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.358876 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:51.359469 (+   593us) log.cc:844] Serialized 52 byte log entry
0709 12:19:51.739352 (+379883us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:19:51.751939 (+ 12587us) log.cc:844] Serialized 14 byte log entry
0709 12:19:51.837613 (+ 85674us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b18001cee20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228905cc0a 00007f229279c219 00007f22890e4a40 00007f22890df4e4 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287 00007f228c1ea208
0709 12:19:51.857015 (+ 19402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.857117 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:52.270882 (+413765us) raft_consensus.cc:1685] finished
0709 12:19:52.291496 (+ 20614us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:52.291657 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47160,"prepare.queue_time_us":14754,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":130648,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":190,"spinlock_wait_cycles":184744704,"thread_start_us":379485,"threads_started":1,"wal-append.queue_time_us":566838}
I20260709 12:19:52.374015  6024 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:52.360873  5844 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:52.398003 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.398777 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.399457 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:52.451683 32040 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:52.245038  6091 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.286567  6097 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:19:52.301272  5961 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.311308  5869 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.557497  5869 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:52.301818  6064 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:52.443886 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:52.401109  6124 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.183s	user 0.005s	sys 0.000s
I20260709 12:19:52.454526 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:52.377211  6045 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.564925 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:52.544375 32040 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:52.536549 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.450359  5907 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.563684 31970 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:52.400909  5823 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.551805 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:52.583827 31968 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:52.446678  6040 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:52.575275 32044 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:52.582764 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:52.577281 31965 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:19:52.575021 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 126877us
I20260709 12:19:52.453686  6024 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.698843 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:52.371335  6139 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.585472  5913 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:52.734009 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 151335us
I20260709 12:19:52.570560  6142 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:52.583084  6062 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:52.708201  6126 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.466s	user 0.001s	sys 0.000s
I20260709 12:19:52.581373  6099 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:52.583369  6115 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:52.515254  6119 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.243s
I20260709 12:19:52.590633  5823 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.761282 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:52.773904 32113 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:52.706467  6101 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:52.777936 32660 raft_consensus.cc:2393] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:52.778991 31970 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:52.778887 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:52.764842  6091 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:52.727505  6097 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
W20260709 12:19:52.789685  6121 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.564s	user 0.001s	sys 0.000s
I20260709 12:19:52.956791 32121 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:52.748477  6058 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:19:52.752113  5913 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.027228 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2625) took 1470 ms. Trace:
I20260709 12:19:53.027802 32047 rpcz_store.cc:276] 0709 12:19:51.556391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.578935 (+ 22544us) service_pool.cc:224] Handling call
0709 12:19:51.579383 (+   448us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:51.634336 (+ 54953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.634377 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:51.645848 (+ 11471us) log.cc:844] Serialized 52 byte log entry
0709 12:19:52.004765 (+358917us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:52.004805 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.004870 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:52.371975 (+367105us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b240010b668. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f228912109d 00007f228bd5310e 00007f228bd6a88c 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221
0709 12:19:53.014791 (+642816us) raft_consensus.cc:1685] finished
0709 12:19:53.014815 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.027138 (+ 12323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32155,"raft.queue_time_us":61419,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8713,"spinlock_wait_cycles":3372800,"thread_start_us":302760,"threads_started":1,"wal-append.queue_time_us":542321}
I20260709 12:19:52.768916  6062 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:52.069414 31969 rpcz_store.cc:276] 0709 12:19:50.700248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:50.701644 (+  1396us) service_pool.cc:224] Handling call
0709 12:19:50.702036 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:50.702481 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:50.702533 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:50.717365 (+ 14832us) log.cc:844] Serialized 52 byte log entry
0709 12:19:50.984721 (+267356us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:51.722828 (+738107us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.012312 (+289484us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.017767 (+  5455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.017842 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:52.017883 (+    41us) raft_consensus.cc:1685] finished
0709 12:19:52.017894 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:52.018009 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22638,"prepare.queue_time_us":157456,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":44541,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":8870,"spinlock_wait_cycles":93430144,"thread_start_us":1216799,"threads_started":3,"wal-append.queue_time_us":407082}
W20260709 12:19:53.022518  6126 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:19:53.023028  6061 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:52.763674  6142 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.064705 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.065455 32039 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:52.788350  6115 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:52.795114  6101 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.060039  5760 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.053791  6125 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:53.100234  6137 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
W20260709 12:19:53.135443  6143 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:53.135916  6134 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:53.138190  6154 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:19:53.100507  6083 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.138743  6123 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:53.139866  5913 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.141201  6142 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.143209 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2929) took 1640 ms. Trace:
I20260709 12:19:53.173722 32115 rpcz_store.cc:276] 0709 12:19:51.502520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.515812 (+ 13292us) service_pool.cc:224] Handling call
0709 12:19:51.516094 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.535090 (+ 18996us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:51.535137 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:51.589568 (+ 54431us) log.cc:844] Serialized 26 byte log entry
0709 12:19:51.931019 (+341451us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:51.931064 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.931143 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.142989 (+1211846us) raft_consensus.cc:1685] finished
0709 12:19:53.143010 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.143147 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18508,"raft.queue_time_us":24770,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":16965,"thread_start_us":307274,"threads_started":1,"wal-append.queue_time_us":527133}
I20260709 12:19:53.070406  6091 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:53.126998 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54134.9us
W20260709 12:19:53.138906  6131 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.558s	user 0.001s	sys 0.000s
I20260709 12:19:53.070823  6152 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.166464 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:53.166754 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:53.141242  6157 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:19:53.169401 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:53.141407  6024 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:53.141786  6126 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:53.191696 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:53.165187  5441 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.254192  6044 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:53.255017  6044 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.175949 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:53.125742  5907 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.124877  6061 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.257105  6061 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.259658 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:53.144305  6089 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.187084 32659 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:53.168545  6125 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.188997 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:53.260847  6089 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.260488  6044 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.262562  6089 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.261396  6125 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:53.268916 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.011s	sys 0.019s
W20260709 12:19:53.271863 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.012s	sys 0.019s
W20260709 12:19:53.272260 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 583950us
I20260709 12:19:53.144814  5760 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:53.304150  6132 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.692s	user 0.001s	sys 0.000s
I20260709 12:19:53.241225  6142 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.302794  6155 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.267091  6089 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.323292 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2942) took 1034 ms. Trace:
I20260709 12:19:53.320039 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:53.332346 32116 rpcz_store.cc:276] 0709 12:19:52.288862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.289024 (+   162us) service_pool.cc:224] Handling call
0709 12:19:52.289480 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.298200 (+  8720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:52.298232 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.298745 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:19:52.481530 (+182785us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:52.481594 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.481716 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.323057 (+841341us) raft_consensus.cc:1685] finished
0709 12:19:53.323080 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.323231 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9271,"spinlock_wait_cycles":8784384,"thread_start_us":114716,"threads_started":1,"wal-append.queue_time_us":489847}
I20260709 12:19:53.240396  6133 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:53.241925  6112 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.105893  6099 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:53.337757 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.025s	sys 0.008s
W20260709 12:19:53.343647 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.025s	sys 0.009s
W20260709 12:19:53.344518 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 565654us
I20260709 12:19:53.346077 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:53.329751 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56597.6us
W20260709 12:19:53.317546  6157 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:19:53.332737 32654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 1082) took 1354 ms. Trace:
I20260709 12:19:53.187880  6091 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.260532  6061 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.306203  6139 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.360131 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:53.350987  6045 consensus_meta.cc:287] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.787s	user 0.009s	sys 0.002s
I20260709 12:19:53.361047 32653 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:53.378404  6062 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.383615 32117 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:53.351754 32654 rpcz_store.cc:276] 0709 12:19:51.978637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.978830 (+   193us) service_pool.cc:224] Handling call
0709 12:19:51.979179 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:51.981305 (+  2126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.981345 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:52.013287 (+ 31942us) log.cc:844] Serialized 52 byte log entry
0709 12:19:52.397746 (+384459us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:52.397820 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.397919 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.332499 (+934580us) raft_consensus.cc:1685] finished
0709 12:19:53.332523 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.332677 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5643,"raft.queue_time_us":2078,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":18354,"thread_start_us":308006,"threads_started":1,"wal-append.queue_time_us":386557}
W20260709 12:19:53.383653 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 63726.5us
W20260709 12:19:53.386435 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 210631us
I20260709 12:19:53.274951  5441 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.387365 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:53.354081  6046 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.009s	sys 0.010s
I20260709 12:19:53.396021 32659 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
W20260709 12:19:53.397507 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51519.5us
I20260709 12:19:53.352968  5989 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.407970 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:53.347965  6119 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:19:53.331897  6142 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.424553 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:53.425772 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:53.427201 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:53.379413  5823 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.430720 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:53.402779  6046 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.010s	sys 0.010s
I20260709 12:19:53.386086  5848 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.436038 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.437068 32040 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:53.447834 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.437472 32041 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:53.437572 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.451229 32042 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:53.438344 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.439761 32038 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:53.395381  5913 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:53.464224 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.466509 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1173) took 1874 ms. Trace:
I20260709 12:19:53.469440 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.423349  6139 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.470136 32041 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:19:53.469836 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.474030 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.429404  6061 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:53.388159  6045 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.432031  6123 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:53.469735 32651 rpcz_store.cc:276] 0709 12:19:51.592596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.592789 (+   193us) service_pool.cc:224] Handling call
0709 12:19:51.593199 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.869520 (+276321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.869553 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:51.875623 (+  6070us) log.cc:844] Serialized 26 byte log entry
0709 12:19:52.245710 (+370087us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:19:52.245944 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.246014 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.466206 (+1220192us) raft_consensus.cc:1685] finished
0709 12:19:53.466228 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.466425 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2815,"raft.queue_time_us":177980,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":563375,"threads_started":2,"wal-append.queue_time_us":442392}
I20260709 12:19:53.424098  5870 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
W20260709 12:19:53.510468  6157 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:19:53.473389  6066 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.422487  5989 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.497782  6139 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:53.540938 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46116.4us
I20260709 12:19:53.518081  6123 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.468451  5848 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.544524 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:53.545486 31969 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:53.538875  6111 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.399s	user 0.015s	sys 0.016s
I20260709 12:19:53.525589  6101 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.526876  6115 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:53.551357 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 147629us
W20260709 12:19:53.542616  6144 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
W20260709 12:19:53.555861 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 123713us
W20260709 12:19:53.539662 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 181936us
I20260709 12:19:53.560328 31970 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:53.347662  6155 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.569186 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2842) took 1008 ms. Trace:
I20260709 12:19:53.565861 32039 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:53.182415  6083 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.168598  6134 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.572981 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.532037  6061 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:19:53.547710  6123 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:53.334518  6133 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.195672  6152 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.569618 31971 rpcz_store.cc:276] 0709 12:19:52.560345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.563054 (+  2709us) service_pool.cc:224] Handling call
0709 12:19:52.563581 (+   527us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:52.563957 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:52.564249 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.750106 (+185857us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:52.763506 (+ 13400us) log.cc:844] Serialized 52 byte log entry
0709 12:19:52.763683 (+   177us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:52.763924 (+   241us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.778697 (+ 14773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.778764 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.568904 (+790140us) raft_consensus.cc:1685] finished
0709 12:19:53.568926 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.569068 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50540,"prepare.queue_time_us":8620,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":8402,"raft.queue_time_us":322808,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":137422,"thread_start_us":185552,"threads_started":1,"wal-append.queue_time_us":456975}
I20260709 12:19:53.335862  6112 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.539623  6062 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:19:53.536667  6175 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:19:53.579453 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:53.556450  6145 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
W20260709 12:19:53.571468  6064 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.013s	user 0.013s	sys 0.000s
I20260709 12:19:53.542219  6066 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.581372 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:53.602599 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:53.603210 31969 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:53.618038 32047 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:53.592787  6101 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:53.623859 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2599) took 2139 ms. Trace:
I20260709 12:19:53.577723  6163 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.575876  5913 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 12:19:53.581682  6168 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:19:53.631974 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:53.633854 32651 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:53.624351 31972 rpcz_store.cc:276] 0709 12:19:51.476853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.499978 (+ 23125us) service_pool.cc:224] Handling call
0709 12:19:51.511715 (+ 11737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.512124 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.512163 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:51.512722 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:19:51.861400 (+348678us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:51.861459 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.861556 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.191785 (+1330229us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:53.598877 (+407092us) raft_consensus.cc:1685] finished
0709 12:19:53.598898 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.615986 (+ 17088us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":868,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":14566,"spinlock_wait_cycles":9433472,"thread_start_us":341800,"threads_started":1,"wal-append.queue_time_us":384129}
I20260709 12:19:53.599892  6177 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.577482  6045 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.605010  6165 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:53.592901  6167 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:53.618985  6091 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.627213  6140 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:19:53.623673  5870 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:53.619083  5823 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.600678  6064 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.622632  6158 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:53.652756 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:53.628561  5760 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.619597  6141 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:53.696760  6141 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:53.697788  6141 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:53.699366  6176 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 12:19:53.701234  6171 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:19:53.702986 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2846) took 1141 ms. Trace:
I20260709 12:19:53.643375  6119 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:53.686861  6159 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:19:53.694479  6183 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:53.695647  6185 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:53.686353  6173 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.705541 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2628) took 1780 ms. Trace:
I20260709 12:19:53.736660 32040 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:19:53.737648 32651 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:53.646335  6167 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.656625  6181 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:53.710646  6184 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:53.710927  6187 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.711357  6158 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:53.716125  5760 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:53.654953  5870 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:53.718289  5949 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.720151  6033 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.721618  6058 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:53.723141  6151 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.584s	user 0.000s	sys 0.006s
I20260709 12:19:53.725785 32653 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:53.747879  6167 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:53.730818  6134 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.735152 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.703488 31965 rpcz_store.cc:276] 0709 12:19:52.561439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.589846 (+ 28407us) service_pool.cc:224] Handling call
0709 12:19:52.590252 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:53.055371 (+465119us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:53.068582 (+ 13211us) log.cc:844] Serialized 14 byte log entry
0709 12:19:53.194650 (+126068us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:53.268051 (+ 73401us) log.cc:844] Serialized 52 byte log entry
0709 12:19:53.268223 (+   172us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:53.268257 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.268328 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.702709 (+434381us) raft_consensus.cc:1685] finished
0709 12:19:53.702736 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.702915 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192532,"prepare.queue_time_us":13000,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":227124,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":49745,"thread_start_us":393640,"threads_started":2,"wal-append.queue_time_us":306596}
I20260709 12:19:53.735807 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:53.705850 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 98175.5us
I20260709 12:19:53.736424 32043 rpcz_store.cc:276] 0709 12:19:51.925185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.928314 (+  3129us) service_pool.cc:224] Handling call
0709 12:19:51.928706 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.983084 (+ 54378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.983113 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.013571 (+ 30458us) log.cc:844] Serialized 26 byte log entry
0709 12:19:52.375307 (+361736us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:52.375363 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.375457 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.705337 (+1329880us) raft_consensus.cc:1685] finished
0709 12:19:53.705362 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.705484 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93609,"raft.queue_time_us":73665,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":10810,"thread_start_us":264481,"threads_started":1,"wal-append.queue_time_us":653410}
I20260709 12:19:53.740098 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.830485 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:53.742204  6156 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.831755  6156 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.832285 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:53.735659  6119 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.745937  6066 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.746510  6152 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.746836 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:53.707705 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55291.6us
I20260709 12:19:53.752385 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:53.840885 31970 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:53.841078 32040 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
W20260709 12:19:53.734730  6170 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
W20260709 12:19:53.778149  5527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.379s	user 0.010s	sys 0.038s
I20260709 12:19:53.781857  5869 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:53.795048 31969 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
W20260709 12:19:53.799450  6182 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:19:53.801206 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:53.812434  5913 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.813578  6205 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:53.747442  6083 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:53.729333  6090 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.425s	user 0.007s	sys 0.003s
I20260709 12:19:53.750134  5848 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.667646  5823 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:53.801060  6159 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:53.642128  6163 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.647980  6177 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.783120  6062 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:53.799779  5870 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:53.902189 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:53.799798  6091 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:53.835963  6058 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:53.870039 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68914.8us
I20260709 12:19:53.908468 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:53.819388  5989 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.790468  6187 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.872253 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:53.926471 32039 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:19:53.863701  5527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.467s	user 0.011s	sys 0.038s
I20260709 12:19:53.879781 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:53.844945  6033 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:53.748878  6173 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.843443  6166 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:53.863246  6066 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:53.862766  6175 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:53.896875 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:53.748720  6185 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.816579  5949 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.899185 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 838) took 1981 ms. Trace:
I20260709 12:19:53.871587  6107 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.872674  6115 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:53.938228  5989 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:53.770772  6181 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.877767  6119 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:53.930960 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 111760us
W20260709 12:19:53.936954 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64773.8us
I20260709 12:19:53.896072  6155 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.877496  5913 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.948202 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2840) took 1388 ms. Trace:
I20260709 12:19:53.879877  6090 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.960304 31970 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:53.963572 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:53.909979  6112 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.974720 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:53.990196 31963 rpcz_store.cc:276] 0709 12:19:52.559672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.566428 (+  6756us) service_pool.cc:224] Handling call
0709 12:19:52.566712 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.772356 (+205644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:52.772394 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:53.011785 (+239391us) log.cc:844] Serialized 26 byte log entry
0709 12:19:53.156102 (+144317us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:53.156161 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.156261 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.947958 (+791697us) raft_consensus.cc:1685] finished
0709 12:19:53.947983 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.948133 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54769,"raft.queue_time_us":22852,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":82433,"thread_start_us":302472,"threads_started":2,"wal-append.queue_time_us":324918}
I20260709 12:19:53.996435 31969 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:53.904680  6083 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.955540 31964 rpcz_store.cc:276] 0709 12:19:51.917481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.921990 (+  4509us) service_pool.cc:224] Handling call
0709 12:19:51.928273 (+  6283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.946709 (+ 18436us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:19:51.985344 (+ 38635us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.409760 (+424416us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:53.728887 (+1319127us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:53.729355 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:19:53.781616 (+ 52261us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:53.781654 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.782267 (+   613us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.898879 (+116612us) raft_consensus.cc:1685] finished
0709 12:19:53.898978 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.899112 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":978,"prepare.queue_time_us":39063,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1951,"raft.queue_time_us":92476,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":18126,"spinlock_wait_cycles":64364800,"thread_start_us":437485,"threads_started":3,"wal-append.queue_time_us":287316,"wal-append.run_cpu_time_us":1200,"wal-append.run_wall_time_us":1001223}
W20260709 12:19:53.983472 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 75072.6us
I20260709 12:19:53.936662  6133 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:53.906246  6062 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.880033  6205 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.911602  6123 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.947213  6139 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:53.976250  6195 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:53.934302  6091 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:53.999496 32623 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:54.000252 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.013176 31963 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:19:54.009438 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:53.985564  6064 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.016897  6090 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.995009  6174 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.987813  6186 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:53.968861  6066 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.029429 31972 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:54.026923 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:53.945760  6166 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.999133  6156 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:53.909762  6158 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.001142  6112 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:54.033514 31969 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:53.958906  6175 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:53.977236  6107 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.038357 32119 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:53.998474  5808 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:19:54.019865  6155 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.023017  5989 raft_consensus.cc:2764] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:19:54.043998  5913 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.007934  6083 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.047730 32115 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:54.077335 31969 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:54.025774  6195 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.051424  6156 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:54.093973  6195 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.047170  6112 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.081099  6064 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:19:54.120590  6107 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.033543  6186 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.088025  6196 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.186s
I20260709 12:19:54.094758  5848 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:54.048485  6167 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.097388  6191 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:54.033367  6174 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.083073  6155 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.110102  6156 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:54.138653  6083 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.138299  5870 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:54.140987  6212 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:19:54.145145  5844 consensus_meta.cc:287] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.565s	user 0.015s	sys 0.000s
I20260709 12:19:54.160501 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.161576  6162 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:54.164883  6155 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.147094  6191 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.207825  6191 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:54.212603  6166 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.148841  5848 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.217271  6112 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.219918  5848 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.220149  6181 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.220317 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.221264 32653 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:54.223795  5870 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:19:54.148502  6177 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.168377 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.154035  6156 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:54.174366  6109 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.940s	user 0.014s	sys 0.018s
I20260709 12:19:54.318691 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:54.319348 32047 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:54.320201 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:54.320724 32038 raft_consensus.cc:2376] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22. Re-sending same reply.
I20260709 12:19:54.181401  6123 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.184027 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 839) took 2236 ms. Trace:
I20260709 12:19:54.143421  6185 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.149328  5949 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.205627 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 1084) took 1623 ms. Trace:
I20260709 12:19:54.340875 32657 rpcz_store.cc:276] 0709 12:19:52.582752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.584902 (+  2150us) service_pool.cc:224] Handling call
0709 12:19:52.697611 (+112709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:53.150001 (+452390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.150051 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:53.150115 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:53.150164 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.205264 (+1055100us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400154d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:54.205339 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.205556 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1039712000}
I20260709 12:19:54.166635  6062 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.230232 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2844) took 1669 ms. Trace:
I20260709 12:19:54.271888 32651 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:54.342726  6125 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.170128  6163 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.347849  6119 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.348572  6116 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.348656  6119 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.351118  6119 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.352304  6215 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:54.353330  6215 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:54.353722  6215 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:54.354768  6215 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.356359  6215 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:54.351105  6175 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.349351  6116 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.190788  5844 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.329663 31966 rpcz_store.cc:276] 0709 12:19:51.947250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.979206 (+ 31956us) service_pool.cc:224] Handling call
0709 12:19:52.011745 (+ 32539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.021824 (+ 10079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:52.021871 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.205123 (+183252us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b18002ce480. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a
0709 12:19:53.925756 (+1720633us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:53.936200 (+ 10444us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.030521 (+ 94321us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:19:54.030556 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.030644 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:54.183794 (+153150us) raft_consensus.cc:1685] finished
0709 12:19:54.183815 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.183964 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1770,"raft.queue_time_us":2036,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4679,"spinlock_wait_cycles":106395136,"thread_start_us":50924,"threads_started":1,"wal-append.queue_time_us":171479}
I20260709 12:19:54.180371  6173 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.428606 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.429534 32651 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 15.
I20260709 12:19:54.432237 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:54.433127 32040 raft_consensus.cc:2376] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22. Re-sending same reply.
I20260709 12:19:54.342301 31968 rpcz_store.cc:276] 0709 12:19:52.560905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.584650 (+ 23745us) service_pool.cc:224] Handling call
0709 12:19:52.584942 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:53.042201 (+457259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.042239 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:53.042286 (+    47us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:19:53.042359 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.230006 (+1187647us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54000f6254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:54.230048 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.230177 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865622912}
I20260709 12:19:54.342171  5961 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.441527  6116 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.447374  6162 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:54.447948  6162 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:54.454001  6164 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.454646  6164 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.328313  6196 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.459381  6196 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.460054 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.460630  6164 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.326361  5808 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.466897 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.467864  6196 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.327641  6177 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:54.328929  6064 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:54.474305 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:54.477962 32047 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:54.488399 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:54.462320  6091 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.496562 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:54.498121 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:54.500236 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.500910 31963 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:54.500142 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.501968 31966 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:54.502393  6186 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.504402  6215 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:54.504500 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.505792 32040 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 17: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:54.477373  6066 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.509591  5844 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.513629  6031 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.522078  6175 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:54.522725  6175 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.523252 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:54.481797  6205 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.481040  6177 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.515239  6167 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.516937  6164 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.517979 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.506023  6215 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.529022 31966 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:54.532349 31968 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:54.479211  5808 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.531932 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.536633 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.538058 31968 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:54.486124  6064 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.528796  6184 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.550987 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:54.561370  6033 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.622s	user 0.007s	sys 0.001s
I20260709 12:19:54.533977  6215 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:54.582278  6210 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.582543  5808 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.583014  6210 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.585701  6221 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:54.535095  6164 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.586373  6221 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.599273  6187 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.602564  6177 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.573262  6175 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.571731  6184 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.624636 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.627315 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.625404 32121 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 13.
I20260709 12:19:54.630053  6184 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.630743 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2610) took 2552 ms. Trace:
I20260709 12:19:54.631161 31967 rpcz_store.cc:276] 0709 12:19:52.078734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.078918 (+   184us) service_pool.cc:224] Handling call
0709 12:19:52.079350 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.144030 (+ 64680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:52.144064 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.144577 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:19:53.730572 (+1585995us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b18002bbc40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f0d1 00007f228c18aa0f
0709 12:19:53.773678 (+ 43106us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:53.773718 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.630388 (+856670us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b5400315954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:54.630449 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:54.630512 (+    63us) raft_consensus.cc:1685] finished
0709 12:19:54.630530 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.630681 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43649,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9659,"spinlock_wait_cycles":227547136,"thread_start_us":42905,"threads_started":1,"wal-append.queue_time_us":1605574}
I20260709 12:19:54.633157  6058 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.622267  6022 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.635251  6022 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.637840  6022 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.640935 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.643188 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.655011 31970 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:19:54.656502 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.657073 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.657289 32113 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:54.658500 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:54.657378 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 126492us
I20260709 12:19:54.659747 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.660391 31966 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:19:54.661963 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.667224 32112 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:54.660481 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:54.696023 31967 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:19:54.673640 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.699715 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.705055 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:54.705302 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.707959 32116 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:54.707845 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:54.715621 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.716104 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:54.662496  6167 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:54.720854 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60433.8us
I20260709 12:19:54.647210  5870 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:19:54.725131 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:54.713249  6101 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:54.728390 32121 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:19:54.741000 31972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.018s
W20260709 12:19:54.741556 31972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.002s	sys 0.018s
I20260709 12:19:54.717433  6163 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:54.767122  5823 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.738651  6101 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:54.752920  6179 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:54.782014  6042 raft_consensus.cc:993] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:19:54.782624  6042 raft_consensus.cc:1081] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:19:54.777105 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:54.756763  6163 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:54.787211 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:54.788656  6177 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:54.788864  5870 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:54.789124  6177 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.789592  6177 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:54.789522  5870 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.793540  6142 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.728259  6167 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.794843  6163 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:54.784272  6101 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:54.800319  6024 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:54.800755  6024 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.801156  6024 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:54.801425  6161 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.453s	user 0.001s	sys 0.005s
I20260709 12:19:54.803041 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2936) took 2626 ms. Trace:
I20260709 12:19:54.748090  5989 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:19:54.803402 32114 rpcz_store.cc:276] 0709 12:19:52.176466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.181661 (+  5195us) service_pool.cc:224] Handling call
0709 12:19:52.211646 (+ 29985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.688547 (+476901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:52.688581 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.697048 (+  8467us) log.cc:844] Serialized 26 byte log entry
0709 12:19:53.031207 (+334159us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:53.031265 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.451437 (+1420172us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540043b2d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:54.451495 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:54.800721 (+349226us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:54.802758 (+  2037us) raft_consensus.cc:1685] finished
0709 12:19:54.802784 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.802939 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":393532,"raft.queue_time_us":178613,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":30662,"spinlock_wait_cycles":448875648,"thread_start_us":304413,"threads_started":1,"wal-append.queue_time_us":579053}
I20260709 12:19:54.774858  6064 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.767146  6221 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:54.662127  6191 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.784657  6179 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.815596  5869 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:54.816090  5869 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:54.796023  5870 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.786026  6156 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.812366  6179 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:54.820034  6164 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:54.832955 31964 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.007s	sys 0.018s
W20260709 12:19:54.833330 31964 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.007s	sys 0.018s
I20260709 12:19:54.818877  6162 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:54.824368  6158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.015s	sys 0.010s
I20260709 12:19:54.840459 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:54.840149  6178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.020s	sys 0.066s
W20260709 12:19:54.849361  6178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.020s	sys 0.066s
I20260709 12:19:54.849190  6175 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:54.856097  6175 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:54.856633  6175 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:54.863323  6142 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:54.864079  6142 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.867643  6142 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.868407 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:54.869182 32653 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
W20260709 12:19:54.847874  6158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.015s	sys 0.010s
I20260709 12:19:54.852226  6177 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.875440  6177 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.879369  6156 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:54.880997  6167 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.881394  6163 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.882207  6191 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:54.896199 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.847280  6164 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 12:19:54.898370 31969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.003s	sys 0.020s
I20260709 12:19:54.902295 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:54.913141 31969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.004s	sys 0.020s
I20260709 12:19:54.922335 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.934007 32045 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:54.940832 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:54.941759 32046 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
W20260709 12:19:54.902419  5760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.002s	sys 0.021s
I20260709 12:19:54.951047 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:54.952770 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:54.953372 31970 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:54.953495 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:54.954099 31966 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
W20260709 12:19:54.910876  6209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.009s	sys 0.022s
I20260709 12:19:54.872001  6175 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.965031 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:54.971554 31970 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:19:54.972335 32040 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:54.888707  6024 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:54.973359 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:54.982790 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.013s	sys 0.013s
W20260709 12:19:54.985057 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.013s	sys 0.013s
W20260709 12:19:54.985395 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 934514us
I20260709 12:19:54.986052 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:54.991835 32040 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
W20260709 12:19:54.952418  5760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.002s	sys 0.021s
W20260709 12:19:54.973456  6190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.013s	sys 0.029s
W20260709 12:19:54.957963  6120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.011s	sys 0.014s
W20260709 12:19:54.972034  6209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.009s	sys 0.023s
W20260709 12:19:55.011327  6120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.012s	sys 0.014s
W20260709 12:19:55.015002 32087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.015s	sys 0.031s
W20260709 12:19:55.015444 32087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.015s	sys 0.032s
I20260709 12:19:55.016361  6178 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:55.016919  6178 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.019420  6178 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:55.007716  6190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.013s	sys 0.033s
I20260709 12:19:55.021055  6209 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:55.021566  6209 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:54.911573  6215 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:55.032121 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46125us
I20260709 12:19:55.032969 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:55.033665 32117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2951) took 1860 ms (client timeout 1727 ms). Trace:
W20260709 12:19:55.034222 32117 rpcz_store.cc:269] 0709 12:19:53.165956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.166528 (+   572us) service_pool.cc:224] Handling call
0709 12:19:55.025827 (+1859299us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400433fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:55.025963 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456220032}
I20260709 12:19:55.036978 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.037647 32114 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:55.037664 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.038980 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:55.039234 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:55.039557 32121 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:55.039767 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.040525 32114 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:55.041252 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:55.037271 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:55.080286 32117 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:55.083441 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.034960  6169 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.033787  6031 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:55.031476  6066 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:55.038730  6138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.017s	sys 0.011s
I20260709 12:19:55.042459  6190 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:19:55.088297 31963 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.029s	sys 0.020s
I20260709 12:19:55.076634  6216 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:55.081538  6167 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:55.095446 31963 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.029s	sys 0.021s
I20260709 12:19:55.085655  6163 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:55.175484 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2958) took 1273 ms. Trace:
I20260709 12:19:55.082449 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.188802 32117 raft_consensus.cc:2468] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:55.080880  6178 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:55.187830 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2639) took 1467 ms. Trace:
W20260709 12:19:55.091601  6138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.018s	sys 0.011s
I20260709 12:19:55.188400 32115 rpcz_store.cc:276] 0709 12:19:53.901829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.901951 (+   122us) service_pool.cc:224] Handling call
0709 12:19:55.175187 (+1273236us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001234d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:55.175403 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332994944}
I20260709 12:19:55.280030 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.190425 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:19:55.262676  6196 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.027s	sys 0.030s
I20260709 12:19:55.199703  6024 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:55.078655  6158 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:19:55.202394  6101 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.292407 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:55.089824  6169 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:55.208091  6179 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.181565  6216 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:55.214979  6209 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:55.159833  6167 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.105044  6162 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:55.219847  6226 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:19:55.248570 32043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.014s	sys 0.019s
I20260709 12:19:55.252756 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.253206 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:55.252022 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:55.220029  6215 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:55.267586 32039 rpcz_store.cc:276] 0709 12:19:53.720156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.723348 (+  3192us) service_pool.cc:224] Handling call
0709 12:19:53.723703 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:53.926917 (+203214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.176333 (+1249416us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001a4fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:55.187767 (+ 11434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465492864}
I20260709 12:19:55.268575 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:55.272879 32121 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:55.187757  6177 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:55.275384 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:55.092015  6066 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:55.274094 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.021s	sys 0.011s
I20260709 12:19:55.172516  5870 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:55.282099 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:55.271974  6115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.013s	sys 0.035s
I20260709 12:19:55.267503  6178 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.305791 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:55.287878  6196 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.027s	sys 0.031s
I20260709 12:19:55.310129 32115 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:55.249279  6190 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.839974 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2869) took 1322 ms. Trace:
W20260709 12:19:55.845701 32043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.368s	user 0.022s	sys 0.038s
W20260709 12:19:55.579612 31938 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.002s	sys 0.027s
W20260709 12:19:55.848716 31938 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.002s	sys 0.036s
W20260709 12:19:55.448799 31935 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.605s (SENT) [suppressed 10 similar messages]
W20260709 12:19:55.846125 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.027s	sys 0.021s
I20260709 12:19:55.798494 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 875) took 1326 ms. Trace:
W20260709 12:19:55.698167 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.022s	sys 0.033s
W20260709 12:19:55.850257 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.12626e+06us
W20260709 12:19:55.279277  6193 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.558s	user 0.003s	sys 0.036s
W20260709 12:19:55.316593 32047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.018s	sys 0.046s
W20260709 12:19:55.315621 32112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.011s	sys 0.023s
I20260709 12:19:55.384485 32041 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:55.471009 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.847761 32657 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 12:19:55.850052 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 401265us
W20260709 12:19:55.861945 32112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.020s	sys 0.040s
I20260709 12:19:55.858459 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:55.852860 32652 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:55.840601  6239 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.591s	user 0.002s	sys 0.000s
W20260709 12:19:55.861701 32047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.031s	sys 0.066s
W20260709 12:19:55.762849  5869 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.946s	user 0.011s	sys 0.007s
W20260709 12:19:55.658061  6231 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:19:55.517563  6158 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.867216 32116 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:55.852699  6178 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:55.847240 32117 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:55.848958 32651 raft_consensus.cc:2410] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 12:19:55.865653 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.024s	sys 0.035s
I20260709 12:19:55.872454  6248 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:55.849027 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 1.00787e+06us
I20260709 12:19:55.850670 32038 rpcz_store.cc:276] 0709 12:19:54.471768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.471915 (+   147us) service_pool.cc:224] Handling call
0709 12:19:54.472292 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.480338 (+  8046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.480375 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.486838 (+  6463us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.799427 (+312589us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b48000fe358. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:54.899651 (+100224us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:19:54.899692 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.899756 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.781949 (+882193us) raft_consensus.cc:1685] finished
0709 12:19:55.781982 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.798379 (+ 16397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5503,"raft.queue_time_us":2753,"raft.run_cpu_time_us":2279,"raft.run_wall_time_us":321812,"spinlock_wait_cycles":9768832,"thread_start_us":394555,"threads_started":1,"wal-append.queue_time_us":785990}
W20260709 12:19:55.424186  6119 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.030s
W20260709 12:19:55.369226  6058 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.022s
W20260709 12:19:55.852809  6193 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.133s	user 0.015s	sys 0.056s
W20260709 12:19:55.326450  6225 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:19:55.881807 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 872) took 1563 ms. Trace:
I20260709 12:19:55.896075 32043 rpcz_store.cc:276] 0709 12:19:54.318247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.321382 (+  3135us) service_pool.cc:224] Handling call
0709 12:19:54.321631 (+   249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.332654 (+ 11023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.332701 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.333282 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.848801 (+1515519us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:55.881016 (+ 32215us) log.cc:844] Serialized 14 byte log entry
0709 12:19:55.881289 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.881432 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.881564 (+   132us) raft_consensus.cc:1685] finished
0709 12:19:55.881579 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.881735 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10601,"prepare.queue_time_us":34910,"raft.queue_time_us":16085,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":60716,"thread_start_us":1370636,"threads_started":1,"wal-append.queue_time_us":1289968}
I20260709 12:19:55.605656  6101 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:55.845438  6115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.023s	sys 0.047s
W20260709 12:19:55.606572  6235 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:19:55.847210  6177 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.908980  6177 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:55.909471 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2950) took 2755 ms. Trace:
I20260709 12:19:55.909858 32118 rpcz_store.cc:276] 0709 12:19:53.154058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.154737 (+   679us) service_pool.cc:224] Handling call
0709 12:19:53.154985 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:53.602612 (+447627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.602654 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:53.602695 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:53.602725 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.909239 (+2306514us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54003e4ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:55.909288 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.909419 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028993408}
I20260709 12:19:55.911321 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.912402 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.913386 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.913911 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:55.652158  6179 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:55.849179 31968 rpcz_store.cc:276] 0709 12:19:54.517209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.549801 (+ 32592us) service_pool.cc:224] Handling call
0709 12:19:54.550734 (+   933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.566012 (+ 15278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.566061 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.580021 (+ 13960us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.089766 (+509745us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:55.089811 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.089901 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.660661 (+570760us) raft_consensus.cc:1685] finished
0709 12:19:55.660687 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.839913 (+179226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9687,"thread_start_us":477478,"threads_started":1,"wal-append.queue_time_us":1047436}
I20260709 12:19:55.881474  6112 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:55.885488 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:55.887166 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.17865e+06us
I20260709 12:19:55.887612 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.915567 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:55.915921 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.852293  6167 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:55.650169  6169 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:55.907429  6191 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:19:55.895454  6058 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.024s	sys 0.024s
W20260709 12:19:55.892798 32013 connection.cc:256] RPC call timeout handler was delayed by -1.1444806879999305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 12:19:55.892289  6119 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.017s	sys 0.041s
I20260709 12:19:55.926007 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.940908 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.861052  5870 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.916671 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.931754 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.943089 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.945214 32121 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:55.936859  5441 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:55.942998 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2620) took 2224 ms. Trace:
I20260709 12:19:55.932375 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:55.944097 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:55.946192 31972 rpcz_store.cc:276] 0709 12:19:53.718367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.052369 (+334002us) service_pool.cc:224] Handling call
0709 12:19:54.090867 (+ 38498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.091375 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.091418 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.112118 (+ 20700us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.741991 (+629873us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:54.742049 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.742134 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.942821 (+1200687us) raft_consensus.cc:1685] finished
0709 12:19:55.942841 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.942953 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":306,"raft.queue_time_us":3432,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":12856,"thread_start_us":595537,"threads_started":1,"wal-append.queue_time_us":873404}
I20260709 12:19:55.948876 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2617) took 2231 ms. Trace:
I20260709 12:19:55.884078  5869 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.928541  6215 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:55.951496  5949 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.951843  6215 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:55.952332  6215 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:55.952122  5949 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:55.949361 31964 rpcz_store.cc:276] 0709 12:19:53.717474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.042772 (+325298us) service_pool.cc:224] Handling call
0709 12:19:54.079286 (+ 36514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.081916 (+  2630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.081952 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.116102 (+ 34150us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.833733 (+717631us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:54.833775 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.833859 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.948644 (+1114785us) raft_consensus.cc:1685] finished
0709 12:19:55.948673 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.948815 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1865,"raft.queue_time_us":4872,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":8789,"thread_start_us":690043,"threads_started":1,"wal-append.queue_time_us":926660}
I20260709 12:19:55.961089 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.961815 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:55.964305 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.966044 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.967387 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.969900 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.886857  6158 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.970542 32118 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:55.970821 32653 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:19:55.966830 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:55.971268 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:55.972687 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.975023 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:55.975803 31971 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:55.919373  6162 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:55.993094 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.927990  6191 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:55.997761 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:55.888576  6248 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:55.999253 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:55.993829 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:56.002457 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.989964 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:55.988083 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:55.991873 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.003713 32118 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:56.004591 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.004459 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.005447 32118 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:55.973402 32117 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:56.001196 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:56.002547 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:55.987162 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:55.995648 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:55.956130  5949 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.006671 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:56.004515 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:56.012320 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:56.013986 31965 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:56.014000 31970 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:55.970007  6115 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:56.019454 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:55.928184  6112 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.018093 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.017203 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.017933 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:56.019707 32013 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.020613 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.027760 32113 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:56.016785 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:56.023610 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.017s	sys 0.012s
W20260709 12:19:56.028407 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.017s	sys 0.012s
I20260709 12:19:56.023432  5949 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:56.010891  5723 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:19:55.996970  5441 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:56.000535  6153 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:56.027943 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.038762 31964 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
W20260709 12:19:56.023883 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.011s	sys 0.018s
W20260709 12:19:56.029564 32011 connection.cc:256] RPC call timeout handler was delayed by -1.3542278810000425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 12:19:56.041555 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.016011  6149 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:55.978300  6245 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:56.031159 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:56.048324 32117 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:56.028467 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.035687 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:56.039700 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.012s	sys 0.018s
I20260709 12:19:56.039695 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:55.934726  6216 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.042706 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.052410 32121 raft_consensus.cc:2410] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:19:56.053428 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:56.030637 31966 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.009s	sys 0.020s
W20260709 12:19:56.056787 31966 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.009s	sys 0.020s
W20260709 12:19:56.049930 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.01266e+06us
I20260709 12:19:56.057492 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.043643 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:56.052460 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.053043 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.057521 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:56.060185 31964 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:56.061899 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:56.063918 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:56.064544 32113 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:56.031172  6068 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:56.069077 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.030362  5949 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:56.070997 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.074501 31971 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:19:56.055233  5760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.024s	sys 0.028s
W20260709 12:19:56.075872  5760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.024s	sys 0.028s
I20260709 12:19:56.075934  6245 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:56.025965  6191 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.048625  5441 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.080341  6179 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.080463  6101 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:56.080883  6179 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.081699  6246 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:56.083283  6183 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:56.084005  6183 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:56.084431  6183 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:56.085224  6183 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.086412  5760 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.086901  5760 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.087587  6025 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:56.088326 32113 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:56.089012  5989 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:56.082114  5441 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.089433  5989 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:56.089864  5441 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:56.089859  5989 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:56.091122  6175 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.092036  6246 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:56.093779 32117 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:56.094861 32120 raft_consensus.cc:2410] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:19:56.098984 32113 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:56.100888 32113 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:56.101502  6115 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:56.102449 32113 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:56.104043 32113 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:56.104431  5989 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.106683 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2889) took 1015 ms. Trace:
I20260709 12:19:56.114935 32121 rpcz_store.cc:276] 0709 12:19:55.091896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.070262 (+978366us) service_pool.cc:224] Handling call
0709 12:19:56.106607 (+ 36345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.116261 32115 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:56.116746 32121 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:56.079834  6162 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:56.117059 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2891) took 1011 ms. Trace:
I20260709 12:19:56.080960  6101 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.049927  6149 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.121955 32115 rpcz_store.cc:276] 0709 12:19:55.105428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.073109 (+967681us) service_pool.cc:224] Handling call
0709 12:19:56.073474 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.116762 (+ 43288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.116995 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.124306 32121 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:56.128752 32119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.019s	sys 0.023s
W20260709 12:19:56.178907 32119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.020s	sys 0.024s
I20260709 12:19:56.183586 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:56.207510 31963 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.019s	sys 0.082s
W20260709 12:19:56.207908 31963 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.019s	sys 0.082s
I20260709 12:19:56.207993 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2959) took 1590 ms. Trace:
I20260709 12:19:56.106427  5949 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:56.215035 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2865) took 1699 ms. Trace:
I20260709 12:19:56.214915 32115 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:56.215582 31963 rpcz_store.cc:276] 0709 12:19:54.515949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.518987 (+  3038us) service_pool.cc:224] Handling call
0709 12:19:54.520750 (+  1763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.521170 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.521209 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.530176 (+  8967us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.160254 (+630078us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:56.208392 (+1048138us) log.cc:844] Serialized 14 byte log entry
0709 12:19:56.214638 (+  6246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.214741 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:56.214799 (+    58us) raft_consensus.cc:1685] finished
0709 12:19:56.214816 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.214969 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":548,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6507,"thread_start_us":1553057,"threads_started":2,"wal-append.queue_time_us":1209065}
W20260709 12:19:56.120522  6184 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.015s	sys 0.039s
W20260709 12:19:56.133487  6233 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.789s	user 0.005s	sys 0.013s
I20260709 12:19:56.260497 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.121851  6162 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.126698  6090 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:56.248023  6184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.016s	sys 0.039s
I20260709 12:19:56.208043  6149 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:56.286854 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.025s	sys 0.062s
W20260709 12:19:56.287362 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.025s	sys 0.062s
I20260709 12:19:56.214325  6153 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.274619  6210 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.282024  6162 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.304003 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.291064  6066 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.304788 31963 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
W20260709 12:19:56.310750 32012 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.233s	user 0.028s	sys 0.068s
W20260709 12:19:56.311159 32012 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.028s	sys 0.069s
I20260709 12:19:56.306010 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2887) took 1223 ms. Trace:
I20260709 12:19:56.311846 32116 rpcz_store.cc:276] 0709 12:19:55.082973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.083199 (+   226us) service_pool.cc:224] Handling call
0709 12:19:56.305797 (+1222598us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400434ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:56.305939 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800632576}
I20260709 12:19:56.316373 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2858) took 2319 ms. Trace:
I20260709 12:19:56.316866 31969 rpcz_store.cc:276] 0709 12:19:53.996648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.091997 (+ 95349us) service_pool.cc:224] Handling call
0709 12:19:54.111096 (+ 19099us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.111409 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.111428 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.111838 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.913710 (+801872us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:54.913771 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.913876 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:56.081894 (+1168018us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:19:56.308180 (+226286us) raft_consensus.cc:1685] finished
0709 12:19:56.308219 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.316285 (+  8066us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2669,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3318,"spinlock_wait_cycles":187120512,"thread_start_us":765921,"threads_started":1,"wal-append.queue_time_us":1139578}
I20260709 12:19:56.288228  6090 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.300370  6190 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.304271  6244 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:56.307737  5870 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.295094  6177 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.296250  6153 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.313408 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:56.314527  6210 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:19:56.414259  6196 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.416414  6244 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:56.417487  6158 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.437103 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.437286 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.446686 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.450037 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:56.436789 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 123473us
I20260709 12:19:56.454025  6244 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.438170  5907 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:56.458907  5760 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.461637  6179 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.444990  6196 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.464710  6090 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:56.465233 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.466606 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54754 (request call id 877) took 1984 ms. Trace:
I20260709 12:19:56.466960 32047 rpcz_store.cc:276] 0709 12:19:54.481738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.481939 (+   201us) service_pool.cc:224] Handling call
0709 12:19:54.482282 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.482773 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.482815 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.483385 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.879772 (+1396387us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:55.879831 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.879920 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:56.466413 (+586493us) raft_consensus.cc:1685] finished
0709 12:19:56.466433 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.466550 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1290,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":4579,"thread_start_us":1355333,"threads_started":1,"wal-append.queue_time_us":1773181}
I20260709 12:19:56.476311 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:56.451572 31936 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:56.478605 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.478820 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:56.480219 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:56.480953 32044 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.438344  5869 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:56.452044  6247 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:19:56.503502 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.504407 32047 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:56.212075 32119 rpcz_store.cc:276] 0709 12:19:54.618015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.618208 (+   193us) service_pool.cc:224] Handling call
0709 12:19:54.625668 (+  7460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:54.625917 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:55.086161 (+460244us) log.cc:844] Serialized 14 byte log entry
0709 12:19:56.207528 (+1121367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:56.207593 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:56.207638 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.207707 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.207932 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55227,"prepare.queue_time_us":121183,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":31166,"thread_start_us":1477845,"threads_started":2}
W20260709 12:19:56.521590 32085 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.152s	user 0.041s	sys 0.153s
W20260709 12:19:56.522015 32085 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.041s	sys 0.153s
W20260709 12:19:56.522372 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 1.18148e+06us
I20260709 12:19:56.464743  6196 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.483450  6248 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.444301  6257 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.551126  6257 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.552160 32039 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:56.553479  6257 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.555219 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2970) took 1852 ms. Trace:
I20260709 12:19:56.555692 32112 rpcz_store.cc:276] 0709 12:19:54.702774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.702901 (+   127us) service_pool.cc:224] Handling call
0709 12:19:54.706911 (+  4010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.707267 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.707297 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.715240 (+  7943us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.879761 (+1164521us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:55.879823 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.879916 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:56.554957 (+675041us) raft_consensus.cc:1685] finished
0709 12:19:56.554984 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.555149 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":666,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9511,"thread_start_us":1112554,"threads_started":1,"wal-append.queue_time_us":1522456}
I20260709 12:19:56.558260  5989 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.559396  6177 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:56.560173  6167 consensus_meta.cc:287] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.629s	user 0.008s	sys 0.003s
I20260709 12:19:56.560237  6066 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:56.573791 32653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 982) took 1291 ms. Trace:
I20260709 12:19:56.582490 32653 rpcz_store.cc:276] 0709 12:19:55.282290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.282662 (+   372us) service_pool.cc:224] Handling call
0709 12:19:56.573540 (+1290878us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:56.573682 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279191808}
I20260709 12:19:56.584718 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.595386 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.596294 31969 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:56.597463 32041 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:56.599510 32040 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:56.602098 32046 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:56.605885 32040 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 23. Current term is 24. Ops: [23.11-23.11]
W20260709 12:19:56.607165 32120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 2963) took 1966 ms (client timeout 1598 ms). Trace:
W20260709 12:19:56.607640 32120 rpcz_store.cc:269] 0709 12:19:54.640405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.640624 (+   219us) service_pool.cc:224] Handling call
0709 12:19:56.606945 (+1966321us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001487d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:56.607089 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3341342080}
I20260709 12:19:56.612138  6174 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:56.615123  6112 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.616766 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 1029) took 1352 ms. Trace:
I20260709 12:19:56.617237 32113 rpcz_store.cc:276] 0709 12:19:55.264785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.122714 (+857929us) service_pool.cc:224] Handling call
0709 12:19:56.306102 (+183388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.306338 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:56.306375 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:56.306431 (+    56us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:56.306476 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.306548 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.616543 (+309995us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400434ed0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:56.616690 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420077824}
I20260709 12:19:56.487349  6158 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:56.549901  6253 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:56.543778 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.626458 32038 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
W20260709 12:19:56.618440  6215 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.003s
I20260709 12:19:56.560921  6167 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:56.624677  6227 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.003s	sys 0.030s
I20260709 12:19:56.545457 32047 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
W20260709 12:19:56.630085  6227 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.003s	sys 0.030s
I20260709 12:19:56.546813  6101 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.636477 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.637301 32041 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:56.638149 32044 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:56.630208  6066 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.661077  6066 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.623773  6158 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.662572  6158 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:56.648471 31969 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
W20260709 12:19:56.643078  6193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.031s	sys 0.076s
W20260709 12:19:56.645414  6115 consensus_meta.cc:287] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.000s
W20260709 12:19:56.654551 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.004s	sys 0.019s
I20260709 12:19:56.631155  6215 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:56.665828 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.003s	sys 0.019s
W20260709 12:19:56.654850 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.018s	sys 0.005s
W20260709 12:19:56.666774 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.018s	sys 0.005s
W20260709 12:19:56.667599 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 650834us
I20260709 12:19:56.495746 32043 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:56.659125 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.660038 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2879) took 1731 ms. Trace:
W20260709 12:19:56.656742 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.016s	sys 0.007s
W20260709 12:19:56.664935 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.004s	sys 0.019s
W20260709 12:19:56.655125  6178 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.003s
W20260709 12:19:56.669117 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 662497us
W20260709 12:19:56.664266  6193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.031s	sys 0.077s
I20260709 12:19:56.673013 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.673763 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.674429 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:56.666801 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 181386us
I20260709 12:19:56.668680 32041 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:56.676479 31936 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:56.668455 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:56.668991 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.016s	sys 0.007s
W20260709 12:19:56.677506 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 619998us
I20260709 12:19:56.677748 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.678606 32119 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:19:56.668489 31966 rpcz_store.cc:276] 0709 12:19:54.929008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.966077 (+ 37069us) service_pool.cc:224] Handling call
0709 12:19:54.966444 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.966843 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.966873 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.980806 (+ 13933us) log.cc:844] Serialized 26 byte log entry
0709 12:19:56.057369 (+1076563us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:56.057427 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.057530 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:56.659832 (+602302us) raft_consensus.cc:1685] finished
0709 12:19:56.659852 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.659984 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6576,"thread_start_us":1045670,"threads_started":1,"wal-append.queue_time_us":1431692}
I20260709 12:19:56.674052 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:56.682255 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:56.656474  6058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.007s	sys 0.015s
W20260709 12:19:56.666175 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.003s	sys 0.019s
W20260709 12:19:56.684790 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 632375us
I20260709 12:19:56.679891 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:56.683238 31968 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.005s	sys 0.017s
W20260709 12:19:56.693667 31968 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.006s	sys 0.018s
I20260709 12:19:56.695151 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.666210  5441 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:56.685613 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:19:56.683065  6119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.010s	sys 0.013s
I20260709 12:19:56.670101  6115 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:56.732115 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.002s	sys 0.025s
W20260709 12:19:56.737977 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.003s	sys 0.025s
W20260709 12:19:56.739076 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 695452us
W20260709 12:19:56.739902 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54411.6us
I20260709 12:19:56.675830  6178 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.742041 32118 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:56.750618 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.751874  6149 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.754493  6153 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:56.755761 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.757226 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:56.705845 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:56.757349 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:56.758134 31965 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:56.759585  6210 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:56.759778  6066 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:56.760012  6210 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.760459  6210 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.762049  6215 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:56.762881 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57148us
I20260709 12:19:56.763093  6196 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:56.763589  6196 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.763991  6196 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:56.696400  5869 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:56.766650  5869 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.767040  6167 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:56.767124  5869 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:56.767784  6167 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:56.768165  6167 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:56.735188  6119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.010s	sys 0.018s
I20260709 12:19:56.768760  6167 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.768913  6153 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:56.768189 31965 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:56.769829  6153 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:56.770243  6153 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:56.683738  6174 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 12:19:56.771313  6153 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.772557  6174 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:56.772773  6153 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:56.774667  6259 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:19:56.772953  6174 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:56.776980 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.790787 31971 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:56.683948  6262 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:56.777805 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.791882 31970 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:19:56.778807 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:56.684425  6058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.008s	sys 0.015s
I20260709 12:19:56.784969 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.801908 31965 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:19:56.754094  6149 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:56.802363 32114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.024s	sys 0.011s
I20260709 12:19:56.705992  5441 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:56.800619 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:56.803234 32114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.025s	sys 0.011s
I20260709 12:19:56.735469  6016 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:19:56.809962 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69216.9us
I20260709 12:19:56.810984 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:56.811487 32043 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:56.811612 32044 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:56.811039 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.790028  6112 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:56.814272 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:56.825563 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.798048  6262 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.837175 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 2910) took 1625 ms. Trace:
I20260709 12:19:56.812955  5441 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:56.838076 32118 rpcz_store.cc:276] 0709 12:19:55.211537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.038277 (+826740us) service_pool.cc:224] Handling call
0709 12:19:56.836901 (+798624us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b5400433fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:56.837098 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1581366144}
I20260709 12:19:56.813824  6156 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:56.845513 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.834730 32044 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:19:56.846469 32043 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 14: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:56.831755 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 43721.5us
I20260709 12:19:56.839430 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.847251 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:56.841986 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:56.848399 31970 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:56.838608 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:56.852188 31971 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 16: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:56.854827 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.802042  6046 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:56.801311  6166 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:19:56.870030  6166 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:56.870364  6184 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:56.870451  6166 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:56.871500  6166 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.872763  6166 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.875921  6090 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:56.876497  6090 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:56.865865 32624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.000s	sys 0.023s
W20260709 12:19:56.877565 32624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.000s	sys 0.024s
I20260709 12:19:56.807746  6162 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.879320 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.823417  6112 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:56.830785  6195 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:56.866959 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.857929 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.881573 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:56.882014 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.882396 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:56.892006 32660 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:19:56.883311 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:56.892822 32652 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 23: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:56.844776  6179 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:56.897509 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.023s	sys 0.021s
I20260709 12:19:56.822371  6190 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:19:56.850255  5989 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:56.899974 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.023s	sys 0.021s
I20260709 12:19:56.827688  6149 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:19:56.822671  6016 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:56.901106 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:56.927563 32043 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:56.885221  6112 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:56.902360 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:56.931375 31971 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:56.850471  6156 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.839116  6262 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.907184 31970 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:56.894136  6195 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:56.922432 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:56.901252  6179 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.955237 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:56.956035 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.925995 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:56.894562 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:56.963266 32043 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
W20260709 12:19:56.921802 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 929951us
I20260709 12:19:56.969991 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:56.979475 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:56.930609 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:56.980456 32652 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:56.933764 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:56.896965  6239 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:56.937831 32657 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:19:56.941895  6195 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:56.937763  6156 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.980332 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.936384 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:56.987345 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 92846.7us
I20260709 12:19:56.954761  6179 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.994392 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:57.000136 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:56.989313 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:56.939971  6160 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:56.983422  6119 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:57.013947  6119 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:57.014537  6119 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:57.025465  5949 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.748s	user 0.007s	sys 0.002s
I20260709 12:19:57.001775  6248 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:57.032222  6162 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.034200 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:57.034435  6016 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.035212 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:57.036470 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:57.036862  5869 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.953528  6271 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:57.037822 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:56.878942  6174 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:56.927771  5989 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.037313 32653 raft_consensus.cc:2410] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:19:57.038962 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:19:57.039482 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 54182.2us
I20260709 12:19:57.043182 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:57.043203 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 117268us
I20260709 12:19:57.044044 32044 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:19:57.039800 32659 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:56.985267 31970 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:56.940374  6210 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.045490 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.045671 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:57.047204  6267 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:57.042587  6274 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:57.048032 32044 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:57.049431  5989 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:57.050937 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:57.078516 31971 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:57.077898 32654 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:57.082140 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:57.084431 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75807us
I20260709 12:19:57.025950  5949 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:57.087847 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 93573.3us
I20260709 12:19:57.085618 32010 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:57.089672  5760 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:57.089978 31970 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:57.087867  5870 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:56.984154  6090 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.051558  6115 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:57.078449 32045 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:19:57.078644 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.112943 32040 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:57.117506 31971 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:19:57.118863 31971 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:57.079627  6215 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.120474 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.121311 32045 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.136487 32041 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:57.079466 32043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:19:57.139029 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:57.140539 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:57.118193  6183 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.024s	sys 0.022s
W20260709 12:19:57.144013 32039 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.023s
W20260709 12:19:57.149536 32039 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.005s	sys 0.024s
I20260709 12:19:57.117409  6068 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:57.099123  6274 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:57.156592 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.157928 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:57.118048  5870 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.123061  6268 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:19:57.141402  6215 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.167198  5870 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:19:57.149334  6183 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.024s	sys 0.023s
I20260709 12:19:57.136737  5760 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:19:57.183161 31963 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.016s	sys 0.021s
W20260709 12:19:57.181882 32117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.026s	sys 0.031s
W20260709 12:19:57.187390 32117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.026s	sys 0.031s
I20260709 12:19:57.188405 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2909) took 1977 ms. Trace:
I20260709 12:19:57.189940 32043 raft_consensus.cc:476] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Not starting forced leader election -- already a leader
I20260709 12:19:57.218401 32117 rpcz_store.cc:276] 0709 12:19:55.211274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.113600 (+902326us) service_pool.cc:224] Handling call
0709 12:19:56.113977 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.114225 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:19:56.114580 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 12:19:57.187957 (+1073377us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:57.188028 (+    71us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:19:57.188097 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.188183 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.188338 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":557,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":91,"thread_start_us":1069638,"threads_started":1}
I20260709 12:19:57.181789  5870 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:19:57.231350  6162 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:57.232111  6162 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.191798 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.192703  6215 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.195060  5760 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.235641 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.177572  6068 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:57.236874 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.237437  6068 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.199770  5441 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:57.202194 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:57.183866  6158 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.002s
I20260709 12:19:57.176627  6177 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:57.204162  6167 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.205955  6179 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.194116  5949 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.202183  6196 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.213567 32045 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:57.245378 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.238293 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.268286 32045 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:57.246395 31965 raft_consensus.cc:2368] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18.
I20260709 12:19:57.264436 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.246734 32657 raft_consensus.cc:2368] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:57.263737 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.271263 32117 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:57.271636 32040 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:57.272192 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.273628 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.274981 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.275705 32045 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:57.276782 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.165910  6112 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.280370 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:57.289529 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:57.290428 32121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.010s	sys 0.036s
W20260709 12:19:57.290828 32121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.010s	sys 0.036s
I20260709 12:19:57.291774 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:57.296195 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:57.296783 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 2976) took 1427 ms. Trace:
I20260709 12:19:57.297165 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:57.240164  6191 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:57.298260 31938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.020s	sys 0.013s
I20260709 12:19:57.214095  6195 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.242053  5441 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:57.302736 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.246044  6196 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:57.293910  6193 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.004s	sys 0.028s
I20260709 12:19:57.244527  6179 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.255404  6178 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:57.246035  6167 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.305964 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:57.314428  5949 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:57.322135 31965 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
W20260709 12:19:57.323601 31968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.026s
W20260709 12:19:57.304569 31938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.020s	sys 0.014s
W20260709 12:19:57.329567 31936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.007s	sys 0.028s
I20260709 12:19:57.297164 32121 rpcz_store.cc:276] 0709 12:19:55.868905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.211022 (+342117us) service_pool.cc:224] Handling call
0709 12:19:56.211353 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.211625 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:56.279702 (+ 68077us) log.cc:844] Serialized 14 byte log entry
0709 12:19:56.681152 (+401450us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b1800263c80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890c4e07 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890e1c02 00007f228bd6920b 00007f228bd6ed23 00007f228bd6f23a 00007f228c1d9a2a
0709 12:19:57.296473 (+615321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:57.296525 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:57.296576 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.296638 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.296736 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13399,"prepare.queue_time_us":67881,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":1810,"raft.queue_time_us":5018,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":8722,"spinlock_wait_cycles":6044928,"thread_start_us":609876,"threads_started":1}
I20260709 12:19:57.328202 32010 leader_election.cc:400] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18.
I20260709 12:19:57.304386  6244 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:57.349193 32047 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
W20260709 12:19:57.351321 31936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.007s	sys 0.028s
I20260709 12:19:57.354720 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:57.355455 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2882) took 2317 ms. Trace:
I20260709 12:19:57.355671 32654 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:57.351171 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 653629us
I20260709 12:19:57.316726  6256 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:57.241906  6158 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.289171  6271 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:19:57.355890 32114 rpcz_store.cc:276] 0709 12:19:55.037924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.045980 (+  8056us) service_pool.cc:224] Handling call
0709 12:19:55.187592 (+141612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:55.187874 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.187904 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:56.104723 (+916819us) spinlock_profiling.cc:243] Waited 9.72 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:19:56.105120 (+   397us) log.cc:844] Serialized 26 byte log entry
0709 12:19:56.805861 (+700741us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:56.805919 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.806007 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:57.010521 (+204514us) raft_consensus.cc:1685] finished
0709 12:19:57.010542 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.355300 (+344758us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54001234d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:57.355403 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319134336,"thread_start_us":700325,"threads_started":1,"wal-append.queue_time_us":896174}
I20260709 12:19:57.355890 32117 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
W20260709 12:19:57.401688 31972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.012s	sys 0.044s
I20260709 12:19:57.351784 32010 leader_election.cc:403] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:19:57.403353 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 75404.5us
W20260709 12:19:57.402699 31972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.013s	sys 0.045s
I20260709 12:19:57.404278 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:57.350962 31968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.006s	sys 0.026s
W20260709 12:19:57.349361  6193 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.004s	sys 0.028s
I20260709 12:19:57.403057 32626 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:57.410759 31969 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.017s	sys 0.037s
W20260709 12:19:57.411290 31969 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.017s	sys 0.037s
I20260709 12:19:57.408946 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 852) took 1473 ms. Trace:
W20260709 12:19:57.361258  6223 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.005s	sys 0.012s
W20260709 12:19:57.415865 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.013s	sys 0.013s
W20260709 12:19:57.418211 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.013s	sys 0.014s
I20260709 12:19:57.420068 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.256372  6274 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.439481 31966 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:57.441228  6167 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:57.415225  6196 raft_consensus.cc:2804] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:57.442297  6167 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.442611  6196 raft_consensus.cc:697] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:57.443552  6196 consensus_queue.cc:237] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.444416  6273 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:57.444845  6273 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:57.446419 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.446882  5895 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:57.447319  5895 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:57.447433 32043 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:57.448189  5895 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.449316  5895 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.452693  6195 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.406489  5441 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:19:57.422890 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.410023  5907 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:57.427986  6256 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:57.430723 32654 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:57.411201  6177 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:57.435115 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:57.435624 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.436416 32038 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:57.350373  6140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.021s
I20260709 12:19:57.445621  6273 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.486547 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:19:57.488308 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53280.1us
I20260709 12:19:57.485244  6273 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.448521 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.490494 32043 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:57.455822 32047 raft_consensus.cc:2376] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19. Re-sending same reply.
I20260709 12:19:57.491897 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.433410 32659 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:19:57.492808 32659 pending_rounds.cc:85] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 14
I20260709 12:19:57.493109 32659 pending_rounds.cc:107] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:19:57.493391 32659 raft_consensus.cc:2889] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:57.465194 32086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.003s	sys 0.032s
W20260709 12:19:57.495106 32086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.003s	sys 0.032s
I20260709 12:19:57.416558  6210 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:57.424015  5869 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:57.496577 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:57.512702 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:57.513283 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.412492 31968 rpcz_store.cc:276] 0709 12:19:55.935066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.954685 (+ 19619us) service_pool.cc:224] Handling call
0709 12:19:55.955042 (+   357us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:55.955459 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.955495 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:55.982412 (+ 26917us) log.cc:844] Serialized 78 byte log entry
0709 12:19:56.694144 (+711732us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:19:56.774694 (+ 80550us) log.cc:844] Serialized 14 byte log entry
0709 12:19:56.775284 (+   590us) log.cc:844] Serialized 14 byte log entry
0709 12:19:57.231042 (+455758us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b480036c0d8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:57.242556 (+ 11514us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b18003a7a40. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:57.408630 (+166074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.408703 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:57.408757 (+    54us) raft_consensus.cc:1685] finished
0709 12:19:57.408772 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.408895 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4474,"prepare.queue_time_us":79930,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":490474,"raft.run_cpu_time_us":3161,"raft.run_wall_time_us":242234,"spinlock_wait_cycles":26929664,"thread_start_us":1300422,"threads_started":2,"wal-append.queue_time_us":972317}
I20260709 12:19:57.499011 32085 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:19:57.423813  6162 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:57.515689 32085 leader_election.cc:403] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 12:19:57.496773  6264 log.cc:927] Time spent T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:19:57.498051 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.498559 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.523123 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:57.527004 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 116335us
I20260709 12:19:57.528204 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.528997 32117 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:57.530340 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.531395 32121 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:57.532684 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.536561 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:57.485695  6140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.007s	sys 0.022s
I20260709 12:19:57.537372 31966 raft_consensus.cc:2368] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
W20260709 12:19:57.540810 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 50817.6us
I20260709 12:19:57.538419 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.523962 32114 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:57.543272 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:57.511534 32041 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:57.545348 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:57.555148 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.555160 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:57.575161 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.575881 32118 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:57.576164 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.495663  6066 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:57.516908  6149 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:57.581526 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.582002 32010 leader_election.cc:400] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:57.587677 32010 leader_election.cc:403] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:19:57.535969  5441 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:57.538632  6115 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:57.594233  6115 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.442801  6167 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.598371  6167 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:57.598413  5441 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:57.599383  6158 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:57.601157 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.598531 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.554805  6210 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:57.555080  6216 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:57.604699 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.606051 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.607511 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.611339 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.612084 32117 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.615164 32047 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 16: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:19:57.603238  6119 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
I20260709 12:19:57.640640 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.641360 32120 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.642367 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.536345  6286 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.100s
I20260709 12:19:57.616498  6066 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:57.599615  5441 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.685835 32041 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:19:57.712895 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.713922 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.714874 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.620182  5949 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.617283  6153 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:57.690043 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.038s	sys 0.053s
W20260709 12:19:57.643445 32623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.012s	sys 0.016s
W20260709 12:19:57.700574 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 296485us
I20260709 12:19:57.582458  6272 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 12:19:57.707986 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.719766 32112 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
W20260709 12:19:57.747638 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 338584us
I20260709 12:19:57.734916 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.750209 31936 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:57.685034 32117 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
W20260709 12:19:57.702215  6262 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.764s	user 0.012s	sys 0.000s
W20260709 12:19:57.707334  6156 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 12:19:57.760792 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:57.777209 32659 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:57.706400  6066 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.763957 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:57.781440 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:57.643999  6119 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:57.767457 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.041s	sys 0.056s
W20260709 12:19:57.770025 32623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.012s	sys 0.022s
I20260709 12:19:57.764027 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.785610 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.717860  6112 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:57.815893 32041 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:57.817835 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:57.818709 32657 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:57.819633 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 1.00534e+06us
I20260709 12:19:57.820520 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:19:57.817819 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 693125us
I20260709 12:19:57.824219 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.825608 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.768460  5441 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:57.822845 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:57.767925  6149 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:57.841871 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51059.9us
I20260709 12:19:57.635654  6256 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:57.841053 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:57.757591  5760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.006s	sys 0.027s
I20260709 12:19:57.845208 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2653) took 1170 ms. Trace:
I20260709 12:19:57.859863 31969 rpcz_store.cc:276] 0709 12:19:56.674926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.675100 (+   174us) service_pool.cc:224] Handling call
0709 12:19:56.705769 (+ 30669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.705997 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:57.411833 (+705836us) log.cc:844] Serialized 14 byte log entry
0709 12:19:57.844791 (+432958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:57.844855 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:57.844921 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.845001 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.845146 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122556,"prepare.queue_time_us":68271,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":1015468,"threads_started":2}
I20260709 12:19:57.847965 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:57.862612 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 283025us
I20260709 12:19:57.821607 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:57.852010 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:57.865231 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43713.8us
I20260709 12:19:57.852854 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.786550  6066 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:57.866740 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.867056 32085 leader_election.cc:400] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
I20260709 12:19:57.867852 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.853632 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.858508 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.869278 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.869325 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:57.870013 32041 raft_consensus.cc:2376] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23. Re-sending same reply.
I20260709 12:19:57.870448 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.870895 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:57.854280 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.865801 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.872310 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:57.865834 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:57.873143  6196 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:57.873538 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.873648  6196 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:57.872521  5907 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:57.868428 32112 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:57.864403 32115 raft_consensus.cc:2376] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19. Re-sending same reply.
I20260709 12:19:57.871438 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.875821 32115 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:19:57.874680  5907 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:57.877159  6196 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.879055  6066 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.881357  6066 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:57.874382 32113 raft_consensus.cc:2393] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:19:57.885946  5907 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:57.890007 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.888466  6066 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:57.893651 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:57.895224 32116 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:57.817494  5869 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:57.902376 32659 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 16.
I20260709 12:19:57.867974 32085 leader_election.cc:403] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:19:57.920605 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.903887 31936 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:57.971320 32112 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
W20260709 12:19:57.859390  5760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.006s	sys 0.027s
W20260709 12:19:57.977027 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 110362us
I20260709 12:19:57.848665  6090 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:57.785147  6262 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.013881 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2661) took 1408 ms. Trace:
I20260709 12:19:57.787140  6167 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 12:19:57.844733  6149 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:57.826994  6295 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.215s
I20260709 12:19:57.787604  6156 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.028260 32039 rpcz_store.cc:276] 0709 12:19:56.605013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.607234 (+  2221us) service_pool.cc:224] Handling call
0709 12:19:56.607597 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.607823 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:19:57.150077 (+542254us) log.cc:844] Serialized 14 byte log entry
0709 12:19:57.640512 (+490435us) log.cc:844] Serialized 14 byte log entry
0709 12:19:57.868924 (+228412us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b18003660a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:57.870195 (+  1271us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003660a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890b02a6 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f
0709 12:19:57.871563 (+  1368us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b48000fe358. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f22910f8aa1 00007f22910efb37 00007f22910ef9ca 00007f22911e726f 00007f22911e6f37 00007f22911fb55c 00007f22911fb508 00007f22911fb488 00007f22911fb443 00007f22911fa6ec 00007f2290124a96
0709 12:19:58.013513 (+141950us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:58.013570 (+    57us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:19:58.013622 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.013697 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.013816 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400084,"prepare.queue_time_us":843819,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":251767,"raft.run_cpu_time_us":3179,"raft.run_wall_time_us":170189,"spinlock_wait_cycles":52221696,"thread_start_us":1235235,"threads_started":3,"wal-append.queue_time_us":594371}
W20260709 12:19:58.052410 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 49414.2us
I20260709 12:19:57.906992  5869 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:57.827260  6294 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.203s
I20260709 12:19:58.109923  6292 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17.
W20260709 12:19:58.110849  6299 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.322s	user 0.000s	sys 0.000s
W20260709 12:19:58.111709  6297 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.365s	user 0.000s	sys 0.000s
W20260709 12:19:58.112959  6304 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.294s	user 0.000s	sys 0.000s
W20260709 12:19:58.115554  6307 log.cc:927] Time spent T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:19:58.116258  6302 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.333s	user 0.000s	sys 0.000s
W20260709 12:19:58.117022  6309 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 12:19:58.119470  6179 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:58.120270  6119 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.125444 32657 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:19:58.147082  6153 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:19:58.149893  6191 consensus_meta.cc:287] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.843s	user 0.003s	sys 0.006s
W20260709 12:19:58.405762  6244 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.005s	user 0.010s	sys 0.003s
I20260709 12:19:58.151329  6293 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:58.154537 32660 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 1105) took 1160 ms. Trace:
I20260709 12:19:58.155642  6262 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:58.060546  6284 log.cc:927] Time spent T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.448s	user 0.001s	sys 0.001s
I20260709 12:19:58.159066  6248 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:58.167249  5895 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.717s	user 0.013s	sys 0.000s
I20260709 12:19:58.174714 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:19:58.175596  6115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.001s	sys 0.001s
I20260709 12:19:58.017325  6166 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:58.026840  6184 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:19:58.018325  6090 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:58.070209  6149 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:58.257788  6316 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:19:58.209578  6158 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:58.206202  6174 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:58.040439  6268 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.194s
I20260709 12:19:58.209141  6314 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:58.239920  5907 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:58.279479  5989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.011s	sys 0.014s
W20260709 12:19:58.346139  6273 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.857s	user 0.010s	sys 0.003s
W20260709 12:19:58.350234  6216 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.734s	user 0.007s	sys 0.000s
I20260709 12:19:58.311249  6329 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:19:58.355178  6156 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:58.351595  6272 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:58.448499  6216 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:19:58.449204  5989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.012s	sys 0.015s
I20260709 12:19:58.449016  6272 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.450598  6119 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:58.451030  6119 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:58.451449  6272 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:58.364310  5949 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.616s	user 0.007s	sys 0.004s
I20260709 12:19:58.452261  5949 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22.
I20260709 12:19:58.451978  6119 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:57.896293  6066 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:58.373370  6338 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:58.453471  6294 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:58.453800  6338 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.454362  6294 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:19:58.403247  6310 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 12:19:58.408108  6210 consensus_meta.cc:287] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.001s
I20260709 12:19:58.456329  6149 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:58.408767  6290 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.792s	user 0.006s	sys 0.003s
I20260709 12:19:58.123621  5869 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.458479  6314 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.459239  6149 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:58.460834  6314 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:58.388314  6313 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:58.421192  6271 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.398828  6179 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:58.428894  6244 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.429780  6153 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:19:58.424793  6337 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.206s	user 0.001s	sys 0.004s
W20260709 12:19:58.437227  6177 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.946s	user 0.011s	sys 0.003s
I20260709 12:19:58.437784  6248 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:19:58.439162  6115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.001s	sys 0.001s
I20260709 12:19:58.439713  6166 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:58.467034  6166 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:58.463948  6244 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.456277  6338 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.464639  6271 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.468919  6179 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.329990 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:58.315528 31935 leader_election.cc:336] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.501s (SENT)
W20260709 12:19:58.601589 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 286043us
W20260709 12:19:58.190122 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 286321us
I20260709 12:19:58.124001 31967 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:58.524941 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2673) took 1433 ms. Trace:
I20260709 12:19:58.270610 32039 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:58.169153 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2828) took 1118 ms. Trace:
I20260709 12:19:58.611856 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2837) took 1520 ms. Trace:
W20260709 12:19:58.280239 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 488860us
I20260709 12:19:58.612636  6162 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:19:58.173794 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 299900us
W20260709 12:19:58.613394 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 284001us
I20260709 12:19:58.614768 31938 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:58.526857 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 100956us
W20260709 12:19:58.228901 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.526703 32041 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:19:58.293860 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:58.626986 32112 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:58.529330 32117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.014s	sys 0.032s
W20260709 12:19:58.628101 32117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.015s	sys 0.032s
W20260709 12:19:58.255441 32120 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.000s
I20260709 12:19:58.631856 32120 raft_consensus.cc:2410] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:19:58.253985 32121 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:58.225804 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2831) took 1142 ms. Trace:
I20260709 12:19:58.633316 32047 rpcz_store.cc:276] 0709 12:19:57.083617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.616995 (+533378us) service_pool.cc:224] Handling call
0709 12:19:57.617379 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.706669 (+ 89290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:57.706703 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:57.707185 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:19:57.913265 (+206080us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:57.913331 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.913438 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.225528 (+312090us) raft_consensus.cc:1685] finished
0709 12:19:58.225555 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.225734 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67732,"raft.queue_time_us":105360,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":17838,"thread_start_us":146471,"threads_started":1,"wal-append.queue_time_us":357602}
I20260709 12:19:58.635061 32116 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:58.636662 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.288144 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:58.530941 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2829) took 1451 ms. Trace:
I20260709 12:19:58.638509 32044 rpcz_store.cc:276] 0709 12:19:57.079741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.081047 (+  1306us) service_pool.cc:224] Handling call
0709 12:19:57.085319 (+  4272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.085632 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:57.085657 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:57.086112 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:19:57.527634 (+441522us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:57.527675 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.527759 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.530275 (+1002516us) raft_consensus.cc:1685] finished
0709 12:19:58.530338 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.530699 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3825,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":437237,"threads_started":1,"wal-append.queue_time_us":609729}
I20260709 12:19:58.641004 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.299723 32113 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:58.298305 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 274557us
I20260709 12:19:58.430922 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.433645 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.424549 32660 rpcz_store.cc:276] 0709 12:19:56.994559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.996601 (+  2042us) service_pool.cc:224] Handling call
0709 12:19:56.997019 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.000462 (+  3443us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:57.000507 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:57.001025 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 12:19:57.513941 (+512916us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:57.514002 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.514113 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.154255 (+640142us) raft_consensus.cc:1685] finished
0709 12:19:58.154280 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.154473 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30071,"raft.queue_time_us":3133,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":17942,"thread_start_us":414606,"threads_started":1,"wal-append.queue_time_us":1099798}
I20260709 12:19:58.645448  6162 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.611299 32042 rpcz_store.cc:276] 0709 12:19:57.050848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.079487 (+ 28639us) service_pool.cc:224] Handling call
0709 12:19:57.079818 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:57.085319 (+  5501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:57.085346 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:57.092607 (+  7261us) log.cc:844] Serialized 52 byte log entry
0709 12:19:57.509064 (+416457us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:57.509104 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.509177 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.168868 (+659691us) raft_consensus.cc:1685] finished
0709 12:19:58.168898 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.169083 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5020,"raft.queue_time_us":7554,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":416021,"threads_started":1,"wal-append.queue_time_us":933503}
I20260709 12:19:58.396940 32653 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:58.431651 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:58.433108 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.434816 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:58.429563 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.441926  6090 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.657064 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2861) took 1022 ms. Trace:
I20260709 12:19:58.657519 32041 rpcz_store.cc:276] 0709 12:19:57.634478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.870687 (+236209us) service_pool.cc:224] Handling call
0709 12:19:58.648963 (+778276us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540013ced4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:58.656928 (+  7965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505513216}
I20260709 12:19:58.543440  5949 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:58.398478  6016 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:58.659186 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:58.659826  6016 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:58.463738  6066 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:58.453316  6119 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:58.464270  6153 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:58.447615  6158 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:58.457525  6210 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:19:58.670274 32658 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:58.672214  6293 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:58.462957  6313 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:58.456007  6294 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.674796  6327 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:58.675364 32658 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:58.675421  6327 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:58.675678  6294 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:58.677444  6327 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.467514  6332 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:58.686757  6332 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:58.688270  6332 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.465534  6177 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18.
I20260709 12:19:58.548771 32656 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:58.553045  6248 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.571874 32040 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
W20260709 12:19:58.574594 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.691594  6248 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:58.577447  6184 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:58.692330  6184 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:19:58.580377 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.562391  6179 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.587281 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:58.698975 32118 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:58.659842  5949 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:58.700507 32040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2842) took 1602 ms (client timeout 1880 ms). Trace:
W20260709 12:19:58.700990 32040 rpcz_store.cc:269] 0709 12:19:57.097598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.289177 (+191579us) service_pool.cc:224] Handling call
0709 12:19:58.700266 (+1411089us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002fce54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:58.700408 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902303488}
I20260709 12:19:58.605918  6271 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:58.703329 32656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1197) took 1346 ms. Trace:
I20260709 12:19:58.702255 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.703912 32656 rpcz_store.cc:276] 0709 12:19:57.356968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.357148 (+   180us) service_pool.cc:224] Handling call
0709 12:19:57.359825 (+  2677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.691136 (+1331311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.703039 (+ 11903us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400323f54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:58.703259 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2728758528}
I20260709 12:19:58.704535 32040 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:19:58.705255 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:58.677819  6158 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.612427 32039 rpcz_store.cc:276] 0709 12:19:57.091786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.234174 (+1142388us) service_pool.cc:224] Handling call
0709 12:19:58.270375 (+ 36201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.611316 (+340941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.611692 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:58.705925 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.619510 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:58.621770 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2891) took 1730 ms. Trace:
W20260709 12:19:58.624871 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.690228  6332 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.301524 32046 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:58.608382 32038 rpcz_store.cc:276] 0709 12:19:57.090860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.231085 (+1140225us) service_pool.cc:224] Handling call
0709 12:19:58.231477 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.231761 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:58.380792 (+149031us) log.cc:844] Serialized 14 byte log entry
0709 12:19:58.523690 (+142898us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:58.523865 (+   175us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:58.524046 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.524232 (+   186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.524596 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":173189,"prepare.run_cpu_time_us":1498,"prepare.run_wall_time_us":14684,"raft.queue_time_us":19190,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":357,"thread_start_us":259909,"threads_started":2,"wal-append.queue_time_us":83651}
I20260709 12:19:58.663293  6016 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.646238 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.396708 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.734323 32654 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:58.647660 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.658980 32655 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:58.738032 32042 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:58.738463 31972 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:58.738895 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:58.739598 31971 raft_consensus.cc:1240] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:19:58.739743 32042 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:19:58.740190 31969 raft_consensus.cc:2410] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:58.652294 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.740460 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2846) took 1601 ms. Trace:
I20260709 12:19:58.741072 32013 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:58.741693 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:58.741972 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:58.742550 31970 raft_consensus.cc:2376] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16. Re-sending same reply.
I20260709 12:19:58.743120 31964 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 17: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:58.744519 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:58.745489 31963 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
W20260709 12:19:58.667493 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:58.476804  6326 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.258s	user 0.002s	sys 0.002s
I20260709 12:19:58.679138 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:58.750238 31968 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:58.751971 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2843) took 1654 ms. Trace:
I20260709 12:19:58.752471 32040 rpcz_store.cc:276] 0709 12:19:57.097862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.717377 (+1619515us) service_pool.cc:224] Handling call
0709 12:19:58.732873 (+ 15496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.751477 (+ 18604us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:58.751525 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:58.751600 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:58.751682 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.751757 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.751888 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17864}
W20260709 12:19:58.753266 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 101628us
I20260709 12:19:58.756809  6255 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.758212  6295 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:58.758814  5895 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.759121  6295 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.759438  5895 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.761761  5895 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.755527 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:58.679466  6153 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.675421  6313 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:58.681484 31935 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.502s (SENT)
I20260709 12:19:58.691242 32045 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:58.764298  6162 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:19:58.764763 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:58.765760 32045 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2850) took 1449 ms (client timeout 1501 ms). Trace:
W20260709 12:19:58.766347 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 84919us
I20260709 12:19:58.764171  6271 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:58.766371 32045 rpcz_store.cc:269] 0709 12:19:57.316304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.407174 (+ 90870us) service_pool.cc:224] Handling call
0709 12:19:58.765520 (+1358346us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540037a854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:58.765684 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2909755136}
I20260709 12:19:58.768008  6162 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.706295 32656 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:19:58.710134 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90865.2us
I20260709 12:19:58.740901 32042 rpcz_store.cc:276] 0709 12:19:57.139400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.738829 (+1599429us) service_pool.cc:224] Handling call
0709 12:19:58.739308 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:58.740194 (+   886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.740385 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:58.659042 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:58.713038 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2678) took 1620 ms. Trace:
I20260709 12:19:58.709355 31967 rpcz_store.cc:276] 0709 12:19:56.891108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.893070 (+  1962us) service_pool.cc:224] Handling call
0709 12:19:56.925536 (+ 32466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.606999 (+1681463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.621078 (+ 14079us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400312754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:19:58.621411 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2755722112}
I20260709 12:19:58.766755  5895 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.773016 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:58.790699 32042 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:58.795102 32043 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:19:58.773916 32041 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:58.783356 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:58.797868 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:58.779366 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:58.725585 32038 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:58.774106 32046 rpcz_store.cc:276] 0709 12:19:57.092269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.299544 (+1207275us) service_pool.cc:224] Handling call
0709 12:19:58.299952 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.712440 (+412488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.712828 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:58.702102  5949 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.800186 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2840) took 1707 ms. Trace:
I20260709 12:19:58.707568  6327 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:58.800592 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:58.800988 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.744686 31966 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 12:19:58.768296 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.753997 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.764457  6153 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.802640  5895 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:58.576327  6244 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.761790  6295 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:58.757580  6255 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.771284  6313 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:58.772832 31938 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:58.818578  6255 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:19:58.819293 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46687.2us
I20260709 12:19:58.800619 32038 rpcz_store.cc:276] 0709 12:19:57.097008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.715077 (+1618069us) service_pool.cc:224] Handling call
0709 12:19:58.715654 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.799816 (+ 84162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.800055 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:58.821151 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:58.709616  6294 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:58.821839 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.800714  5949 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:58.822552  5949 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:58.823024  5949 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.823268 31963 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:58.823786  6016 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:58.823844 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:58.799232 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69110.2us
I20260709 12:19:58.733300 32118 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:58.826822  6162 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:58.828502 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:58.829327  6244 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:58.831224 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2857) took 1242 ms. Trace:
I20260709 12:19:58.831667 32043 rpcz_store.cc:276] 0709 12:19:57.588781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.609452 (+ 20671us) service_pool.cc:224] Handling call
0709 12:19:58.830981 (+1221529us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000339d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:19:58.831137 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1953824128}
I20260709 12:19:58.815630 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:58.815963 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.834515 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:58.821974 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:58.821244  6295 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:58.834587  6294 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:58.820670 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:19:58.769305 31935 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.515s (SENT)
I20260709 12:19:58.836351 32654 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:58.837499 31967 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:58.839125 32044 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:58.839778 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:58.850217 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:58.851071 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:19:58.858615 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:58.863440 32041 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:58.819756  6184 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:58.864725  5949 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.877872  5949 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.873224  6294 raft_consensus.cc:3060] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:58.887094 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47484.4us
I20260709 12:19:58.888172 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:58.888247  6294 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:58.889367 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:58.890548 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:58.892213  6156 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:58.892123  6090 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.893030  6156 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:58.901459 31968 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:58.903254 31967 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:19:58.905412 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 105174us
I20260709 12:19:58.908100 32659 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 14.
I20260709 12:19:58.909237 32658 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:19:58.918076 31966 raft_consensus.cc:2393] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:58.919320 32042 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:58.923297 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:58.924968 32040 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:19:58.926745 32043 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:19:58.922616  6294 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:58.930780 32047 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 27: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:19:58.930756 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.947432 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.927745  6294 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:58.956094 32651 raft_consensus.cc:2410] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
W20260709 12:19:58.956169 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.955924 32657 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
W20260709 12:19:58.957792 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 188534us
W20260709 12:19:58.958508 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.958740 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:58.959205 31935 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.605s (SENT)
I20260709 12:19:58.959709 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:58.960032 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:58.961181 31935 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.646s (SENT)
I20260709 12:19:58.961318 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.949656  5949 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:58.970782  5949 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:58.969898  6262 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:19:58.961736  6314 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.973278 32045 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:19:58.962405  6162 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:58.962690 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2679) took 1870 ms. Trace:
I20260709 12:19:58.974735 32039 rpcz_store.cc:276] 0709 12:19:57.092561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.707947 (+1615386us) service_pool.cc:224] Handling call
0709 12:19:58.708445 (+   498us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:58.747057 (+ 38612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:58.747178 (+   121us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:58.748088 (+   910us) log.cc:844] Serialized 78 byte log entry
0709 12:19:58.913502 (+165414us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:58.913548 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.913904 (+   356us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.962408 (+ 48504us) raft_consensus.cc:1685] finished
0709 12:19:58.962429 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.962606 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21423,"raft.queue_time_us":38148,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":223,"thread_start_us":123041,"threads_started":1,"wal-append.queue_time_us":207768}
I20260709 12:19:58.977078 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:58.949460  6255 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.979274  6262 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:58.963563  6119 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:58.981294  6262 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:58.982565  5869 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:58.984263  6166 consensus_meta.cc:287] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.004s
I20260709 12:19:58.984879  6166 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.985579  5869 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:58.963088 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:19:58.966464  6313 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:58.971271  5949 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.987232 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:58.989991 32042 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:19:58.979044  6255 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.031868  6255 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:58.980477 32039 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:58.961676 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:58.965803 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:59.034739 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73634.3us
W20260709 12:19:59.035190 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69508.3us
I20260709 12:19:59.034694  6314 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:59.037678  6332 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:59.039419  6332 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:59.039852  6332 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:59.040777 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:59.041625 32657 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:19:58.972865 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:19:59.042387 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
W20260709 12:19:58.992969 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 166389us
W20260709 12:19:59.043325 32084 connection.cc:256] RPC call timeout handler was delayed by -1.7933944040000824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 12:19:59.044152 32084 connection.cc:256] RPC call timeout handler was delayed by -1.7717944040000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 12:19:59.044269 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:59.037672 31966 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:19:58.982565  6262 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.994381  6294 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:58.995445 31866 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 153) took 19920 ms (client timeout 15000 ms). Trace:
I20260709 12:19:59.047073  6262 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:59.046702 31866 rpcz_store.cc:269] 0709 12:19:39.074683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.074993 (+   310us) service_pool.cc:224] Handling call
0709 12:19:58.995229 (+19920236us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:53.185409 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:53.430358 (+244949us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:53.432631 (+  2273us) write_op.cc:435] Acquired schema lock
0709 12:19:53.446114 (+ 13483us) tablet.cc:662] Decoding operations
0709 12:19:53.660580 (+214466us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:53.672377 (+ 11797us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:53.704744 (+ 32367us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:19:53.981501 (+276757us) tablet.cc:701] Row locks acquired
0709 12:19:53.981529 (+    28us) write_op.cc:260] PREPARE: finished
0709 12:19:53.999451 (+ 17922us) write_op.cc:270] Start()
0709 12:19:54.009635 (+ 10184us) write_op.cc:276] Timestamp: P: 1783599593995809 usec, L: 0
0709 12:19:54.009665 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:19:54.116660 (+106995us) log.cc:844] Serialized 4297 byte log entry
0709 12:19:56.097924 (+1981264us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cae28 00007f228fc05581 00007f228fc04d58 00007f228fc099f1 00007f228fc099a9 00007f228fc09939 00007f228fc09901 00007f228fc08b2d 0000000000388634 0000000000388479
0709 12:19:56.935420 (+837496us) op_driver.cc:464] REPLICATION: finished
0709 12:19:57.699146 (+763726us) write_op.cc:301] APPLY: starting
0709 12:19:57.708416 (+  9270us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:57.863958 (+155542us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cc32a 00007f228c151f14 00007f228c151ec3 00007f228c151e38 00007f228c151df3 00007f228c1510dc 00007f228c17ae16 00007f228c162cbb 00007f228c14d39d 00007f228c150d75
0709 12:19:58.310075 (+446117us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:58.310127 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:58.351479 (+ 41352us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:58.351752 (+   273us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 12:19:58.351798 (+    46us) write_op.cc:312] APPLY: finished
0709 12:19:58.352110 (+   312us) log.cc:844] Serialized 105 byte log entry
0709 12:19:58.352319 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:58.352467 (+   148us) write_op.cc:454] Released schema lock
0709 12:19:58.352515 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:58.356836 (+  4321us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b18002a38e0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e2522 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:19:58.449759 (+ 92923us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b540036c5a8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f229258b9e1 00007f229258a64c 00007f228fc1b3fd 00007f228fc07389 00007f228fc06b53 00007f228fc0ad61 00007f228fc0ad19 00007f228fc0aca9 00007f228fc0ac71 00007f228fc09e9d 0000000000388634
0709 12:19:58.450223 (+   464us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:58.450251 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":166689,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":689946,"apply.run_cpu_time_us":29436,"apply.run_wall_time_us":807308,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50063,"num_ops":11,"prepare.queue_time_us":1130109,"prepare.run_cpu_time_us":41874,"prepare.run_wall_time_us":2932980,"raft.queue_time_us":1246111,"raft.run_cpu_time_us":22254,"raft.run_wall_time_us":997389,"replication_time_us":2887353,"spinlock_wait_cycles":127574144,"thread_start_us":2967502,"threads_started":4,"wal-append.queue_time_us":1486186,"wal-append.run_cpu_time_us":34875,"wal-append.run_wall_time_us":2033061}]]}
I20260709 12:19:59.053189 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:58.998885  6248 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:59.056612  6248 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:19:58.964622 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 148057us
I20260709 12:19:59.057180  6248 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:59.057754 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:58.999136  5949 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.001427 32660 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:59.059158 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:59.059544 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.059850 31966 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 13.
I20260709 12:19:59.060101  5949 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.060364  6016 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:59.060806  5949 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:59.058202  6248 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.075162 32114 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:59.007156  6119 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.075831  6119 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:59.010012  6090 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.012534 32041 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:19:59.013775 32045 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:19:59.017618  5869 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.018805 32046 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:19:58.934127  6156 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.026516  6318 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:59.027762  6166 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.038041 31967 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:58.990609 31963 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:59.038161 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:59.041046  6332 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:58.990633  6313 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:58.998163 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:59.055143 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6754944040003465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 12:19:59.091511 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6583944040003189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
I20260709 12:19:59.056666  5895 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:58.917028  6160 consensus_queue.cc:1059] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:19:59.063812 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.002135  6184 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:59.065075  6272 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:59.066177 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:59.066498  6066 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.067792 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:59.070068 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:59.070072 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:59.071074 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:19:59.071264 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:59.071497 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.071965 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:59.072392 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:19:59.087734 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.092607 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.084276 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.097328 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.098467 31964 raft_consensus.cc:2468] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:59.104079 31967 raft_consensus.cc:1275] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:19:59.105795 32084 connection.cc:256] RPC call timeout handler was delayed by -1.5269944040001064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 12:19:59.109119 31970 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:19:59.106832 32084 connection.cc:256] RPC call timeout handler was delayed by -1.4999944040000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22519999999999998s for a response
I20260709 12:19:59.112941 31968 raft_consensus.cc:2410] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:19:59.116288 32084 connection.cc:256] RPC call timeout handler was delayed by -1.0196127159997559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
I20260709 12:19:59.117290 31971 raft_consensus.cc:2393] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:59.117724 31969 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:19:59.114276 31972 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:19:59.123914 32044 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:59.100924  6295 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:59.084304  5869 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:59.127681 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.105305  6314 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:59.140177  6294 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.104888  6162 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 12:19:59.142762 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 144797us
I20260709 12:19:59.144842 31970 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:59.142230  5895 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:59.141969  6313 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.147675  6338 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:59.148765  6174 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:59.149374  6174 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:59.150337  6174 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:59.154899 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 83478.6us
I20260709 12:19:59.156132 31935 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:19:59.158535 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.123296  6327 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.163594 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:59.167184 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.168036  5895 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:59.123045  6166 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:59.171478  5895 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.171768  6166 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.173985  6166 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.117969  6318 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.176438 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.165316  6066 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.181229  5869 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.183056  6179 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:59.183459  6179 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:59.184403  6179 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.185673  6179 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:59.175837 31864 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 14 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:59.187804  6184 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:59.188243  6184 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:59.188410  6327 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.188526 31864 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:59.190697  6327 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.191151 32113 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:19:59.191512  6272 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:59.192299  6272 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.194536  6272 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:59.191512 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 127873us
I20260709 12:19:59.195780  6318 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.196403 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.198853  5949 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:59.199652 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.205248 31966 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:19:59.207535  6346 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:59.208081  6346 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:59.208570  6346 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:59.208998 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:59.209223 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 101150us
I20260709 12:19:59.209836  6294 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.210525 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.210773  6016 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:59.211030 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.212350  6016 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:59.212980 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:59.216413  6184 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.222926  5869 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:59.233966 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.227705  6294 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:59.246513 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70206.2us
I20260709 12:19:59.232991  6160 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.237859  6090 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:59.243530  6262 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:59.251573  6160 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.254606  5949 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:59.287547 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:19:59.288128 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.241921  6119 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.239241  6016 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 12:19:59.288775 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.293295 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 132545us
I20260709 12:19:59.280731  6294 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:19:59.302802 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.303215  6294 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.304410 32651 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:59.307274 32660 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 12:19:59.307542 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84748.9us
I20260709 12:19:59.282588  6160 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.287292  6184 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.276443  6244 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:19:59.318106 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:19:59.318023 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 98438.3us
W20260709 12:19:59.321229 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57990.6us
W20260709 12:19:59.323534 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.312433  5869 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:59.343876 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.323362  6090 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.349753 31935 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.315089  6262 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.356027  6191 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.265277 31864 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 16 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:59.361057  6357 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:19:59.322376  6338 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:19:59.351619  6090 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:59.350807  6119 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:59.366966 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54721.4us
W20260709 12:19:59.368813 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 73980.1us
I20260709 12:19:59.373507  5869 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:59.368160  6090 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.387082 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.392118 32113 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:59.394516  6364 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:59.407037 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.407759 32121 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:19:59.408782  6248 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:59.412325 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.380553  6191 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.416757  6191 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.425490 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.426465 32117 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:19:59.426662 32116 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:19:59.425588  6346 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.427646 32657 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:59.427945 32119 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
W20260709 12:19:59.428220 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47326.1us
I20260709 12:19:59.428602  6346 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.428947 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.429391  6262 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.430574  6262 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:59.432637 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.432965 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.433620  6294 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.433938  6346 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:19:59.434324  6294 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:59.434564 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47602.3us
I20260709 12:19:59.436332 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.438519  6364 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:59.440366  6210 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:59.441288  6327 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:59.430079 32114 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
W20260709 12:19:59.453310 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.450459 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.456614 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.460724  6314 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:59.461016 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.461359  6314 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.462272 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.462893 32113 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:59.463609  6314 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.463677 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.464906  6314 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.465556  6314 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.460724 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.468592 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.469493  6328 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:59.469883  6328 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:59.470240  6328 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:59.474555  6318 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:59.475788  6318 raft_consensus.cc:2764] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:19:59.476490  6318 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:59.476871  6318 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:59.441881  6166 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.477799  6318 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.479185  6166 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:59.481559  6119 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:59.482110  6166 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.482609  6119 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:59.483424  6119 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:59.483798  6119 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:19:59.484614  6119 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.459347  6244 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:19:59.482712 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.486162  6119 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.443424  6327 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:59.494060 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.489869  6210 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:19:59.496891 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:19:59.497249  6210 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:59.497516 31971 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:59.497661  6210 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:19:59.499596 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.498661  6210 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.501187  6210 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:59.505319 31971 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:59.448141  6153 consensus_queue.cc:1059] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:19:59.447345  6368 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:19:59.519755 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:19:59.524355 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 55864.9us
W20260709 12:19:59.525559 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.413939 31864 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 10 to 13, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 13 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:59.547788 31864 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 13 to 16, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 16 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:59.551374  6314 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.502707  6160 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.557525  6179 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:19:59.561961  6179 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.565079  6179 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:19:59.572988  6375 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:19:59.573761 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.574774  6153 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:59.575313  6153 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:59.576260  6153 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.579907  6160 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:59.511757  5869 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:59.532835  6318 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:59.583364 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.586099 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 147754us
I20260709 12:19:59.534755  6255 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.588310 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.590240 32113 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:59.594394 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 111799us
I20260709 12:19:59.599510 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:19:59.600339 32658 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.606222 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.561064  6166 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.618367 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.594826  6184 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:19:59.621320 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.595878  6255 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.598199  6090 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.611562  6370 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:59.607368  6368 consensus_queue.cc:1048] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:19:59.629428 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.601231  6327 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.624619  6313 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:59.635210 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.635864 32118 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:19:59.641736  6293 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:59.651425 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53661.6us
W20260709 12:19:59.652796 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.663138  6374 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:19:59.677930 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:59.677403 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.679095 32658 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:19:59.680649 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.672013 32112 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:59.682220 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.681774 32112 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:59.689031 32116 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:19:59.690991  6377 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.007s
I20260709 12:19:59.691715 31864 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:19:59.694906 32112 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.696255 32112 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.640403  6313 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:19:59.703027 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.707157  5869 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:19:59.707528 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.707862  5869 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.708938 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.710057  5869 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.677958  6293 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:19:59.712774  6293 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.713434  6328 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.715054  6255 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:59.715250  6244 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.715783  6244 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.716001  6313 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.717324 32114 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:59.718499  6371 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:59.718926  6371 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:19:59.719018  6166 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.719337  6371 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:59.719686  6166 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:19:59.720237  6166 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.722157 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.680896  6174 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:59.741683 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 126795us
I20260709 12:19:59.741854 32114 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:59.739008 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:59.748088 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.743498  6327 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:59.763099 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:59.767501 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:19:59.768883 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:19:59.763072  5895 raft_consensus.cc:2764] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:59.744418  6255 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:19:59.783597 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.774060  5949 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:59.775707  6179 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:19:59.764611  6191 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
W20260709 12:19:59.784932 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 111182us
I20260709 12:19:59.763835  6346 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:19:59.788384 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66393.6us
I20260709 12:19:59.790232  6381 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.790486 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.790788  6381 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.790820  6255 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:59.791306  6381 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.791419  6255 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:59.792624  6255 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.793923  6255 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:59.767402  6272 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 12:19:59.794828  6378 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:19:59.795586 32653 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:59.796687  6191 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:59.797176  6191 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:19:59.797538 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.797977  6191 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:19:59.787534  5949 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.799335  6191 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.799279 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:19:59.800716  6294 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:59.801424 32119 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 14.
I20260709 12:19:59.803544 32114 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:59.767417  6244 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.753137 31864 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 15 to 20, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 20 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:59.814920 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:59.786638  6090 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.819707 32116 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:19:59.799309  5949 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:59.803282  6248 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:59.831933 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:19:59.837595 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.838598  6391 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:59.839136  6382 log.cc:927] Time spent T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:19:59.839388  6391 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:19:59.841058  6385 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:19:59.843041  6391 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:19:59.845299  6391 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:59.847250  6383 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:19:59.851713  6371 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:19:59.853988  6371 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:19:59.830731  6346 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.854604 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:19:59.817893  6066 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:19:59.854534 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 94323.2us
W20260709 12:19:59.894654 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.855530  6362 raft_consensus.cc:2764] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:19:59.861209  6381 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:19:59.855134 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64742.4us
I20260709 12:19:59.882980 32651 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 12:19:59.883276 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 107981us
I20260709 12:19:59.830880  5895 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:59.884817 32112 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:19:59.835721  6166 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:19:59.888839 32113 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 15.
I20260709 12:19:59.890120 32657 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:19:59.883647 32114 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:19:59.901602 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:19:59.912781 32118 raft_consensus.cc:606] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:19:59.913303 32118 raft_consensus.cc:614] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:59.902428 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:19:59.903852  6332 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:59.899762 32112 pending_rounds.cc:85] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 10
I20260709 12:19:59.915473 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:19:59.917317  6191 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:19:59.932457 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:19:59.918160 32112 pending_rounds.cc:107] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:19:59.934335 32112 raft_consensus.cc:2889] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:59.939632 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.940505  6119 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:59.952349 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.952343 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:19:59.981988 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 181121us
I20260709 12:19:59.984553 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:19:59.985334 32113 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:19:59.986855 32112 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
W20260709 12:19:59.996131 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 89169.7us
I20260709 12:19:59.999513 32084 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:19:59.941143  6119 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:20:00.001840  6216 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:20:00.002036 32660 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:00.003302 32657 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:00.003657 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:19:59.979984  6294 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:59.990289  6066 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:00.047812 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.066828 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.013890  6362 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:00.099503 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.060302  5895 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:00.018640  6066 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:00.125181 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 136319us
W20260709 12:20:00.103874  6392 log.cc:927] Time spent T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:20:00.134761 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.016813  5869 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:20:00.175117  5895 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.075830  6272 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 12:20:00.138171  6179 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:20:00.177959  6381 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:00.096997  6362 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:00.075021  6166 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.111s
W20260709 12:20:00.182199 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 198893us
I20260709 12:20:00.014168  6318 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.107s
W20260709 12:20:00.184841 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 274847us
I20260709 12:20:00.134496  6066 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:00.194804 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:00.183866  6396 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:20:00.197500  6362 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:00.221946 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.212759 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:00.204608  6016 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:00.249634 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:00.254696 32118 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:00.267336 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 242816us
I20260709 12:20:00.198902  6119 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.277856  5869 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.259317  6396 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:00.256114  6016 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:00.281791 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78241.9us
W20260709 12:20:00.285243 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 147365us
I20260709 12:20:00.270792  6371 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:00.282821  6328 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:00.280267  6404 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:00.297672 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:00.346918 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.361457 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65008us
I20260709 12:20:00.288656  6396 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:20:00.356326 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84758.7us
I20260709 12:20:00.377449 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:00.378057 32118 raft_consensus.cc:606] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:00.378412 32113 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
W20260709 12:20:00.378639 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 175089us
I20260709 12:20:00.378464 32118 raft_consensus.cc:614] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:00.296881  6371 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:00.371330 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:00.381783 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:00.393193 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 198526us
I20260709 12:20:00.357734  6294 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:00.373351  6293 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.394097 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:00.299333  6272 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:20:00.395560 32114 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:20:00.377166  6386 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:00.403326 32121 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:00.386603  6179 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:00.380908  6404 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:00.421912 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:00.423278 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:00.430289 32041 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:00.430437 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:20:00.372051  5869 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.438850  6396 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.445230  6403 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:00.446382  6390 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:20:00.446784  6408 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:00.379279  6016 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.424952  6179 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:00.458513 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.748s (SENT) [suppressed 34 similar messages]
I20260709 12:20:00.408810  6386 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:00.464485 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70459.9us
I20260709 12:20:00.470973 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:00.448228  6346 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.003s
W20260709 12:20:00.447863  6400 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:20:00.428610  6318 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:00.470060  6179 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:00.503261 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.508440 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 152325us
I20260709 12:20:00.409036  6293 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.470765  6332 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:00.512408  6410 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:00.512940  6410 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:00.513792  6293 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:00.514132  6410 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.516520  6407 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:00.517145  6407 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:00.518796 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47915.7us
I20260709 12:20:00.518612  6294 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.519585 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:00.520478  6362 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.520977 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:00.524039 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:00.496407  6318 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:00.524943 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.526257 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:00.509177  5949 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.003s
I20260709 12:20:00.502506  6346 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:00.520608  6411 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:00.524823 32660 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:00.538745  5949 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.448570  6371 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.522023  6314 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:00.564996 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:00.565254 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:00.573367 32651 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:00.534730  6318 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:00.466156  6403 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:00.542284  6397 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:20:00.636492  6383 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:00.637198  6383 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:00.647451 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.651459  6395 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:00.652127  6395 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:00.653270  6395 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.658282 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:00.659071 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:00.706555  6395 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:00.663795 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:00.707484 31967 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:00.717111 31967 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:20:00.575662  6293 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:00.584584  6191 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:00.780479 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.781314 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.781728 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.785383 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:00.785660 32653 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 12:20:00.787825 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 222902us
I20260709 12:20:00.756521  6293 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:00.760958  6379 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 12:20:00.806272 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 268790us
W20260709 12:20:00.778717  6387 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:20:00.789317  6399 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:00.795883  6413 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:00.817270  6399 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.821705  6399 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:00.792210  6394 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:20:00.761307  6160 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.817317  6413 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:00.822968 31967 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:00.806581  6293 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:00.761174  6166 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:00.792065  6314 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:00.806185  6408 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.854250  6332 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:00.805346  6184 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.839287 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:00.797197  6407 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:00.771857  6191 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.840565 31972 raft_consensus.cc:1240] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:00.552723  6386 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:00.844414  6419 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:20:00.844336 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:00.839313 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:00.846704  5869 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:00.577183  6404 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.842409  6314 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:00.815828  6432 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:00.850723  6415 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:00.851666  6422 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:20:00.761735  6421 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.402s
W20260709 12:20:00.826841 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 153393us
W20260709 12:20:00.838466  6066 consensus_meta.cc:287] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.002s
W20260709 12:20:00.863577 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.980553 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 529168us
I20260709 12:20:00.827913  6362 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:00.861091 32660 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:00.924021 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:00.947216 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.951805 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.953359 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:00.983748 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:00.984643 31969 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:20:00.841948  6413 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.850102  6412 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:00.965819 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:00.860309  6411 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.841514  6160 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:00.968255  6327 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.202s	user 0.010s	sys 0.003s
W20260709 12:20:00.976962 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 178202us
W20260709 12:20:00.978551 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.923218  6184 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.848340  6403 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:00.967473  6407 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.970894  6314 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.988940  6402 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:00.861514  6408 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.990223 31965 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:00.976244  6436 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.951229  6332 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.871861  6166 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:00.858027  5895 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.131s
I20260709 12:20:00.981310 32623 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.008328 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:01.011027 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.012035 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:00.989282  6381 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.013355 32651 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 26. Current term is 27. Ops: []
W20260709 12:20:00.986846 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 127637us
I20260709 12:20:00.985351  6439 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.843501  6313 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.132s
I20260709 12:20:00.987251  6066 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.021133  6433 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:20:01.022130  6416 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
I20260709 12:20:01.021710  6389 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
W20260709 12:20:01.023128  6429 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:20:01.023401  6417 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.190s
W20260709 12:20:01.025573  6424 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:20:01.026003  6426 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:20:01.031605  6191 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.008750  6411 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.036804  6160 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.045298  6396 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.977608  6415 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.089615  6433 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:01.047416 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.019325  6407 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:00.986342  6432 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:01.073189 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 91969.9us
I20260709 12:20:01.061870 32114 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:01.015383 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.061100 31969 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:20:01.080722 32113 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:01.096472 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.019445  5949 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:00.996085  6412 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.150864 31965 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:01.121968 32117 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:01.125391 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.088068  6174 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:01.149626 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.121555  6395 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.158936 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.081573  6381 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:01.163452 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 146459us
I20260709 12:20:01.083858  6436 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:01.164690  6371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.031s	sys 0.045s
W20260709 12:20:01.176121 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56489.7us
I20260709 12:20:01.176146 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.089092  6439 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.161774  6417 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:01.181936 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.186782 31965 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:01.187111 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:01.172111  6371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.032s	sys 0.046s
W20260709 12:20:01.193758 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.173918  5949 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:01.180606  6403 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:01.139453  6443 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 12:20:01.204422 31965 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:20:01.209115 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.189285  6417 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.167053  6395 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.174420  6404 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:20:01.200570  6313 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.223680 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.225869 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.230743 32114 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:01.217231  5949 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.238688  6066 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.244886  6386 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:01.226835  6433 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.233877  6412 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:01.246896 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.229589 31969 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:01.250833 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.296847 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:01.297618 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.297755 31970 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:01.298097 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.255344 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.298842 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:01.218751  6417 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:01.271920 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.309342 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 118326us
I20260709 12:20:01.244071  6410 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.260478 32113 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 11. (term mismatch)
I20260709 12:20:01.233052  6408 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.298951 32112 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:01.257014 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.323822 32114 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:01.289397 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.324888 32116 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:01.238160  6444 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:20:01.248864  6411 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.349383 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:01.349702 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 190827us
I20260709 12:20:01.288822  6160 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.320559 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:01.350441 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:01.288583  6179 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.780s	user 0.014s	sys 0.000s
W20260709 12:20:01.328840  6425 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.523s	user 0.000s	sys 0.002s
I20260709 12:20:01.319362 32113 pending_rounds.cc:85] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 10
I20260709 12:20:01.312976 32013 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:01.326531 31969 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:01.349460 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.352780 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:01.318722  6451 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:20:01.238907  6415 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.355891 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.353521 32113 pending_rounds.cc:107] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 23.11
I20260709 12:20:01.240221  6439 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.361893 32113 raft_consensus.cc:2889] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:01.362620 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:01.363394 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.363610 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.364220 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.355154 32044 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:20:01.367545 32038 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:01.369091 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.377844 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.378044 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.378369 31966 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:01.378588 32041 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:01.250811  6313 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.374555 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.380306 32045 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:20:01.383574 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.384063 32047 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:01.384521 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.386945 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.387604 32042 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:01.384196 32114 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:01.388532 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:01.353518  6433 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.359359  6179 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.371292  6174 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:01.419358 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.403573 32045 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:01.427484 31968 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.428803  6314 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.431007  6314 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.431237 32045 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:01.409766 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.432989 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.433843 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.434486 32045 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:01.434845  6450 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 12:20:01.435463  6427 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.674s	user 0.003s	sys 0.000s
I20260709 12:20:01.435845 32045 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:01.436054 32658 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:20:01.436586 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 52141.3us
W20260709 12:20:01.437330  6430 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.586s	user 0.000s	sys 0.002s
I20260709 12:20:01.440923 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:01.441570 32659 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:01.446723 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.447446  6426 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:01.447890  6426 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:01.410030  6389 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.448964  6426 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 19.18, Last appended by leader: 18, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.368204  6457 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:01.368337  5895 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.450193  6457 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.404460  6449 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:20:01.450829  6457 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.450656  5895 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.452653  5949 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.416749  6432 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.453043  5895 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.453928 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.454023 32040 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:20:01.405752  6436 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.455228  6436 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:01.455583  6436 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:01.455884  6293 consensus_meta.cc:287] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.000s
I20260709 12:20:01.406643  6395 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.456481  6293 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.457386 32039 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:20:01.457700  6439 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:01.414455 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.458626  6381 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.362640  6160 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.460871  6160 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.461603 31967 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:01.435976 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:01.448889  6389 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.462674 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.463443 32042 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:20:01.464460 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:01.465022  6389 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.429193  6407 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.414969 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.466610  6179 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.466782 32041 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:01.427526  6174 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.469259 32010 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.455948  6436 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:01.472121  6447 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 12:20:01.472344 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.465055 32658 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:01.462746 32655 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:20:01.447598 32038 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:01.476078 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.477948 32653 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:01.389873 31969 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:01.481225 32117 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:01.401623 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.389809 32013 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:01.504575 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64312.5us
I20260709 12:20:01.505553 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.505817 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.503523 31970 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:01.507710 32116 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
W20260709 12:20:01.509133 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 119433us
W20260709 12:20:01.511288 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.514334 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:01.515097 32042 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:01.476681  6318 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.862s	user 0.009s	sys 0.000s
I20260709 12:20:01.518666 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.519037 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.368626  6371 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:01.521450  6371 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.521624 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.522068  6371 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.522468 31970 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:01.523696 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.387101  6166 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.525018 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.530000 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:01.479206  6403 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:01.545974  6403 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:01.515525  6438 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:01.550374  6438 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:01.432983  6413 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.550776  6438 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:01.551535 31963 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:01.551873 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:01.554168  6442 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:01.531864  6318 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.555941 32118 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:01.562566 32042 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:01.527118 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.520479 31968 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:01.482504  6426 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.530177 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.525389 31970 raft_consensus.cc:2376] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21. Re-sending same reply.
I20260709 12:20:01.531582 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.574409 32041 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:01.575129 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:01.575909 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 205001us
I20260709 12:20:01.533186 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.534530  5989 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.536921  5949 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:01.529724  6314 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:01.537968 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:01.580672 32038 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:01.538901 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.541275  6433 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:01.531157  6389 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:01.545496 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.524711  6293 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.579425 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:01.581859 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.582499 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43685.7us
I20260709 12:20:01.532169  5895 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:01.530963 32658 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:01.592267 31969 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:01.554520  6381 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:01.576836 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.589670 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.579214 32010 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.582043  6411 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:01.557453  6318 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.408401  6313 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.554303  6413 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.597477  6381 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.598009  6294 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.598485  6408 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:01.602373  6415 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:01.584529  6432 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:01.582459  6439 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.609203 31970 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:01.587596  6262 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:01.599128  6411 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.628993  6411 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.634557  6191 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:01.635037  6191 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:01.635548  6191 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:01.607612  6405 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:01.557626  6417 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.620142 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.624850 31970 raft_consensus.cc:606] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:01.596441  6338 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.607982  6381 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:01.625771 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46678.5us
I20260709 12:20:01.637794 31967 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:01.605336  6434 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:01.594583  5895 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.638203  6405 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.646085 31970 raft_consensus.cc:614] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:01.586678  5989 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.648043 31971 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:01.666041 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.615984  6432 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.603858  6160 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:20:01.602480  6412 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:20:01.669472 31966 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:01.622828  6184 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.639251  6262 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:01.725596  6262 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:01.669795  5895 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.669310  6405 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:01.668375  6338 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.616628  6294 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:01.731618 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.732154  6184 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.732635  6184 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.733417  6294 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.711400 31968 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:01.734874  6184 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.719868  6191 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.741509  6184 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.742031  6184 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.744251  6184 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.746098  6191 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.747622  6332 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.719095  6432 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:01.749981 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:01.749296  6457 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.756008  6457 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.712651  6455 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.666393  6434 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:01.756855  6455 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.757598  6434 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.713042  6407 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:01.759809  6338 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.664625  6255 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:01.715536  6417 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:01.716245 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45561.6us
I20260709 12:20:01.668025  6408 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:01.625020  6413 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 12:20:01.720785  6461 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 12:20:01.762434 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.723179  6016 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:01.724867 31972 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:20:01.669934  6433 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:20:01.758675  6434 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:01.764523 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.771699 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:01.764971  6408 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.772686  6408 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.772961 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.716959  6415 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:01.767473 31964 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:01.775262 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.778115  6399 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.778447 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.779052 31970 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:01.780428 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.762485  6455 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.722832  6438 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.783524 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.785454 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.785734 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.768477  6413 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:01.786051 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.791249 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.791867 31965 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.794034 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.747675  6262 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.782806  6399 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.867458  6332 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.868045  6332 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.868788 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.781459  6457 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.767066  6411 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.869415  6436 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.796489  6402 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:01.873238  6402 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:01.873618  6402 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:01.874790  6402 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.876205  6402 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.873762  6403 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.870213  6332 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.881431  6262 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.881589  6403 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:01.780843  6407 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:01.883172  6407 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:01.886293 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.889204 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.888805  6407 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:01.866097 31969 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:01.887125  6328 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.892504  6328 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.892941 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.893537 31969 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:20:01.777549  6415 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:01.894512  6415 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.897578 32046 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:20:01.897671 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.898073 32046 pending_rounds.cc:85] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 12
I20260709 12:20:01.898402 32046 pending_rounds.cc:107] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:20:01.898648 32046 raft_consensus.cc:2889] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:01.899210  6272 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.899701  6272 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.871327  5989 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.900851 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.871978  6411 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.869155  6371 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:01.838160 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.858101 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.902595 31970 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:01.903100  6272 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.872928  6408 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.903870  6371 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.882524 31972 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:01.777228  6338 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:01.905200  6338 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:01.905282  6408 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.883462  6436 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.905714  6338 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:01.884995 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 560217us
I20260709 12:20:01.906170 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:01.906744 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.906903 31972 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:01.907377 32013 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:01.907786 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.908519 31972 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:20:01.888242 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.884394  6438 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:01.859055 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63496.9us
I20260709 12:20:01.911330 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.912231 32119 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:20:01.886077  6383 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.889896  6441 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:01.913367 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.913832  6441 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.914271  6442 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:01.915627  6442 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:01.916074  6441 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:01.916498  6442 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.917022  6415 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:01.917598  6184 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:01.917758 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.918180 31968 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:01.914971  6399 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:20:01.918674 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.919327 32120 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:01.919767 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.915120  6191 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:20:01.920269 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.920917 32120 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 15.
I20260709 12:20:01.921507 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.922616 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.923189 32112 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:20:01.923979 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.918104  6184 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.924619 32112 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:01.924870  6184 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.925096  6415 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:01.925438 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.926004 32112 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.913849 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.926973 32115 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:01.928586  6328 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:01.886445 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 427342us
I20260709 12:20:01.930895 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.924631 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.932046 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.932471 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:01.932752 32119 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:01.933199 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.900940 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:01.900710  6405 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.910547 32117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:01.951843 32117 raft_consensus.cc:606] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:01.952399 32117 raft_consensus.cc:614] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:01.933277 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:01.863055 31965 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:01.934039 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.864661  6455 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.934463 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:01.935289 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:01.918337 32114 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.909956 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:01.953994 32660 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:01.890808 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:01.955054 31964 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:01.939466 32115 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:01.869379 31966 raft_consensus.cc:2376] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21. Re-sending same reply.
I20260709 12:20:01.921690 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.960500 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.961525 32119 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:01.941838 31970 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:01.934002 31969 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:01.960538 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:01.957912 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:01.964771 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54893.3us
I20260709 12:20:01.966814  6436 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:01.967351  6272 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:01.967749  6272 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.968106  6272 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:01.968761  6440 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:01.969198  6440 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:01.969547  6440 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.972550  6455 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:01.973052  6455 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.866888  6413 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:01.990829 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:01.991282 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:01.977690 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:01.965344 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:01.992383 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.993185 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.994186 32112 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:01.995262 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.954623  6415 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.954489  6016 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.973361  6293 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:01.957015  6327 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:01.973428  6455 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.955327  6441 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.003242 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:02.012871 32119 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:02.013345 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.013923 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:01.959559  6399 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.999238 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:02.009583 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:02.011598 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:01.969842  6407 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:01.998066  6191 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:02.017228 31964 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:02.016023 32112 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:20:02.018285 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:02.030369 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64979.6us
I20260709 12:20:02.031459 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:02.002799  6449 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:02.003722  6318 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:02.039055 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.010490  6327 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:02.021646  6191 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.054543  6191 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:02.019469  6255 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:02.060966 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:02.063710 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.067229 32114 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:02.072852 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.073771 32653 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:02.076529 31970 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:02.020377  6293 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:02.080611 32046 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:02.070926 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:02.088214 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89168.4us
I20260709 12:20:02.096109 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.100159 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.101150 31970 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:02.095018 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:02.107204 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.108328 32041 raft_consensus.cc:2368] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21.
I20260709 12:20:02.108566 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.110877 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.113576 31964 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:02.095296  6328 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:02.114566 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.114836 31964 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:02.115069  6328 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:02.116099 31964 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 23. Ops: []
I20260709 12:20:02.056409  6327 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:02.117831 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.118968 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.120074 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.122208 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.122900 32657 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:02.122346 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:02.079255  6443 raft_consensus.cc:2764] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:20:02.135500 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:02.135875 32653 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:02.126499 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 161200us
I20260709 12:20:02.023134  6399 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.137933  6399 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.138532  6399 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:02.143324  6407 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.130455 32043 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:02.104529  6411 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:02.126240 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:02.132397 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.020753  6441 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.135756 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:02.149401 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.130653 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.117190  6415 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.057091  6318 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.157907 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.155293 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.154814 32660 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:02.159740  6438 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:02.160902 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.161540 32042 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:02.162462 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.162575 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.163178 32039 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:02.117347  6458 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:20:02.116066  6328 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.166419  5989 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:02.104842  6393 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:20:02.172672 32657 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:02.173506 32043 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:02.146548  6436 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:02.176141 31965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:02.177103 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.176779  6293 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.121014  6413 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.114104  6272 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.184062  6440 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.186120  6184 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.187820  6327 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.263854 32117 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:02.264791  6184 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:02.188297  6318 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:02.127256  6255 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:02.190847 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.199867 31965 raft_consensus.cc:606] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:02.266646 31965 raft_consensus.cc:614] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:02.266949 32038 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:02.142973  6481 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:02.267732  6481 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:02.268060 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:02.268134 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.268594  6481 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.269958 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:02.210083 32114 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:02.209455 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.272090  6417 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:02.272606 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.273795 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.167641  5989 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:02.221865  6441 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.274477  5989 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:02.221066  6436 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:02.275600  5989 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.277139  5989 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:02.276625  6436 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.277534 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 881) took 1607 ms. Trace:
I20260709 12:20:02.277930 31963 rpcz_store.cc:276] 0709 12:20:00.670876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.671437 (+   561us) service_pool.cc:224] Handling call
0709 12:20:00.671849 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:01.552041 (+880192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:02.277293 (+725252us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400414454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:02.277474 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2022493568}
I20260709 12:20:02.227715  6405 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:02.206394  6426 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.002s
I20260709 12:20:02.229379 32042 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:02.281035 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.230167 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.231043 32045 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:02.232273 32118 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:02.184067  6191 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.228299  6432 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.250524 32115 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
W20260709 12:20:02.290822 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 187554us
I20260709 12:20:02.175064  6415 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.290035 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:02.262837  6272 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:02.250056  6440 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:02.304442  6455 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.305032  6393 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:02.305850 32047 raft_consensus.cc:2368] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 16.
I20260709 12:20:02.308583  6191 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:02.269747 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:02.311472 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 176025us
I20260709 12:20:02.247520  6399 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:02.249715  6338 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.286896 32114 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:20:02.288126 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:02.290380 32084 connection.cc:256] RPC call timeout handler was delayed by -2.074125978000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 12:20:02.255750 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3053) took 1023 ms. Trace:
I20260709 12:20:02.293107 32042 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:02.293624 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.311939 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.313382 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:02.313714 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:02.314667 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:02.272647  6417 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:02.264851  6327 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:02.318950 32084 connection.cc:256] RPC call timeout handler was delayed by -2.04262597800016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 12:20:02.319007 31967 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:02.322826 32084 connection.cc:256] RPC call timeout handler was delayed by -2.0273259780001354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:20:02.314680 32121 rpcz_store.cc:276] 0709 12:20:01.232300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.238264 (+  5964us) service_pool.cc:224] Handling call
0709 12:20:01.239558 (+  1294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:01.884135 (+644577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:01.884166 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:01.884646 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:20:02.007990 (+123344us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:02.008049 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:02.008119 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:02.255454 (+247335us) raft_consensus.cc:1685] finished
0709 12:20:02.255485 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:02.255667 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":644105,"raft.queue_time_us":644257,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":61,"thread_start_us":122931,"threads_started":1,"wal-append.queue_time_us":213548}
W20260709 12:20:02.323251 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9922259780000786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 12:20:02.323666 32084 connection.cc:256] RPC call timeout handler was delayed by -1.963425978000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 12:20:02.324102 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9499259780000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260709 12:20:02.324426 32084 connection.cc:256] RPC call timeout handler was delayed by -1.9445259780000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
W20260709 12:20:02.324725 32084 connection.cc:256] RPC call timeout handler was delayed by -1.6943259780000517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 12:20:02.325086 32084 connection.cc:256] RPC call timeout handler was delayed by -1.2551259780002511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2431s for a response
I20260709 12:20:02.320916 31971 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:02.302580  6066 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.325147 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:02.332927 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.333678 31965 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:02.334965 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.226329  5895 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.316911  6439 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:02.332789 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.342839 32657 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:02.333927  6216 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:02.323911  6433 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:02.368635 32119 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:02.338092  6272 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:02.331966  6399 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:02.368372 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:02.360872  6381 consensus_meta.cc:287] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.002s
I20260709 12:20:02.392702 32118 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:20:02.379534 32040 raft_consensus.cc:2368] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20.
I20260709 12:20:02.382014 31965 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:02.383879 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.359443  6439 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.356166  6449 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:02.401898 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.369778  6332 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:02.408918 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:02.332784  6408 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:20:02.417467 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.417906 31969 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:02.421954 31968 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:02.389791  6272 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.337488  6389 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:20:02.351534  6066 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.424621  5895 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:02.425863  6486 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:02.426087  6441 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:02.408747  6439 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:02.337931  6255 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.408651  6318 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:02.378849  6216 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:02.471388 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.472254 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.472416 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.479213 32038 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:02.509404 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.509843 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.462491  6272 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:02.403434  6381 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.518299 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.608438 32651 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:02.519011 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:02.608947 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.609338 32043 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 27.
I20260709 12:20:02.403080  6399 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.426949  6332 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:20:02.521528 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.624205 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.522964 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.525466 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.417725  6458 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.095s
I20260709 12:20:02.422283  6393 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.094s
I20260709 12:20:02.538044 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.654799 32044 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:02.564733 32119 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:02.654961 31967 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:02.538655 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 156984us
I20260709 12:20:02.656710  6399 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.657473 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:02.657765 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.538970  6432 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.610088 32655 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:02.610903 31966 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:02.605028 32660 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:02.616724 31968 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:02.525857 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:02.664566 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 138790us
W20260709 12:20:02.618755 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52347.8us
I20260709 12:20:02.626277 32045 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:02.516860  6433 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:20:02.629444  6482 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:20:02.629259 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.691296 32654 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:02.631276  6411 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.631449  6066 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.532946  6415 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:02.637701  6381 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.638669  6338 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:02.638808 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.639133  6216 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.639940  6455 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.624457  6318 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:02.640156  6484 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.638589 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.715860 32038 raft_consensus.cc:2393] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:02.644483  6459 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:02.644721  6332 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.647845 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.651132 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:02.528173  6487 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:02.718087 32043 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:02.515986  6262 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.542637 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.728844 32658 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:02.666812 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.733870 32660 raft_consensus.cc:2393] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:02.667511 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:02.671047  6441 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:02.672426 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.672904 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 2900) took 1160 ms. Trace:
I20260709 12:20:02.674414 31966 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:02.729483 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.741189 32044 rpcz_store.cc:276] 0709 12:20:01.512731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.514092 (+  1361us) service_pool.cc:224] Handling call
0709 12:20:02.672731 (+1158639us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540032a854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:02.672848 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2481482112}
I20260709 12:20:02.678431  6432 raft_consensus.cc:2764] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:20:02.674059 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.687979 32010 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:02.744211 32046 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:02.744367 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93350.1us
W20260709 12:20:02.744669 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56796.2us
I20260709 12:20:02.692816 32651 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:02.534770 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.413012  6417 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.538482  6413 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.694425 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.696251 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.699132 32011 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:02.701305 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.704499 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.749068 32112 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:02.749423 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:02.749951 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.750675 32121 raft_consensus.cc:2410] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:20:02.698331  6393 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:20:02.538295  6480 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.732513 32045 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:20:02.756760 32041 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:02.704906 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47494.2us
I20260709 12:20:02.752061 31972 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:02.687505  6433 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:20:02.754284 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55275.3us
I20260709 12:20:02.760499 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:02.729774  6442 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:02.761971  6388 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:20:02.707029  6415 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:20:02.757486 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:02.754653 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.743686  6441 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:02.727150  6484 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.755825  6458 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:02.700654  6411 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.756426  6455 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:02.773293  5949 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:02.774722 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.773483  6480 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.775868 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.785445  6293 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:02.835451  6293 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.837353 32112 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:02.777446 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.785970 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.773188  6449 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:02.785904  6272 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:02.773142  6459 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:20:02.773041  6388 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:02.787248  6439 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.773654 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:02.781971  6433 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.782423 32038 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:02.770467  6442 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:02.783591 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:02.794797 32118 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:02.810756 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.811698 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.814373  6318 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.775265 32042 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:02.781471 32046 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:02.766659 32654 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:02.825213 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:02.826812 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45134.4us
I20260709 12:20:02.740178  6262 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.860276 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.872814 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:02.831574 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.832403 31967 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:02.796214 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:02.838146 32121 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 12:20:02.875788 32045 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:02.838913 32116 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:02.854868 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81299.1us
I20260709 12:20:02.868073 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.890067 31964 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:02.890170 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:02.869051 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.869496 31972 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:02.873461 32043 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:02.874354 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:02.859854 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:02.873173 32042 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24. Ops: [23.18-23.18]
W20260709 12:20:02.874925 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47243.4us
I20260709 12:20:02.834805  6480 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.922617 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:02.894835 32042 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:20:02.896837 32038 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:02.825579  6441 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:02.925666 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:02.923161 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.824784  6458 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.924998 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.759634  6413 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:02.853884  5989 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:02.855499  6383 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:02.952556 32043 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:20:02.930745 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:02.932703 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44033.1us
I20260709 12:20:02.954208 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:02.933200 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:02.955636 32116 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:02.927421  6066 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:02.927409  6160 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:02.838263  6293 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.938736 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:02.854786  6402 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:02.858201  6433 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:02.941628 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:02.928125 31967 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:02.980103 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.980929 31967 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 23: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:02.854151  6439 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:02.885282  6484 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:02.983984 31969 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:02.843562  6272 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:02.844305  6388 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:02.987600 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:02.959074 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:02.926276  6436 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:02.921870  6455 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:02.985404 32118 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:02.987691 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:02.929808  6458 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:03.035202 31972 raft_consensus.cc:2368] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21.
W20260709 12:20:03.036598 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 77596us
I20260709 12:20:03.037420 32011 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:02.936894  6184 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.986948 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:03.045576 31967 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:02.941865  5989 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:03.061036 31972 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:03.062662 31972 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:20:03.064424 32041 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:20:03.031729  6066 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:02.976450  6160 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.983369  6399 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:03.046979 31967 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:02.958639 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 85939.7us
I20260709 12:20:02.986431  6381 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:03.048566 32011 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:03.066140  6066 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:03.051819  6272 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:03.042851 31964 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:03.066747 31963 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:02.867298  6442 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.078462 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:03.049789  6426 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:03.117467 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:03.109098 32116 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:20:03.048836  6436 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:20:03.127132 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 78614.5us
I20260709 12:20:03.131482 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:03.132375 32655 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 18: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:03.074532  6160 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:20:03.143878 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 449903us
I20260709 12:20:03.106426  6066 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:03.069096  6184 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.135005  6381 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:03.155256 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:03.156539 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:03.159113 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:03.134099  6399 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:03.159386 32655 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.159947 32041 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:03.159163 32011 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:03.138509  6272 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:03.161104  6479 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.015s
I20260709 12:20:03.161356  6184 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:03.144166  6016 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:20:03.181622 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.189098 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:03.162142  6449 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:03.162604  6442 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:03.189822  6479 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.007s	sys 0.016s
I20260709 12:20:03.210551 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:03.211419 32654 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:03.139104  6388 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:03.217401 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 460575us
I20260709 12:20:03.193291  6184 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:03.154726  5989 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.195752  6426 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:03.221593 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:03.224668 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:03.200562  6480 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:03.201822  6016 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:03.168149  6455 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.216939  6411 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:03.224614  6403 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.016s	sys 0.030s
W20260709 12:20:03.235208 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.002s	sys 0.036s
W20260709 12:20:03.247359 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.002s	sys 0.036s
I20260709 12:20:03.242735 32654 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:03.229734  6371 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:03.251423 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:03.211814  6402 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:20:03.257570 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:03.259591 31963 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:03.231359  6184 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:03.197157  6436 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.236266  6262 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:03.233908  6426 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:03.237044  6407 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:03.284351 32044 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:20:03.285211 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 331075us
I20260709 12:20:03.243237  6388 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:03.247292  6403 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.017s	sys 0.031s
I20260709 12:20:03.231504  6405 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:20:03.299343 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:03.304332  6499 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 12:20:03.316321 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 113763us
W20260709 12:20:03.310981  6496 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 12:20:03.313115  6495 log.cc:927] Time spent T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:20:03.305739  6407 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.238615  6399 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.324488  6318 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:03.326458  6393 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:03.246237  6272 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.360750  6438 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:03.361478  6426 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:03.362730  6500 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 12:20:03.363121  6509 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:20:03.375438 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:03.377154 32119 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:03.365893 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2978) took 1245 ms. Trace:
I20260709 12:20:03.338486  6407 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.368999 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:03.369697  6438 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:03.380028 32119 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:03.383853 32119 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:03.380239 32116 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:03.378763 32115 rpcz_store.cc:276] 0709 12:20:02.120505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.658726 (+538221us) service_pool.cc:224] Handling call
0709 12:20:02.659019 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:02.659338 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:02.659365 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:02.659857 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:20:03.247861 (+588004us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:03.247923 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:03.248017 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:03.365682 (+117665us) raft_consensus.cc:1685] finished
0709 12:20:03.365703 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:03.365839 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33946,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6945,"thread_start_us":556653,"threads_started":1,"wal-append.queue_time_us":694142}
I20260709 12:20:03.389528 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:03.386955  6434 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:20:03.391281  6469 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:03.391760  6469 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:03.392613  6469 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.393875  6469 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:03.394232  6459 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:03.395529 32084 leader_election.cc:400] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21.
I20260709 12:20:03.386264 31938 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:03.384691 32119 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.348559  6440 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:20:03.343422  6016 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.396032  6393 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:03.398264  6434 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.400115  6371 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:03.403746  6411 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:03.436116  6403 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:03.426075 32655 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
W20260709 12:20:03.427016 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89991.6us
I20260709 12:20:03.472066 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:03.475651  6530 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:03.476209  6530 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:03.477087  6530 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.395288  6328 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:20:03.487655 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:03.403719  6293 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:20:03.474148 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:03.365456  6184 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.387012  6216 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:20:03.395080  6318 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:20:03.492919 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 95058.6us
I20260709 12:20:03.441617  6411 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.469782  6403 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:03.502009 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:03.511706  6441 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.003s
I20260709 12:20:03.512260  6441 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.441792  6434 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.501518  6533 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:03.506033  6411 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:03.534924 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:03.469789  6438 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.526997  6533 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.576900  6532 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.577607  6434 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:03.577520  6532 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.578850  6540 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:03.582199 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:03.592975 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:03.593432 32114 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:20:03.593533 32045 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:03.596360 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:03.595455 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:03.544436  6332 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:03.575347  6438 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:03.580295  6533 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.607926  6532 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:03.601651  6528 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:03.607292  6507 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:03.523250  6371 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.598207  6415 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.579958  6539 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:20:03.625814  6332 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:20:03.637581 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:03.644497 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:03.641872 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 140007us
I20260709 12:20:03.527019  6393 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.637553  6540 consensus_queue.cc:1243] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:03.646669  6538 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:20:03.638492  6522 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:20:03.647404  6548 consensus_queue.cc:1059] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:20:03.647323  6551 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:03.644779  6216 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:03.652405  6539 consensus_queue.cc:1243] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:03.660243 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:03.626259  6528 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.637715  6507 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.698845  6507 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:03.699244 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:03.653851  6537 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:20:03.600931  6403 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.706270  6332 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.710601  6413 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:03.716328  6551 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:03.716475  6328 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:03.717088  6328 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:03.717397  6551 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.717991  6328 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:03.720643 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.716368 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2992) took 1026 ms. Trace:
I20260709 12:20:03.678220 32655 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:03.682905 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:03.683939  6548 raft_consensus.cc:3055] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:03.684068 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:03.758541 32654 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:03.685274 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2908) took 1151 ms. Trace:
I20260709 12:20:03.678794  6216 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:03.671896  6538 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:03.765650 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:03.692668 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:03.655776  6415 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.694213 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:03.770232 31963 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:03.695428 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:03.696090 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:03.696966 31971 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:03.697796 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:03.698137 32040 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:03.725330 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.724133 31965 rpcz_store.cc:276] 0709 12:20:02.690085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.727408 (+ 37323us) service_pool.cc:224] Handling call
0709 12:20:02.755347 (+ 27939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:02.755579 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:03.194032 (+438453us) log.cc:844] Serialized 14 byte log entry
0709 12:20:03.715941 (+521909us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b18003e8d80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287
0709 12:20:03.716035 (+    94us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:03.716082 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:03.716126 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:03.716188 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:03.716311 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":531795,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":41226,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7658,"spinlock_wait_cycles":8546560,"thread_start_us":517397,"threads_started":2}
I20260709 12:20:03.758975 32046 rpcz_store.cc:276] 0709 12:20:02.533511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.874088 (+340577us) service_pool.cc:224] Handling call
0709 12:20:02.874404 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:02.890138 (+ 15734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:02.890167 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:02.927117 (+ 36950us) log.cc:844] Serialized 26 byte log entry
0709 12:20:03.277512 (+350395us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:03.277573 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:03.277679 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:03.684947 (+407268us) raft_consensus.cc:1685] finished
0709 12:20:03.684982 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:03.685195 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16088,"raft.queue_time_us":52227,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8845,"thread_start_us":241638,"threads_started":1,"wal-append.queue_time_us":524537}
I20260709 12:20:03.691831 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:03.692813 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:03.767520 32660 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:03.775697 31964 raft_consensus.cc:2368] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21.
I20260709 12:20:03.768338 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:03.776221 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:03.776888 32120 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:20:03.784895 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:03.723002  6528 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:03.799088 32084 leader_election.cc:400] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21.
I20260709 12:20:03.799715  6541 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:03.800097 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:03.757639  6548 raft_consensus.cc:740] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:03.806305 31963 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:20:03.797072  6408 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:03.819208  6553 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:03.820402  6538 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:03.809165  6434 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:03.807089  6485 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.015s	sys 0.004s
I20260709 12:20:03.868628  6452 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:03.868664  6485 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.200s	user 0.015s	sys 0.004s
I20260709 12:20:03.823755  6434 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:03.898546  6449 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.001s
W20260709 12:20:03.965916  6383 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 1.003s	user 0.008s	sys 0.003s
W20260709 12:20:03.966881  6433 consensus_meta.cc:287] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.978s	user 0.012s	sys 0.000s
I20260709 12:20:03.936888  6449 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:03.904750  6293 consensus_queue.cc:1243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:20:03.975986 32654 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 12:20:03.984864 32654 pending_rounds.cc:85] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 17
I20260709 12:20:03.985214 32654 pending_rounds.cc:107] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 12:20:03.985517 32654 raft_consensus.cc:2889] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:03.988097 31966 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 12:20:03.977329  6552 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:20:03.914628  6539 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:03.993687 32113 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:03.824134  6216 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:03.994530  6327 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.017982  6327 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:04.031018  6442 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.003s
I20260709 12:20:04.022955  6179 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.070094  6179 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.072631  6179 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.034991  6449 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:04.035692  6415 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:04.035928  6066 consensus_meta.cc:287] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.873s	user 0.006s	sys 0.006s
I20260709 12:20:03.996877  6433 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:20:04.022505  6498 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:04.038756 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.041204  6408 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:04.043947  6458 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.972s	user 0.011s	sys 0.000s
I20260709 12:20:03.999374  6272 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.995683  6539 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:04.061975  6441 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:04.065068  6160 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.922s	user 0.012s	sys 0.000s
I20260709 12:20:04.018227  6545 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:20:04.076529 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:04.137902 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 337833us
I20260709 12:20:04.090171  6541 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:04.104323 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 286298us
I20260709 12:20:04.118813  6554 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:04.146835 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.147716 31972 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:04.000972  6383 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.129437 32121 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:20:04.130728 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.052270  6533 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.131958 32041 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:04.133121 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 333773us
I20260709 12:20:04.139389 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.164451 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.165748 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:03.904282  6452 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.168193  6452 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.168751  6529 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:04.143129 31965 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:03.907496  6548 consensus_queue.cc:260] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.150358 32084 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.154196 31972 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:20:04.170629 31972 pending_rounds.cc:85] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:20:04.170699  6548 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.170940 31972 pending_rounds.cc:107] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:20:04.171346 31972 raft_consensus.cc:2889] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:04.171345 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.172608 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.173758 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.174964  6529 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:04.184789  6436 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:04.186511 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.187728  6436 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:04.189711 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.210474  6436 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.212168  6436 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:04.212968  6436 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.218019 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.218942 32654 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:04.220880 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:04.222107  6332 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:04.222642  6332 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:04.223152  6332 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:04.230473  6436 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.232417 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.238843  6436 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:04.239948  6436 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:04.241283  6436 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.135234  6458 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.103793  6449 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.127935  6066 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.266837 32652 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:04.267151 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.134753  6272 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.135408  6498 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.140636  6160 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.252651  6332 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.280759 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.155673  6554 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.157738  6383 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.277406  6545 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:04.278139  6549 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.274030  6561 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:04.285463  6507 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
I20260709 12:20:04.281988  6408 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.279743  6441 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.281603  6551 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:20:04.319437 32115 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:04.306813  6433 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.317236  6434 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.317538  6545 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:04.359601 32039 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:04.363457 32046 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:04.364346 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.266330  6539 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.367595 32045 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:20:04.333156  6408 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:04.359756  6438 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.742s	user 0.011s	sys 0.000s
I20260709 12:20:04.376438  6554 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:04.377312 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.379293 32042 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:20:04.380733 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:04.381489  6528 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.575s	user 0.006s	sys 0.002s
I20260709 12:20:04.381538 32038 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:04.383548  6272 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:04.384733 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 211021us
I20260709 12:20:04.385505 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.385285  6545 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.386036 32045 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:04.386766 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.378194 32044 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:04.371061 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:04.361987  6411 consensus_meta.cc:287] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.823s	user 0.010s	sys 0.002s
I20260709 12:20:04.388513  6411 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.389768  6402 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:04.361105  6407 consensus_meta.cc:287] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.961s	user 0.009s	sys 0.004s
I20260709 12:20:04.373615  6525 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:04.365953  6452 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:04.366742  6388 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 1.053s	user 0.013s	sys 0.000s
I20260709 12:20:04.399287  6332 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.382117  6528 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.398118  6562 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.318723  6507 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.366636  6538 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:04.395620 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.398705  6452 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.393994  6539 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:04.412674 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.300252  6541 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.414820 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:04.419452 32653 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 28: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:04.396759  6272 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:04.403903  6554 raft_consensus.cc:2764] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:04.390293  6402 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.427858 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.409991  6411 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.392920  6408 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.435637 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:04.415539  6441 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.437927 32653 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:04.439821  6573 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:04.440768  6570 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:04.441365  6570 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:04.444124  6556 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.404s	user 0.002s	sys 0.000s
I20260709 12:20:04.444306  6570 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.446367 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.417157  6160 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.401042  6407 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.445230  6549 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.447214 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.428694  6533 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:04.434777 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 190986us
I20260709 12:20:04.405939  6525 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.434095  6272 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.453785  6272 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.455495  6498 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.456048  6458 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.457026 32651 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:04.458068  6402 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.458760  6449 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.460289  6160 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:04.460793  6160 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.462303 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.463446  6452 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.464020  6452 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:04.466668  6407 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.467423 32047 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:20:04.467967 32113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47764 (request call id 3073) took 2552 ms (client timeout 1543 ms). Trace:
I20260709 12:20:04.468426  6066 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:04.468395 32113 rpcz_store.cc:269] 0709 12:20:01.915890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.919559 (+  3669us) service_pool.cc:224] Handling call
0709 12:20:04.467783 (+2548224us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54002b66d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:04.467881 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4767886592}
I20260709 12:20:04.470033  6383 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.472086  6408 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.473896  6528 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.474498 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3090) took 1577 ms. Trace:
I20260709 12:20:04.474859 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.479028  6160 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.482621  6403 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:04.483278  6532 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.483886  6532 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.486106 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:04.486297  6532 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:04.486939 32652 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:04.453785  6525 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.489848 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:04.490294 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.491441  6442 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:04.492090  6442 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.493397 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:04.494010 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.494728 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.495215 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:04.495860 32121 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.496186  6484 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:04.497792  6525 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.495141 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.498323  6525 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.502823 32651 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:20:04.503250 32114 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:04.503917 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.504752 32116 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
W20260709 12:20:04.507869 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 60759.8us
I20260709 12:20:04.506705  6525 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.509421  6528 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:04.513425 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.509923  6528 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.518293  6528 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:04.520380 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.521359 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.518519  6553 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:04.530987 32653 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:04.449916  6573 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:04.474879 32118 rpcz_store.cc:276] 0709 12:20:02.897232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.070717 (+173485us) service_pool.cc:224] Handling call
0709 12:20:03.071075 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:03.071315 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:03.281870 (+210555us) log.cc:844] Serialized 14 byte log entry
0709 12:20:04.474022 (+1192152us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b18000f8740. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287
0709 12:20:04.474107 (+    85us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:04.474157 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:04.474199 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.474277 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.474410 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3689,"prepare.queue_time_us":266806,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":141646,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":28531,"spinlock_wait_cycles":2770688,"thread_start_us":570598,"threads_started":3,"wal-append.queue_time_us":326557}
I20260709 12:20:04.552096 32652 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:20:04.553303 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44502.8us
I20260709 12:20:04.556280 31936 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:04.557570 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:04.558216 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.515295  6160 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.558873 32112 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:20:04.559126 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 278422us
I20260709 12:20:04.516096  6442 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.520738  6272 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.567638  6573 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.568756  6580 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.570147 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.557646  6554 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:20:04.562871 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.569713  6549 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:04.562493  6578 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:04.589036 32121 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:04.592628 32084 leader_election.cc:400] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 16.
I20260709 12:20:04.592991 32084 leader_election.cc:403] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:04.593477 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:04.596236 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.598743 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.580896  6408 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:20:04.602362 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.565762  6160 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.569343  6272 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.589965  6554 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:04.614825  6579 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:04.615793  6574 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:20:04.618810  5989 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:04.626955  6575 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:04.608137 32112 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 28.
I20260709 12:20:04.576964  6525 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:20:04.611989 32084 leader_election.cc:400] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20.
I20260709 12:20:04.572726  6560 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:20:04.598490  6578 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:04.570926  6580 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.652302  6580 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:04.653905 31966 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 2997) took 1787 ms (client timeout 1745 ms). Trace:
W20260709 12:20:04.654803 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43552.2us
I20260709 12:20:04.656790 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:04.654397 31966 rpcz_store.cc:269] 0709 12:20:02.866649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.868792 (+  2143us) service_pool.cc:224] Handling call
0709 12:20:04.653634 (+1784842us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540034dd54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:04.653796 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520342400}
I20260709 12:20:04.669363 32115 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:20:04.669938 32115 pending_rounds.cc:85] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 15
I20260709 12:20:04.670238 32115 pending_rounds.cc:107] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:20:04.670552 32115 raft_consensus.cc:2889] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:04.639084  6408 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:04.633100  6332 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:20:04.662959  6480 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.664793  6438 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:04.714095 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57375.5us
W20260709 12:20:04.716949 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 100220us
I20260709 12:20:04.682266  6579 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:04.718123 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.657987  6545 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:20:04.665656  6433 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:20:04.665362  6529 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:20:04.741698  6417 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.741936  6441 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.745234 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 2978) took 2367 ms. Trace:
I20260709 12:20:04.745620 31970 rpcz_store.cc:276] 0709 12:20:02.377856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.377981 (+   125us) service_pool.cc:224] Handling call
0709 12:20:02.378396 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:02.399800 (+ 21404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:02.399850 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:02.400480 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:20:02.706759 (+306279us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:03.083059 (+376300us) log.cc:844] Serialized 14 byte log entry
0709 12:20:03.097519 (+ 14460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:03.097631 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:04.723990 (+1626359us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b2400177308. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f2289120fb0 00007f228bd97488 00007f228bd97be8 00007f228bd6f398 00007f228bd6a76a 00007f228bd69bb6
0709 12:20:04.744899 (+ 20909us) raft_consensus.cc:1685] finished
0709 12:20:04.745010 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.745152 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38730,"prepare.queue_time_us":575084,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":14965,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":29860,"spinlock_wait_cycles":26777472,"thread_start_us":559646,"threads_started":2,"wal-append.queue_time_us":539544}
I20260709 12:20:04.717001  6480 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.748703  6405 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.749220  6405 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.749855  6480 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.722676 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.725286 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:04.726046 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.752123 32039 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:04.726718 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.753429 32038 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:04.727432 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:04.727916  6569 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.296s	user 0.004s	sys 0.000s
W20260709 12:20:04.728865 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.730356  6578 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.735548  6272 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.738505  6545 consensus_queue.cc:1243] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:04.739199  6160 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.739971  6579 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.746942  6216 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:04.762192  6408 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.723367 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:04.751546  6389 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:20:04.754105 32042 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:04.759392 32013 connection.cc:256] RPC call timeout handler was delayed by -1.6000994109999738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
I20260709 12:20:04.766626 32046 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.755916 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.773480 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.774142 32044 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:04.775326 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.776058 32044 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:04.780247 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.780432 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.781055 31969 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:04.781020 31964 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:04.785625 32042 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:04.786586 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.789707 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.751511  6405 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.791411 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.811493 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.797272 32046 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:04.798710 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.799326  6389 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.800875 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:04.768150  6332 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.803606 31964 raft_consensus.cc:2368] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19.
I20260709 12:20:04.803552 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.820771 31968 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:04.804543 31970 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:04.822698 31966 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:20:04.801110  6455 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.004s	sys 0.009s
I20260709 12:20:04.827950 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.800922  6459 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.807713  6402 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.835961 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.841602 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.842396 31964 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:20:04.842957 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:04.843797 31968 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 12:20:04.825948  6455 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.189s	user 0.004s	sys 0.010s
I20260709 12:20:04.845577 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.822089  6381 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.849337 32657 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:20:04.850807  6381 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.853001  6381 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.853096  6575 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 14. Current term is 16.
I20260709 12:20:04.850822  6481 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.854104  6481 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.855484  6441 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:04.855880  6441 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:04.856158  6481 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.856729  6441 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.858636 31964 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:04.859395  6160 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.859925  6160 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.862008 32119 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.864202 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.826079  6389 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.868414  6389 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.826514  6332 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.871412  6332 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.875331 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.876256 32653 raft_consensus.cc:2410] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:20:04.847422  6402 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.877123  6507 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:04.878443 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.878682  6440 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:04.879125  6440 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:04.874176  6564 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.602s	user 0.003s	sys 0.000s
I20260709 12:20:04.879999  6440 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.844445  6459 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:04.844904  6580 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:04.901341  6580 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.901664 32653 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
W20260709 12:20:04.880632  6588 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:20:04.901872  6525 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:04.902599  6459 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:04.902666  6525 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:04.903561 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2716) took 1209 ms. Trace:
I20260709 12:20:04.903937 32041 rpcz_store.cc:276] 0709 12:20:03.694387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.694524 (+   137us) service_pool.cc:224] Handling call
0709 12:20:03.694828 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.156959 (+462131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.903332 (+746373us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b540013ced4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:04.903495 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998203648}
I20260709 12:20:04.904448 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.905578  6438 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.863655 32657 raft_consensus.cc:1240] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:04.900691  6440 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:04.882344 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:04.906874 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:04.907699 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:04.858135  6441 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.862164  6160 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:04.916760 32119 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:04.917565 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2986) took 1548 ms. Trace:
I20260709 12:20:04.916230 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:04.918025 32119 rpcz_store.cc:276] 0709 12:20:03.369196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.377964 (+  8768us) service_pool.cc:224] Handling call
0709 12:20:03.378218 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.861566 (+1483348us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54004d01a0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c144a1d 00007f228c12b1a4 00007f228c1ad447 00007f228c1c5fe7 00007f228c1c77de 00007f228c1d1b49 00007f228c1d2416 00007f228c1ced8f 00007f228c1ce83e 00007f2291193419
0709 12:20:04.917270 (+ 55704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.917501 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333374720}
I20260709 12:20:04.909906  6415 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.910440  6546 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:04.878248  6402 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:04.934960 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:04.935400 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:04.935750 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.906389  6438 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:04.933924  6415 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.938024 32113 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:04.941228 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.944566 32113 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:04.947279 32113 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:04.948827 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.949613 32115 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:04.951223 32114 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:04.952522 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:04.953220 32112 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 28: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:04.953459 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:04.900367  6528 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:04.956036 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.954873 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.958909 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.959687 32112 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:04.961694 32119 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:04.964710 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.965628 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:04.967020 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.934670  6546 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:04.940867  6438 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.978411 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:04.980640 32120 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:04.940068  6540 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:04.982573 32010 leader_election.cc:400] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19.
I20260709 12:20:04.983927 32010 leader_election.cc:403] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:04.984911 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.947764  6332 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:04.945411  6584 consensus_queue.cc:1059] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:20:04.993712 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:04.988296 32043 raft_consensus.cc:1240] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:04.956230  6160 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:05.000295 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.002345 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:04.964170  6399 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:04.988339  6438 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:04.990865  6332 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.018038 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.025568 31964 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:20:05.026607 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.027205 31964 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:05.028132 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.028718 31964 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:04.997624  6160 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.007647  6525 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:04.984043  6570 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.996181  6408 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:04.993090  6442 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:04.987985  6191 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19.
I20260709 12:20:05.005661  6399 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.023931  6529 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:04.994500  6540 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.005172  6016 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:20:05.034184 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.035326 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.012578  6591 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.060098 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.010227  6338 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:05.024159  6579 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:05.043228  6442 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:05.065551  6442 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:05.031394  6332 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:05.070950  6596 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:20:05.036749  6546 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.072964  6546 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:05.081058  6580 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.086162 32119 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:20:05.049794  6399 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:05.081897  6332 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.035668  6525 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.035735  6160 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.059844 31968 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:20:05.040776  6408 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.062875 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.062625  6591 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.094128 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.101920 32041 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:05.097313  6525 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.097227 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.099982 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.053608 31969 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:05.107764  6540 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.105388 32115 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:05.109372 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.104672  6408 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:05.122989 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.124815 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.125473 31967 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:05.127763  6591 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.129558  6327 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.130596  6528 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.130715 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:05.131619 32652 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 28: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:05.126263 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:05.137914 32013 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.086230  6579 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.140185 31966 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:20:05.169945 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.170905 31966 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:05.171936 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.139957  6272 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:05.146296 32041 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:05.151449 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.174746 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.175616 32118 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:20:05.175082 32113 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:20:05.175872 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.176575 32118 pending_rounds.cc:85] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 10
I20260709 12:20:05.176836 32118 pending_rounds.cc:107] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:20:05.167893 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.176870 32119 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 28: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:05.152498 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.177807 32118 raft_consensus.cc:2889] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:05.177896 32115 raft_consensus.cc:2410] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 12:20:05.180553 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 193668us
I20260709 12:20:05.180821 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.181677 32116 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:20:05.182168 32116 pending_rounds.cc:85] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 10
I20260709 12:20:05.182477 32116 pending_rounds.cc:107] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:20:05.182789 32116 raft_consensus.cc:2889] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:05.185060 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77527.5us
I20260709 12:20:05.089186  6442 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.141423  6191 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 12:20:05.194634 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 135950us
I20260709 12:20:05.194340 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.200476 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.203210 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 926) took 1754 ms. Trace:
I20260709 12:20:05.203599 32047 rpcz_store.cc:276] 0709 12:20:03.448404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.448528 (+   124us) service_pool.cc:224] Handling call
0709 12:20:05.202997 (+1754469us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b5400032fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:05.203148 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2244991232}
I20260709 12:20:05.204538  6191 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:05.184587  6338 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:05.206852  6179 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:05.179708  6607 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:05.207230  6179 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.174983  6381 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:20:05.199076  6455 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:20:05.209426  6403 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.197530  6530 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:05.225184  6179 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:05.217716  6338 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.237291  6396 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:05.238384  6399 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.258883  6442 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.259420  6399 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.262532  6396 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.268456 32038 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:05.284103 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.284798 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:05.289542  6393 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.020s	sys 0.028s
W20260709 12:20:05.290123 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 95912.9us
W20260709 12:20:05.290328  6393 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.020s	sys 0.028s
I20260709 12:20:05.315965  6520 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:05.296533  6540 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.290927 32047 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:05.329046  6540 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.331257  6469 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:05.332288  6540 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.329008  6520 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.334231 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.335316  6469 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:05.335767  6469 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.335814  6520 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.336383  6469 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:05.295907  6441 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:05.296706  6575 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.338182  6575 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:05.317446 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.285238  6408 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.341204 32655 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:05.341531 31971 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 24. Current term is 25. Ops: []
W20260709 12:20:05.342238 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.292886  6160 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.345175  6441 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:05.346237  6441 consensus_queue.cc:1243] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:05.346817  6160 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.347214  6520 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:05.347847  6520 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:05.348290  6441 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:05.350090  6575 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.284307  6396 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.349057  6520 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.355266 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:05.356683  6520 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:05.283644  6440 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:05.377642  6440 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.385854 32047 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.233028  6403 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.387372 32042 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:05.388357  6403 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.388728 32042 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:05.389230  6559 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.354s	user 0.000s	sys 0.003s
I20260709 12:20:05.384182  6528 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.390064  6528 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.392081 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.392231  6528 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.393008 32042 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:20:05.393162 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.393266 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.394012 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 2987) took 1994 ms. Trace:
I20260709 12:20:05.234872  6530 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.394271 32041 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 28: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:20:05.403842 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 261152us
I20260709 12:20:05.394042 32046 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:05.405575 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.406217 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.406381 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.406992 32046 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 17.
I20260709 12:20:05.396651 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:05.411870 32121 raft_consensus.cc:2410] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:20:05.394418 32117 rpcz_store.cc:276] 0709 12:20:03.400036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.400303 (+   267us) service_pool.cc:224] Handling call
0709 12:20:03.410904 (+ 10601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:03.496823 (+ 85919us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:20:03.497212 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:20:03.848321 (+351109us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:03.863598 (+ 15277us) log.cc:844] Serialized 26 byte log entry
0709 12:20:03.863752 (+   154us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:03.863805 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:03.863873 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:04.750189 (+886316us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b18003acd80. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:20:05.388668 (+638479us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:05.393812 (+  5144us) raft_consensus.cc:1685] finished
0709 12:20:05.393832 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.393959 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62699,"prepare.queue_time_us":1727,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":58,"raft.queue_time_us":300996,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":2921,"spinlock_wait_cycles":109495936,"thread_start_us":290123,"threads_started":2,"wal-append.queue_time_us":402809}
I20260709 12:20:05.396224 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.414830 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:05.416482 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 235727us
I20260709 12:20:05.417740 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.418200 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.414589 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:05.421638 32042 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:05.426138 32047 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:05.424450 32652 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:05.411621  6328 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:05.436368  6408 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.358289  6441 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.461508 32660 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:05.422813  6528 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:05.469079 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:05.416226  6575 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:20:05.416007  6540 consensus_queue.cc:1059] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:20:05.431588  6332 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.420629  6546 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 12:20:05.480684 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70946.5us
I20260709 12:20:05.478818 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:05.489516  6525 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.475574  6327 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:20:05.460573  6328 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.498202  6179 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.529956  6328 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.492910  6438 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.475368  6528 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:05.531311  6528 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.532037  6438 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.534634 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:05.477098  6440 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.499432  6396 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:05.502475 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:05.536514 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:05.536859 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.507288  6530 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.481370  6272 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:05.514304  6623 raft_consensus.cc:3055] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:05.537544  6396 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.537737  6623 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:05.538666  6623 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:05.546373 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.521413  6441 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.522547 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:05.528746  6179 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:05.567619 32084 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.535s (SENT) [suppressed 7 similar messages]
W20260709 12:20:05.568922 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 368486us
I20260709 12:20:05.569906 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:05.561240 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 143592us
I20260709 12:20:05.571094  6179 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.591091 32043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2717) took 1896 ms. Trace:
I20260709 12:20:05.591495 32043 rpcz_store.cc:276] 0709 12:20:03.694680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.694794 (+   114us) service_pool.cc:224] Handling call
0709 12:20:03.695039 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.995713 (+1300674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.590858 (+595145us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540003a054. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:05.591036 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973595392}
I20260709 12:20:05.574363  6540 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:05.609643 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.606586  6617 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:05.613118  6617 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.613562  6617 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.613114  6619 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:20:05.616055  6619 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:05.616511  6619 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:05.617715  6579 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.612579  6550 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:05.619648  6550 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:05.620812  6550 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.621855  6481 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:05.622351  6481 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:05.623172  6570 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:05.623582  6570 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:05.623723  6550 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.624445  6570 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.617583  6619 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.610899 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.635596 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.558787  6525 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:05.641014 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.641460  6520 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:05.678718  6432 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.679243  6432 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.681342  6432 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.681826  6432 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.642956  6411 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:05.682721  6411 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.683573 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1229) took 1921 ms. Trace:
I20260709 12:20:05.683971 32655 rpcz_store.cc:276] 0709 12:20:03.762047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.768554 (+  6507us) service_pool.cc:224] Handling call
0709 12:20:03.768858 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:05.341802 (+1572944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.683358 (+341556us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400526cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:05.683517 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3615399680}
I20260709 12:20:05.684898  6411 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.627077  6540 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.687845  6540 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.690204 31968 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 22. Ops: [20.12-20.12]
I20260709 12:20:05.690572 31964 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:20:05.691807 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.693274 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:05.694516 32044 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:05.694705 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.695451 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.695724 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.696458 32039 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:05.697573 31968 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:05.656193  6619 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:05.698598 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.702872 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.703465 32038 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:05.703859  6608 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:05.704468  6608 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:05.704794  6613 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 12:20:05.704927  6608 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:05.708997 32045 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:20:05.709457 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.710842 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:05.712019  6403 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:05.712512  6403 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:05.713814 31972 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:05.714695 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.715284 32045 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.721028  6272 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:05.721498  6272 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.717866  6481 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.723886  6481 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.723654  6608 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.727774  6608 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:05.728615  6623 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:05.730386 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:05.730976 32118 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.733119 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:05.733668 32653 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:05.745100 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3005) took 1606 ms. Trace:
I20260709 12:20:05.630214  6570 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.713387  6403 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.746773 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:05.748023 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.751103  6608 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.752099 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.752825 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.682252  6432 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.754750 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.771341 32116 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:20:05.771375 32117 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:20:05.771745 32112 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:05.657529  6415 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.662743 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2702) took 1969 ms. Trace:
I20260709 12:20:05.695580 31970 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 19: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:05.595420 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.765206 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.772722 32653 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:05.773425 31966 raft_consensus.cc:2368] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24.
I20260709 12:20:05.772766 31965 rpcz_store.cc:276] 0709 12:20:03.692777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.773972 (+ 81195us) service_pool.cc:224] Handling call
0709 12:20:03.796474 (+ 22502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.170163 (+373689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.662375 (+1492212us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400068454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:05.662586 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782690176}
I20260709 12:20:05.773941 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.774489 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:05.764775 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 155206us
I20260709 12:20:05.765776 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:05.775472 31969 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.775954 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.766433 32118 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:05.757999 31963 rpcz_store.cc:276] 0709 12:20:04.139016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.269117 (+130101us) service_pool.cc:224] Handling call
0709 12:20:04.276062 (+  6945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.467322 (+191260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:04.467352 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:04.467396 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:04.467431 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.744863 (+1277432us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001e82d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:05.744909 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.745023 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439392512}
I20260709 12:20:05.778229 32112 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:20:05.762084 32038 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:05.783494  6415 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:05.784272  6434 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:05.786055 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 153736us
W20260709 12:20:05.789659 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 194312us
I20260709 12:20:05.798162 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.790944 31972 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:05.803383 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.790347 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.791759 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.793344 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.794178 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.805002 31964 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:05.805402 32112 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:05.790022  6389 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:20:05.797312 31968 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:05.806605 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.811041 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:05.811597 32112 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:05.812686 31969 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:05.813678 32038 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:05.778417  6402 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:05.808127  6606 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:05.800805  6415 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:05.818428 32040 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:05.824280 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.778443  6580 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 12:20:05.825843 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.827016 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.796432  6318 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:05.846134 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.832901 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.785274  6608 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.826103  6426 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:05.800953  6480 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:05.838410 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.883908 31968 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:05.825222 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.841508 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:05.876758 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:05.884900 32117 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:05.833761 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:05.885593 32118 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.857848 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:05.845422  6469 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 12:20:05.840013  6606 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.883874 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.826241  6540 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:20:05.882867  6434 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
W20260709 12:20:05.892532 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68661.8us
I20260709 12:20:05.902700 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.890548 32045 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:05.848798  6579 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:05.881376  6402 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:05.926512 32039 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.926872 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.927811 31964 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:05.928267  5949 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.900995  6606 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:05.908924 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.931381 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.931847  6618 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:05.932512 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:05.933095  6618 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.912835 32118 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:05.908783 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:05.933852  6607 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:05.937354 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.937915 32047 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.938215 32041 raft_consensus.cc:1240] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:05.938664 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.939407 32040 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:05.941692 32046 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:05.942627 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.942864 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.943210 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:05.944056 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.945246 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:05.945379  6449 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.946739 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.947274 32040 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.947454 32047 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:05.890287  6426 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:05.948477  6426 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.926952  6579 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.951081  6272 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.951712 32047 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:05.952440  6579 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.953058  6272 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.848685  6318 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:05.928748  5949 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:05.953145  6449 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:05.953648  6318 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:05.955091  5949 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.957481  6449 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:05.929080 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:05.959012 32013 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:05.959928  6442 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.960275 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:05.960901 32113 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:05.929015  6432 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:05.907428 31972 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:20:05.900915  6469 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.970822 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:05.971591 32038 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 28: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:05.971863  6469 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:05.973543 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.975063  6415 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:05.990936 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:05.991204 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 82514.1us
I20260709 12:20:05.983147 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.984220  6507 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:05.890023  6438 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:20:05.986624  6402 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:05.987175 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:05.989308 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:05.914417 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.998612 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:05.999240 32118 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:06.000097 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:06.000200 32040 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:06.002933 32115 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.003726 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:06.009647 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.010464 32038 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:06.012948 31964 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:06.015837 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:05.846022  6617 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.017359 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:05.994272 32116 raft_consensus.cc:2410] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:20:05.927809  6525 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:06.002821 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:06.018031 31964 raft_consensus.cc:3060] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:06.019462 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:05.996605  6580 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:06.031859 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.005476  6507 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.018436  6433 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:20:06.031152  6402 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 19.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.112556 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:06.140358 31970 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:06.113191 32115 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:20:06.018143  6620 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:06.144989 32118 raft_consensus.cc:2410] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 12:20:06.110134 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:06.121569 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 118829us
I20260709 12:20:06.138441 32039 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
W20260709 12:20:06.125063 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 490217us
W20260709 12:20:06.142427 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.147185 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:05.993182 32113 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:06.148108 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:06.149999 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:06.153352 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 133959us
I20260709 12:20:06.153941 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:06.154512 32117 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:05.981802  6426 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.154913 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:06.111606  6432 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:06.157276 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:06.174328 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:06.158638 32112 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:06.118872  6580 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.165194 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.015384  6411 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 12:20:06.179422 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:06.151816  6530 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:06.149266 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:06.176817 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.171095 31964 raft_consensus.cc:2468] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:06.151682  6396 raft_consensus.cc:2764] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:20:06.189198 32045 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
W20260709 12:20:06.203151 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.203568 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:06.206297 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:06.207095 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:06.212491  6570 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:06.215466 32010 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:06.218142  6389 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:06.219982 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.220279 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:06.221978  6618 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:06.223743 32010 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:06.224787  6570 raft_consensus.cc:2804] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:06.225406  6570 raft_consensus.cc:697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:06.227150  6570 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.233363 32047 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:06.234625 31965 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:06.164683  6408 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.246563  6608 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.249014  6318 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.250443  6417 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.250933  6396 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:06.251495  6396 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:06.178988  6580 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.172799  6623 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.183928  6530 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.257865 32117 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:06.245829 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:06.296415 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50654.8us
W20260709 12:20:06.295149 32119 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.015s	sys 0.002s
W20260709 12:20:06.300595 32119 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.015s	sys 0.002s
I20260709 12:20:06.315647 32117 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:20:06.316228 32117 pending_rounds.cc:85] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 13
I20260709 12:20:06.316495 32117 pending_rounds.cc:107] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:20:06.316762 32117 raft_consensus.cc:2889] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:06.256929  6413 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.290074  6530 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.334021 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:06.336596 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.343848  6617 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:06.347605  6624 log.cc:927] Time spent T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:20:06.290624  6623 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.330765  6591 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.329324  6480 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:20:06.408560  6327 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:06.409210  6327 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:06.419337  6426 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.261950  6396 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.427840  6399 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:06.439060  6575 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.475584 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:06.472601  6606 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.006s
I20260709 12:20:06.437465  6591 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:06.486088  6631 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:20:06.475193  6318 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:06.438377  6327 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:06.497851 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 349775us
I20260709 12:20:06.476707  6575 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.494364  6525 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:06.488811  6606 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:06.503605 32114 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.013s	sys 0.026s
W20260709 12:20:06.523101 32114 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.013s	sys 0.026s
W20260709 12:20:06.542224 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.568216 32046 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:06.568693 31971 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:06.570389  6507 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.571179 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:06.571846  6507 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.572335 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:06.572659 31971 raft_consensus.cc:1240] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:06.573271  6591 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.573486 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:06.574673  6338 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.575232  6338 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.577647  6338 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:06.572357  6507 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.579411  6575 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:06.580003  6507 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:06.580058  6579 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.580818  6579 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.582985  6579 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.584383 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:06.586417  6580 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.589179  6580 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:06.592240  6639 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:20:06.593305  6632 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 12:20:06.594969  6644 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:20:06.601290 32121 pending_rounds.cc:85] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 15
I20260709 12:20:06.601729 32121 pending_rounds.cc:107] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:20:06.602011 32121 raft_consensus.cc:2889] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:06.606813 32042 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
W20260709 12:20:06.608153 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.616029 32046 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:20:06.618187 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.620913 32047 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:06.622862 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.623638 32046 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:06.614184 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:06.630712 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:06.631273 32039 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.624505  6575 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.643381 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 469134us
I20260709 12:20:06.644209 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:06.668491 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:06.670794  6645 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:06.677933  6645 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:06.678937  6645 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:06.682631 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.671283  6411 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:06.683880  6332 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:06.684368  6332 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:06.685405  6332 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.686981  6332 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.675716 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:06.675983  6540 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.694768  6540 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.695379  6411 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:06.696120  6411 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:06.676271  6530 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.697038  6540 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:06.697077  6411 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.703661  6338 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:06.707062  6338 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:06.710037 32084 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.006s	sys 0.003s
W20260709 12:20:06.712798 32084 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.007s	sys 0.004s
W20260709 12:20:06.713414 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 542042us
I20260709 12:20:06.639909  6434 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:06.638967  6318 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.635156  6575 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.731341  6434 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.740633  6434 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:06.758930  6640 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:20:06.709573 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:06.768220 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61831.1us
I20260709 12:20:06.771047 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:06.778124 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.634987  6525 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:06.765641  6439 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.002s	sys 0.009s
W20260709 12:20:06.784513 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.788538  6575 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:06.790613 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:06.791294 31972 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.796312  6606 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:06.779965  6646 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.780908  6579 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:06.797387  6646 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.797966  6579 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.798386  6579 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.799572  6646 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.781844  6653 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:20:06.763924  6318 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.806751  6318 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:06.829207 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 119870us
I20260709 12:20:06.769269  6530 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:06.835755 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:06.836100 32042 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:06.836582 31971 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:06.829988  6608 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:06.839804 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:06.828712  6439 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.003s	sys 0.009s
I20260709 12:20:06.842422 32039 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:06.786592  6415 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:06.821774  6408 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:06.837922  6575 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.855504 32116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.008s	sys 0.007s
W20260709 12:20:06.856006 32116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.008s	sys 0.007s
I20260709 12:20:06.863350 31971 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:06.843235  6399 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:06.868973 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:06.880851 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 378185us
I20260709 12:20:06.887969 32655 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:06.886008 31972 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:06.850170  6432 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:06.853889 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:06.893846 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:06.902362 32114 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.889629  6399 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:06.928433 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:06.899125  6432 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:06.958245 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:06.860477  6408 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.862154  6575 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.859434  6415 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:06.965873 32114 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:06.967754 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:06.901535  6530 consensus_queue.cc:1059] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:20:06.970252 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:06.972599 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:06.976758 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.957326  6469 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:06.960623  6652 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:06.956696  6403 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:20:06.923882  6642 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:20:07.006896 32114 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:07.013047 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3134) took 1011 ms. Trace:
I20260709 12:20:06.951391  6641 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.100s
I20260709 12:20:07.019541 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:07.014581 32112 rpcz_store.cc:276] 0709 12:20:06.001163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.401088 (+399925us) service_pool.cc:224] Handling call
0709 12:20:06.401467 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.401724 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.401762 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.438745 (+ 36983us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.875372 (+436627us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:06.875434 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.875536 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.012815 (+137279us) raft_consensus.cc:1685] finished
0709 12:20:07.012838 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.012981 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":402671,"threads_started":1,"wal-append.queue_time_us":545073}
I20260709 12:20:07.021399 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.022171 32112 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:07.005962  6663 raft_consensus.cc:3055] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:07.024276 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.005029  6661 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:07.027334 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.029291 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.030136 32112 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
W20260709 12:20:07.031255  6659 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:20:07.024861 32114 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.031576  6434 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.014390  6338 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:20:07.025856  6663 raft_consensus.cc:740] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:07.034288  6642 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:07.015430  6652 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:07.056716 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 106112us
I20260709 12:20:07.060277 32623 leader_election.cc:400] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24.
I20260709 12:20:07.066183 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:07.067276 31972 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:07.034016  6530 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
W20260709 12:20:07.069005 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 100841us
I20260709 12:20:07.075330 31938 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:07.075974 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:07.074959 32114 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:07.079623 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:07.077730 32112 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:07.082541 32112 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:07.033480  6661 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.088292 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:07.089679 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:07.093804 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.097234 32112 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:07.098740 32112 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:07.100663 32112 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.058192  6642 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:07.103899  6408 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:07.104136  6669 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:07.104540 32112 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:07.104673  6663 consensus_queue.cc:260] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:07.105583  6666 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:20:07.106324  6663 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.108021  6438 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:07.108695  6438 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:07.109269  6438 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:07.109678  6438 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:07.111835  6661 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.124405 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3135) took 1123 ms. Trace:
I20260709 12:20:07.124862 32121 rpcz_store.cc:276] 0709 12:20:06.001447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.600663 (+599216us) service_pool.cc:224] Handling call
0709 12:20:06.601059 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.606954 (+  5895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.607001 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.607518 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.999430 (+391912us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:07.007206 (+  7776us) log.cc:844] Serialized 14 byte log entry
0709 12:20:07.007627 (+   421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.007711 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.124194 (+116483us) raft_consensus.cc:1685] finished
0709 12:20:07.124218 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.124343 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9724,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2277,"raft.queue_time_us":193,"spinlock_wait_cycles":15132416,"thread_start_us":391470,"threads_started":1,"wal-append.queue_time_us":514490}
I20260709 12:20:07.127872  6438 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:07.132356 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.104584  6408 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:07.136684 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 156688us
I20260709 12:20:07.096355  6332 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:07.135536  6655 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:20:07.130503  6399 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:07.145637  6657 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 12:20:07.157377  6660 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:20:07.157907  6642 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.158622 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3128) took 1160 ms. Trace:
I20260709 12:20:07.154647  6415 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.152983  6575 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.136744  6681 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:20:07.138350  6469 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:07.164132  6469 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.168520 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3141) took 1050 ms. Trace:
I20260709 12:20:07.156965  6675 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:07.161216 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.159021 32116 rpcz_store.cc:276] 0709 12:20:05.998274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.188712 (+190438us) service_pool.cc:224] Handling call
0709 12:20:06.245151 (+ 56439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.251267 (+  6116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.251314 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.251876 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.856633 (+604757us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:06.856671 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.856733 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.158415 (+301682us) raft_consensus.cc:1685] finished
0709 12:20:07.158435 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.158567 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5120,"raft.queue_time_us":5486,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":604345,"threads_started":1,"wal-append.queue_time_us":764703}
W20260709 12:20:07.161777  6667 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:20:07.169071 32113 rpcz_store.cc:276] 0709 12:20:06.118152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.810264 (+692112us) service_pool.cc:224] Handling call
0709 12:20:06.810641 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.838255 (+ 27614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.838284 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.844118 (+  5834us) log.cc:844] Serialized 26 byte log entry
0709 12:20:07.014904 (+170786us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:07.032802 (+ 17898us) log.cc:844] Serialized 14 byte log entry
0709 12:20:07.032986 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.033058 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.168207 (+135149us) raft_consensus.cc:1685] finished
0709 12:20:07.168239 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.168436 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27199,"prepare.queue_time_us":13993,"prepare.run_cpu_time_us":4704,"prepare.run_wall_time_us":11199,"raft.queue_time_us":141466,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":12838,"thread_start_us":145486,"threads_started":1,"wal-append.queue_time_us":226343}
I20260709 12:20:07.177651  6675 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.183588 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3140) took 1065 ms. Trace:
I20260709 12:20:07.184018 32118 rpcz_store.cc:276] 0709 12:20:06.117926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.756741 (+638815us) service_pool.cc:224] Handling call
0709 12:20:06.757114 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.757556 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.757584 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.758117 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:20:07.032906 (+274789us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:07.032944 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.033007 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.183366 (+150359us) raft_consensus.cc:1685] finished
0709 12:20:07.183389 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.183521 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":268113,"threads_started":1,"wal-append.queue_time_us":322318}
I20260709 12:20:07.192174  6681 consensus_queue.cc:1243] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:20:07.193192  6652 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.193364 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:07.194942 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3045) took 1048 ms. Trace:
I20260709 12:20:07.195356 31970 rpcz_store.cc:276] 0709 12:20:06.146291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.146436 (+   145us) service_pool.cc:224] Handling call
0709 12:20:06.146749 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.156622 (+  9873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.156651 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.184115 (+ 27464us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.719695 (+535580us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:06.719770 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.719956 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.194739 (+474783us) raft_consensus.cc:1685] finished
0709 12:20:07.194758 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.194889 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9486,"raft.queue_time_us":14345,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":24957,"thread_start_us":456452,"threads_started":1,"wal-append.queue_time_us":602550}
I20260709 12:20:07.196447  6684 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:07.196990  6684 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:07.198578  6530 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:07.198974  6507 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:07.180470  6434 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.200088  6675 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.200480  6507 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:07.200866  6507 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:07.201304  6642 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:07.198529  6579 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.201876  6507 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.155380  6432 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:07.254168 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 164586us
W20260709 12:20:07.255162 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60188.1us
W20260709 12:20:07.257347 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:07.263891 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 187972us
I20260709 12:20:07.267772 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:07.250507  6434 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.277870  6434 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.197818  6684 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.295900  6399 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:07.297946 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.298108  6579 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.318954 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3038) took 1306 ms. Trace:
I20260709 12:20:07.320494 31963 rpcz_store.cc:276] 0709 12:20:06.012436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.012711 (+   275us) service_pool.cc:224] Handling call
0709 12:20:06.110791 (+ 98080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.521344 (+410553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.521390 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.528442 (+  7052us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.848654 (+320212us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:06.848686 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.276483 (+427797us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540034e254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:07.276564 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.276647 (+    83us) raft_consensus.cc:1685] finished
0709 12:20:07.276668 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.318814 (+ 42146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78956,"raft.queue_time_us":83853,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7970,"spinlock_wait_cycles":147907456,"thread_start_us":607755,"threads_started":2,"wal-append.queue_time_us":444240}
I20260709 12:20:07.326584 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.328227 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.330327 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.318012  6649 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:07.319171  6507 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.333290  6620 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:07.331405  6411 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:20:07.324259  6216 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.355566 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:07.356259 31964 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:07.323069  6318 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.334574 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:07.319380  6652 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:20:07.336793  6649 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.324247 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:07.368688 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44534.7us
I20260709 12:20:07.368763 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:07.369387 32046 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.370461 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.371114 32047 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 24. Current term is 25. Ops: [23.18-24.19]
I20260709 12:20:07.371622 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.348518 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:07.352856 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.360412 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.362869 32041 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:07.364081 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.364439 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 3006) took 1453 ms. Trace:
I20260709 12:20:07.361944 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.376433 32115 rpcz_store.cc:276] 0709 12:20:05.876305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.876491 (+   186us) service_pool.cc:224] Handling call
0709 12:20:07.329092 (+1452601us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400123754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:07.329218 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270536448}
I20260709 12:20:07.325485  6415 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:20:07.388049 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:07.383096 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:07.356077  6550 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.328032  6579 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:20:07.389317 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.434140 31971 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:20:07.389755 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.435129 31964 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:07.367591  6160 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:07.460091 32046 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:07.383625  6575 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.382808  6683 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:07.380733  6649 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:07.382459  6685 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.417755  6184 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:07.431319  6579 consensus_queue.cc:1243] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:07.367614  6216 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.389304  6686 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.380406  6318 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.461004  6160 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:07.478894 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 104264us
I20260709 12:20:07.489094 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.489248 32115 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:07.490417 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.476312  6480 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:07.431497  6550 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.475669  6606 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:07.498021 32118 raft_consensus.cc:2393] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:07.476656  6683 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.529505  6707 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.506081  6700 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:07.482506  6184 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.488309  6575 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:20:07.539683  6184 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:07.539798  6638 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.903s	user 0.000s	sys 0.004s
I20260709 12:20:07.566105 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 3012) took 1605 ms. Trace:
I20260709 12:20:07.566679 32120 rpcz_store.cc:276] 0709 12:20:05.960212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.960999 (+   787us) service_pool.cc:224] Handling call
0709 12:20:05.961336 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:05.961676 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:05.961703 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:05.962144 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.475386 (+513242us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:06.475424 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.475486 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.539266 (+1063780us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:07.565750 (+ 26484us) raft_consensus.cc:1685] finished
0709 12:20:07.565774 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.566032 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5985,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":4391296,"thread_start_us":497789,"threads_started":1,"wal-append.queue_time_us":674555}
I20260709 12:20:07.541028 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3138) took 1423 ms. Trace:
I20260709 12:20:07.568497 32119 rpcz_store.cc:276] 0709 12:20:06.117486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.678601 (+561115us) service_pool.cc:224] Handling call
0709 12:20:06.678930 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.781361 (+102431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.781406 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.781898 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:20:07.009977 (+228079us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:07.010015 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.540617 (+530602us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002b5cd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:07.540675 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.540726 (+    51us) raft_consensus.cc:1685] finished
0709 12:20:07.540743 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.540965 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1378,"raft.queue_time_us":5154,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":13988,"spinlock_wait_cycles":234630016,"thread_start_us":227264,"threads_started":1,"wal-append.queue_time_us":314155}
I20260709 12:20:07.548826  6439 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:07.574862  6439 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:07.575810  6439 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.555473  6160 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.514819  6480 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:07.577087  6439 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:07.526160  6684 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:20:07.527442  6686 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:20:07.531175  6690 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:07.581867  6690 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.585723  6690 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.524834  6606 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.586798  6683 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.587921  6606 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:07.588696 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.559620  6700 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.590920  6700 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.536291  6685 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.577654  6480 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.560902  6318 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:07.562573  6216 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.564011  6550 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.564548  6575 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:20:07.599676  6525 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:07.600296  6525 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.605093  6525 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:07.614044 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 231001us
I20260709 12:20:07.615134 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:07.625237 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.618535  6606 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:20:07.626870 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 136529us
I20260709 12:20:07.613690  6649 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.633620 32043 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:07.630154 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.634476  6575 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:07.635787  6575 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:07.636125  6684 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:07.637539 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.638854 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:07.639886 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:07.640698 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.641546  6318 raft_consensus.cc:2764] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:07.642072  6685 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:07.642252  6318 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.642572  6685 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.643056  6685 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:07.643059  6318 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:07.651124  6685 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.652678  6623 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.653986  6685 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.654462 32084 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.629549  6620 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:07.625777  6507 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:20:07.672833 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.626811  6396 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.031s
I20260709 12:20:07.675454 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.679440 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:07.680095 32042 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.682763 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.684044 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.684206 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.685014 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.685359 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.685119 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.686458 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.689631 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.689994  6623 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.690654  6216 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.690834 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.691586 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.691900  6327 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.692528 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.692870  6607 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.693329  6607 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.693753 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.657316  6606 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:07.694617 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.694965  6606 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.696168 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.692539  6550 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.659514  6641 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:07.708948 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.709805 32038 raft_consensus.cc:2410] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:20:07.699558 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:07.702709 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.705485 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.707434 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.712639 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:07.713457 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.659134  6525 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.716987 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:07.723312 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.724460 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.725330 32038 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:07.718955 31964 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:20:07.722139 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.732933 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.733268 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:07.735199 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.741497 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:07.742684 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:07.676158  6685 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.722599  6700 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:07.697197  6550 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.750346 31968 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.11-24.11]
I20260709 12:20:07.657564  6704 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 12:20:07.708222  6318 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.708238  6641 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:07.768719  6607 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.750147  6216 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:07.771196  6570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.540s	user 0.010s	sys 0.029s
I20260709 12:20:07.797117  6649 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.803730 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.805027 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:07.806999 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.809525 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:07.810926 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.812119 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.813419 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.816895 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.743341  6525 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:07.802945  6570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.572s	user 0.011s	sys 0.029s
I20260709 12:20:07.809510  6216 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.827629  6216 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.773536  6625 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:20:07.833243 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:07.834059 32038 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:07.835290  6713 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:07.770375  6318 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:06.853977 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:07.839564 32041 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:07.802649  6184 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:07.811517  6700 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:07.811823  6411 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.808897  6683 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.857573 31972 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:20:07.843061 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:07.860993 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:07.821962  6675 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.823753  6685 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.823638  6623 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.873406 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:07.902735  6216 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.910506  6396 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:07.911355  6396 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.915407  6550 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.865708  6683 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.919799  6525 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.900332  6411 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.920881  6683 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.921993  6438 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.922739  6438 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.864174  6184 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.925313  6438 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.925657  6184 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.923287  6411 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:07.928561 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51825.8us
I20260709 12:20:07.927237  6318 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.822813  6709 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.858655  6649 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:07.905369 31865 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 15 to 16, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 16 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:07.933739 32041 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:07.935333 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74502.3us
I20260709 12:20:07.937261 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.939906 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:07.942672 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69371.7us
I20260709 12:20:07.945657 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:07.914146  6396 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:07.903040  6436 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.947292 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:07.955220 31970 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:07.957228 32038 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:07.842571  6641 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.960806  6641 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:07.975099  6719 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:20:07.956766  6436 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:07.979249  6436 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:07.980041  6662 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 1.081s	user 0.006s	sys 0.004s
I20260709 12:20:07.981624 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3130) took 1982 ms. Trace:
I20260709 12:20:07.981994 32117 rpcz_store.cc:276] 0709 12:20:05.998752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.317993 (+319241us) service_pool.cc:224] Handling call
0709 12:20:06.321463 (+  3470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.321860 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.321894 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.331767 (+  9873us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.710761 (+378994us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:06.710848 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.711040 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.979461 (+1268421us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:07.981315 (+  1854us) raft_consensus.cc:1685] finished
0709 12:20:07.981338 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.981552 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6727,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2202,"spinlock_wait_cycles":4243328,"thread_start_us":336164,"threads_started":1,"wal-append.queue_time_us":474017}
I20260709 12:20:07.984051  6707 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.941798  6649 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:07.946765  6184 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:07.986713  6184 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:07.986991  6649 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:07.987620  6184 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 24.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.000572 32112 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:20:08.001003 32112 pending_rounds.cc:85] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 9
I20260709 12:20:08.001214 32112 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:20:08.001492 32112 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:08.007546 32041 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:08.014110 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.006003  6327 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.019367  6327 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.020874  6575 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:08.021610  6327 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:08.022959  6607 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:08.024192  6707 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:08.025452  6709 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:08.025812  6709 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:08.026620  6709 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.027707  6709 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.030248 31970 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:08.032260  6399 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:08.033267  6399 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:08.033584  6399 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:08.034425  6399 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.035496  6399 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:08.016456 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71694.4us
I20260709 12:20:08.052932  6591 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:08.057746  6591 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.058710  6707 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:08.061496 32041 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:08.063647 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:08.065356 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.066649 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 933) took 2129 ms. Trace:
I20260709 12:20:08.067034 32044 rpcz_store.cc:276] 0709 12:20:05.936718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.938415 (+  1697us) service_pool.cc:224] Handling call
0709 12:20:08.066425 (+2128010us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400032fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:08.066554 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2552292736}
W20260709 12:20:08.070453 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.071683 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:08.072768 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:08.073489 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:08.073736 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:08.073992 32116 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:08.075326 32114 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:08.075947 32043 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:08.077812 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:08.078480 32043 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:08.079384 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:08.080897 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:08.081334 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.081481  6432 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.083040 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.083650 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.083902 32657 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:08.084573 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.074052 32045 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:08.090503 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:08.090914 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.091127 31970 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:08.093220 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.094732 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.095063 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.096020 32041 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:08.100688 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:08.100827 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.115180 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.132051 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.113605  6327 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:08.132448 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.138065 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.131732 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.098821  6684 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:08.134869  6591 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.103231  6682 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.130668  6399 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.137089  6327 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:08.161175 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.162076 32657 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:08.133953  6717 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:08.143953  6434 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:08.165089 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.171644 32657 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:08.163615 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.116861  6507 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:20:08.132175  6530 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:20:08.144819  6713 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:08.158993  6540 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:08.176921 31967 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.11-24.11]
I20260709 12:20:08.163600  6327 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.180650 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.172712  6570 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.185794 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.154160  6684 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.189826  6684 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.178156  6540 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:08.190261 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.165413  6399 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.179980  6434 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.201292 32041 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:08.177691  6713 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.175711  6716 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:20:08.208344  6713 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:08.225749 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.226666 32047 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:08.227679 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.228902 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:08.231741  6730 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:20:08.234019  6725 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:20:08.235793 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.236413 32046 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:08.237190 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.238233 31967 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 24: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:20:08.240916  6698 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.859s	user 0.003s	sys 0.000s
I20260709 12:20:08.230085 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.243286  6686 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:08.243883  6686 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:08.245041  6686 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.254626  6327 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.257273  6327 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:08.258087 32657 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 19. (term mismatch)
I20260709 12:20:08.258509 32657 pending_rounds.cc:85] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Aborting all ops after (but not including) 18
I20260709 12:20:08.258735 32657 pending_rounds.cc:107] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Aborting uncommitted NO_OP operation due to leader change: 24.19
I20260709 12:20:08.258952 32657 raft_consensus.cc:2889] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:08.265157 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:08.180758  6607 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:20:08.275718 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.280750 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.282871 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.283707 32651 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:08.284200 31971 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:08.183905  6575 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:20:08.287726 31970 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:08.285609 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:08.293391 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.294355 32653 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:08.295347 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.269431  6641 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.303390 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.305127 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.319129 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.292466  6682 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:08.304108  6713 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:08.261768  6434 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:08.329625  6580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.628s	user 0.012s	sys 0.043s
I20260709 12:20:08.342607  6653 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:08.354205  6591 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:08.355309  6653 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.357156 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:08.357625  6653 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.361789  6399 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.347975  6713 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:08.362978  6525 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:08.363277  6713 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.367257 31967 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:20:08.366489 31865 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:08.346856  6580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.645s	user 0.012s	sys 0.044s
I20260709 12:20:08.374914 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:08.375790  6434 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:08.376127 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.376725 32041 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 17.
I20260709 12:20:08.377924 32041 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:08.358247  6653 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:08.325346  6641 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.380353 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 3027) took 1314 ms. Trace:
I20260709 12:20:08.367853 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:08.380803 32112 rpcz_store.cc:276] 0709 12:20:07.066001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.106241 (+ 40240us) service_pool.cc:224] Handling call
0709 12:20:07.106488 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:08.002009 (+895521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.380041 (+378032us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54002b52d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:08.380275 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055379840}
I20260709 12:20:08.382543  6641 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.383563  6653 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:08.384368  6653 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:08.385284  6653 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.389817  6338 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:08.391474 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.392055 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.405884 32011 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.405874 32084 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:08.407689 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.409584 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:08.401813 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:08.413841 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.414458 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:08.415230 32042 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:08.415048 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.422402 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:08.421046 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 220718us
I20260709 12:20:08.424027 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.426148 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.429441 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.432250  6718 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:08.432767  6718 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.433104  6718 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.437369 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:08.437534 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.412300  6625 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.465404 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.465698 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:08.467870 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:08.468623 32042 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:08.408912  6607 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:08.425103  6645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.742s	user 0.009s	sys 0.019s
I20260709 12:20:08.421205  6318 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.421016  6675 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:08.424887  6685 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:08.424978  6550 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.474958 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.425102  6591 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:08.425230  6434 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.437649  6216 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:08.506166 32655 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:20:08.473681  6687 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:08.437573  6606 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.464797  6717 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:08.499913 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:08.509508 32116 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
W20260709 12:20:08.478762  6645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.796s	user 0.010s	sys 0.020s
I20260709 12:20:08.436926  6623 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:08.507762 32038 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
W20260709 12:20:08.514066 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 222810us
I20260709 12:20:08.515560 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.485536  6327 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:08.528899  6684 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:08.533107 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.503232  6591 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.480024  6579 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:20:08.538229 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:08.586920 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48793.3us
I20260709 12:20:08.530584  6507 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.507313  6685 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:08.590811  6685 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.579708  6727 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.594834  6727 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.597275 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:08.587041  6717 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:08.578418 32117 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:08.513829  6016 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.509691  6216 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:08.510715  6687 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:08.571883  6591 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.601334 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:08.522043  6434 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:08.615820 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 186460us
I20260709 12:20:08.597321  6727 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:08.645467 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.647408 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.653396 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:08.654675 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 151107us
I20260709 12:20:08.656106 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:08.614809  6717 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.662237 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 3032) took 1324 ms. Trace:
I20260709 12:20:08.661245 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 921) took 1233 ms. Trace:
I20260709 12:20:08.679003 32118 rpcz_store.cc:276] 0709 12:20:07.337642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.365505 (+ 27863us) service_pool.cc:224] Handling call
0709 12:20:08.661991 (+1296486us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540013a954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:08.662166 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250907008}
I20260709 12:20:08.679136 31966 rpcz_store.cc:276] 0709 12:20:07.427455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.427662 (+   207us) service_pool.cc:224] Handling call
0709 12:20:07.427925 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:07.511895 (+ 83970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:07.511927 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:07.511971 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:07.512029 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.660979 (+1148950us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b5400315e54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:08.661042 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:08.661180 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192649600}
I20260709 12:20:08.676055 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.616786  6338 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:08.681409 32655 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:08.628265  6216 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.627097  6687 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:08.651934  6507 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.652127  6713 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:08.616920  6653 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:20:08.677089  6675 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.689055  6717 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:08.714026 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 112793us
I20260709 12:20:08.716022 31936 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:08.688400  6318 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:08.699072  6623 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.709707  6713 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.716681  6625 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:08.761602  6738 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:20:08.762449  6766 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:08.764101  6766 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.714082  6507 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.743023  6713 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.763903  6718 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.733579  6318 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.789954  6318 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:08.742681  6623 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.791754  6434 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.770183  6550 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:08.758858  6579 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:08.797071  6723 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.864s	user 0.004s	sys 0.000s
I20260709 12:20:08.801239  6507 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.807859  6718 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.809870  6591 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:08.813750  6579 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:08.819207  6623 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:08.887450  6403 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:08.878520  6579 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:08.896106  6769 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:08.896629  6591 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:08.900061  6751 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.203s	user 0.005s	sys 0.000s
I20260709 12:20:08.887609  6713 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.900318  6687 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.902117  6713 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:08.902174  6687 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:08.900154  6717 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:08.904923  6717 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:08.906674  6750 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
W20260709 12:20:08.907591  6757 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 12:20:08.912943  6742 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.323s	user 0.003s	sys 0.000s
W20260709 12:20:08.912981  6752 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.205s	user 0.009s	sys 0.000s
W20260709 12:20:08.914518  6741 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.331s	user 0.003s	sys 0.000s
W20260709 12:20:08.914676  6753 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.211s	user 0.003s	sys 0.000s
W20260709 12:20:08.915534  6755 log.cc:927] Time spent T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.199s	user 0.003s	sys 0.000s
W20260709 12:20:08.952174 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 236056us
I20260709 12:20:08.955874 32041 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:20:08.999996 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:08.891315  6403 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:09.003571 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:09.005121  6528 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:08.969707 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.972325 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:09.006047 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 316266us
I20260709 12:20:08.973099 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3073) took 1034 ms. Trace:
I20260709 12:20:09.011941 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:09.012086  6160 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:08.961947 32039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 2970) took 2105 ms. Trace:
I20260709 12:20:08.973969 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.975462 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:08.994382 31967 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.11-24.11]
I20260709 12:20:08.996225  6411 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:08.963765 32044 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:20:09.020164 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.014810 32039 rpcz_store.cc:276] 0709 12:20:06.856745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.856913 (+   168us) service_pool.cc:224] Handling call
0709 12:20:06.904199 (+ 47286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.904902 (+   703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.904973 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:07.696814 (+791841us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b2c0005c228. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:20:07.697189 (+   375us) log.cc:844] Serialized 26 byte log entry
0709 12:20:07.807866 (+110677us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:07.807901 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.807969 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:08.796535 (+988566us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:08.961331 (+164796us) raft_consensus.cc:1685] finished
0709 12:20:08.961385 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:08.961701 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":37891,"spinlock_wait_cycles":29924864,"thread_start_us":103756,"threads_started":1,"wal-append.queue_time_us":128123}
I20260709 12:20:09.024714 32655 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:09.037253 32660 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 12:20:09.025653 32653 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:09.026059  6724 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:09.010957 31963 rpcz_store.cc:276] 0709 12:20:07.939049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.941171 (+  2122us) service_pool.cc:224] Handling call
0709 12:20:08.386626 (+445455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:08.386843 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:08.387146 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:20:08.512575 (+125429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:08.512645 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:08.512705 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.512803 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:08.912484 (+399681us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:08.972680 (+ 60196us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54000794d0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:08.972922 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33685,"prepare.queue_time_us":568,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":5868,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":8623,"spinlock_wait_cycles":1124028032,"thread_start_us":119109,"threads_started":1,"wal-append.queue_time_us":258043}
I20260709 12:20:09.027240  6016 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:09.041045  6685 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.041296  6016 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.027930  6411 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.042171  6016 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.042675 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:09.045644 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.041558  6724 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.046262 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:09.046662  6724 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.047899  6411 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.035037 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:09.041852  6685 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.005180  6403 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:09.056126 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.056641  6713 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.058971  6773 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:09.059372  6685 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.062772  6707 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:09.063102  6318 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.069802 31965 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:20:09.070484  6687 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:09.071295 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:09.071429 32039 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:09.073391 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.042690 31967 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 12:20:09.080920 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 77716.8us
I20260709 12:20:09.088264  6550 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:09.077214 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:09.101812  6713 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.102635  6754 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.103513  6754 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.104666  6713 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.105312  6713 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.106484  6469 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:20:09.107432  6469 consensus_queue.cc:1243] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:20:09.108681  6318 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.111454  6318 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.112061 32117 pending_rounds.cc:85] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 10
I20260709 12:20:09.112406 32117 pending_rounds.cc:107] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:20:09.112700 32117 raft_consensus.cc:2889] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:09.112931 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.113679 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.114467 32120 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 14.
I20260709 12:20:09.082300  6773 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.105906  6754 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.140208 32046 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:09.144266 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:09.145397 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:09.146348 32118 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:09.120342  6773 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.118866  6707 consensus_queue.cc:1243] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 25.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:09.149634  6747 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.151561  6675 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.152508  6675 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.154860  6318 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.163758  6747 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.106068  6713 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.155690  6318 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.179461  6686 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.180181  6318 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.180914  6570 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.181442 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.181859  6570 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:09.182996 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:09.188621 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 45672.9us
I20260709 12:20:09.171602  6747 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.142107  6016 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:09.205808 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 128846us
I20260709 12:20:09.171415  6754 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:09.210786  6216 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.213169  6216 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.158277  6720 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:20:09.220584 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.199606  6724 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:09.222826 31935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.013s	sys 0.011s
W20260709 12:20:09.233176 31935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.014s	sys 0.011s
I20260709 12:20:09.196864  6704 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 12:20:09.237084 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73532.2us
I20260709 12:20:09.242442 32116 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:09.242584 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:09.243331 32114 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.243782 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.245093 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:09.246446 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 176328us
I20260709 12:20:09.249842 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:09.252017  6620 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.252810  6620 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.257579  6684 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:09.259335 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.264053  6403 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.265172  6682 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:09.265517  6682 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:09.267652  6016 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.269800  6713 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.267378  6682 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.271813 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.271836 32044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2754) took 1092 ms. Trace:
I20260709 12:20:09.272243  6682 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.272495 32044 rpcz_store.cc:276] 0709 12:20:08.179295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.179457 (+   162us) service_pool.cc:224] Handling call
0709 12:20:08.962745 (+783288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:09.021614 (+ 58869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:09.271587 (+249973us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54002f8d50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:09.271755 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785813888}
I20260709 12:20:09.273268  6675 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.279742 31865 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 19 to 20, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 20 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.281179  6706 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.281966  6706 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.285053  6706 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:09.289705 31864 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 246) took 28855 ms (client timeout 15000 ms). Trace:
I20260709 12:20:09.290849  6016 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:09.295275  6618 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:20:09.237630 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 581834us
I20260709 12:20:09.296295  6570 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.295675 31865 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 17 to 21, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 21 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:09.304165 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 97364.6us
I20260709 12:20:09.303488 32657 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:20:09.313238 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:09.315722 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54745.5us
I20260709 12:20:09.316884 32118 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:20:09.320811 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.306063  6216 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:09.309469  6618 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:09.320552  6704 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:09.357695 32044 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:20:09.357973 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:09.358266  6540 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.360039  6540 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.359965 31865 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 18 to 22, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.362682 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:09.366736 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.367545 32041 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:09.367854 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.368855 32039 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:09.370451  6432 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:09.370870  6432 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.371268  6432 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:09.372251 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:09.359033 32651 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.369453 31936 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:09.376374 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.344735  6704 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.387658 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.392571 32114 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:09.393246 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.394382 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.394518 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:09.389370 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:09.362231 31865 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:09.399636  6332 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.381275  6016 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.391522  6704 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.400300  6332 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.400854 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.402335 32119 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:09.402778  6580 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.403218  6580 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.403589  6580 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.404670 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.398005 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.406054  6623 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 12:20:09.406389 32044 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:09.406896 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.407502 32652 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:09.408406  6415 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:20:09.414850 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.415380  6620 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.415599 32043 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:09.400341 32114 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:09.418951 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:09.419112  6480 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:09.420320  6618 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.419996  6016 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.421669  6618 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:09.423233 32626 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.426663  6724 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.427770 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:09.428462 32114 raft_consensus.cc:2368] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:20:09.430642 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.431810 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.434926 32121 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:20:09.435343 32121 pending_rounds.cc:85] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:20:09.435575 32121 pending_rounds.cc:107] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:20:09.435740  6399 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:09.436230  6399 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:09.436352 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:09.437471 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:09.437301  6399 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.438241  6767 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.396790  6675 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.438880  6399 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.439720  6675 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.441234 32114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:09.405706  6579 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.443048 32114 raft_consensus.cc:606] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.443497 32114 raft_consensus.cc:614] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:09.443760 31965 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:09.439029  6767 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.444897 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:09.437322 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:09.446522 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.447098  6767 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:09.297176 31864 rpcz_store.cc:269] 0709 12:19:40.433821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.434091 (+   270us) service_pool.cc:224] Handling call
0709 12:20:09.289420 (+28855329us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:00.809838 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:01.038587 (+228749us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:01.038661 (+    74us) write_op.cc:435] Acquired schema lock
0709 12:20:01.038679 (+    18us) tablet.cc:662] Decoding operations
0709 12:20:01.039899 (+  1220us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:01.040013 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:01.040060 (+    47us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:20:01.040989 (+   929us) tablet.cc:701] Row locks acquired
0709 12:20:01.041048 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:20:01.041281 (+   233us) write_op.cc:270] Start()
0709 12:20:01.041468 (+   187us) write_op.cc:276] Timestamp: P: 1783599601041191 usec, L: 0
0709 12:20:01.041523 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 12:20:01.204635 (+163112us) log.cc:844] Serialized 2824 byte log entry
0709 12:20:01.741248 (+536613us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cae28 00007f228fc05581 00007f228fc04d58 00007f228fc099f1 00007f228fc099a9 00007f228fc09939 00007f228fc09901 00007f228fc08b2d 0000000000388634 0000000000388479
0709 12:20:02.107585 (+366337us) op_driver.cc:464] REPLICATION: finished
0709 12:20:02.423703 (+316118us) write_op.cc:301] APPLY: starting
0709 12:20:02.423835 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:02.541408 (+117573us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:02.541448 (+    40us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:02.680431 (+138983us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:02.680860 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 12:20:02.680948 (+    88us) write_op.cc:312] APPLY: finished
0709 12:20:02.681443 (+   495us) log.cc:844] Serialized 73 byte log entry
0709 12:20:02.682616 (+  1173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:02.682774 (+   158us) write_op.cc:454] Released schema lock
0709 12:20:02.682856 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:02.714246 (+ 31390us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:02.717908 (+  3662us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":105784,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":305643,"apply.run_cpu_time_us":13316,"apply.run_wall_time_us":271529,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33359,"num_ops":7,"prepare.queue_time_us":567636,"prepare.run_cpu_time_us":43376,"prepare.run_wall_time_us":983903,"raft.queue_time_us":250342,"raft.run_cpu_time_us":29053,"raft.run_wall_time_us":400777,"replication_time_us":999291,"spinlock_wait_cycles":77177728,"thread_start_us":917853,"threads_started":4,"wal-append.queue_time_us":372138,"wal-append.run_cpu_time_us":33215,"wal-append.run_wall_time_us":2088927}]]}
I20260709 12:20:09.447676  6767 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.444958  6579 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.435840 32121 raft_consensus.cc:2889] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:09.453305 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.455332 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.451087 31967 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:09.450551  6675 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.458014 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:09.464430 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69445.2us
I20260709 12:20:09.467000 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.459616 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.469596 32117 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:09.470826 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.448386  6767 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.476070  6767 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.476722  6704 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.477007  6773 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.478369 32113 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:09.478609 32119 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:09.479108 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.480027 32115 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:09.481396 32116 raft_consensus.cc:2410] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:20:09.434212  6727 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:09.478396  6686 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:09.488325  6216 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:09.493788 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.490824 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:09.472229  6570 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:09.484099  6675 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:09.515066  6579 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.515331 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.515941  6579 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.518747  6579 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.515964 32043 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:20:09.525432 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:09.526878  6184 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.527652  6184 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.489274  6216 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.533918  6216 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.505848 32047 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
W20260709 12:20:09.536773 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.539168 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:09.584245 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45193.8us
I20260709 12:20:09.480320  6623 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:20:09.585582 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.503822  6570 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.508369  6727 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:09.607105 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 237769us
I20260709 12:20:09.607739 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.609292 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:09.597582  6216 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:09.628113 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.627635  6727 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.618947  6709 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.635329 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:09.636595 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.637250 32114 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:09.639858 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.640491 32117 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:20:09.620631 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.642306 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.643978 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.648608 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.646729 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.653201 32117 raft_consensus.cc:2393] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:09.644573 32114 pending_rounds.cc:85] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 13
I20260709 12:20:09.654106 32114 pending_rounds.cc:107] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:20:09.654510 32114 raft_consensus.cc:2889] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:09.659006  6579 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.668521  6579 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.601338  6570 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:09.672369  6579 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.659076 31865 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:09.676517 31865 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 13 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 19 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:09.660410 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:09.666888 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.665259 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:09.679267  6773 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.656032 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:09.685657 32652 raft_consensus.cc:2410] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:20:09.603500  6332 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.619791  6685 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:09.615991  6184 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.628046  6704 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.691071 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:09.682302  6216 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.705516 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.707984 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:09.712117 32116 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 25: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:09.692764  6579 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.695922  6685 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.704759 31865 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 16 to 18, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 18 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:09.708897  6216 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:09.754278  6432 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.759629  6432 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.760902  6415 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:09.761571  6415 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.762920 32116 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:09.763321  6744 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.725824  6402 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:09.722992  6685 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.748762  6332 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.717140  6675 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.789038 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:09.789690  6415 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.768994  6402 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:09.794166  6684 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:09.794636  6402 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.795186 32116 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:09.796000  6745 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:09.796329 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.797655 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:09.794699  6684 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.796459  6745 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.798693  6684 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.799014  6745 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:09.763933  6744 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.804481  6744 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:09.805850  6790 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:20:09.806048  6332 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.808748  6332 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:09.809368  6332 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:09.809737  6332 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:09.810128  6332 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:09.812227 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.812887 32119 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:09.814385 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:09.815683  6403 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.816304  6403 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.768747  6713 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:09.819892 32654 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 18.
I20260709 12:20:09.764238  6682 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.837939  6580 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.842523 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:09.769274  6579 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.845956 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.835156  6682 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.850962 31865 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:09.859608 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:09.860541 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.865581 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.868330 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.839521  6403 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:09.847987  6709 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.825980  6675 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.871757 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:09.874414  6432 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:09.875349 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.876032  6415 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:09.878463 32120 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:20:09.879262  6579 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:09.871397 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.895285 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:09.897194  6580 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:09.900444 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 86147.7us
I20260709 12:20:09.901728  6184 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:20:09.926607 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.926991 32116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:09.927901 32116 raft_consensus.cc:606] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:09.928463 32116 raft_consensus.cc:614] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:09.908565  6403 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:09.916478  6184 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.951781 32121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3164) took 1438 ms. Trace:
I20260709 12:20:09.959090 32084 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:09.961074 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:09.961357 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:09.914119  6570 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:09.956208 32121 rpcz_store.cc:276] 0709 12:20:08.502477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.656260 (+153783us) service_pool.cc:224] Handling call
0709 12:20:08.660146 (+  3886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.474441 (+814295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:09.920722 (+446281us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b540043a3d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:09.940632 (+ 19910us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780342656}
I20260709 12:20:09.925675  6399 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.907572  6709 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.954955  6184 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:09.974994 32046 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:20:09.958580  6713 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.966361  6745 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:09.991021 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 119387us
I20260709 12:20:09.968709  6686 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:09.925451  6402 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.995592  6727 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:09.968994  6623 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:10.002352 32651 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.611s	user 0.013s	sys 0.000s
I20260709 12:20:09.939904  6684 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.018656 32651 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:10.024070  6767 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.024986  6767 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:09.971014  6415 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:20:09.997093  6618 consensus_meta.cc:287] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.575s	user 0.015s	sys 0.000s
I20260709 12:20:10.031859 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:09.997498  6686 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:10.036880 32116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:10.039760 32116 raft_consensus.cc:606] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.046473 32116 raft_consensus.cc:614] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:10.015571  6623 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.019730  6540 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.048395 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:10.037014 31865 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 17 to 19, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:10.017397  6704 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 12:20:10.055465 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 178423us
I20260709 12:20:10.038110  6727 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.049114  6016 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:09.997882  6713 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.047978  6686 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:10.056412  6402 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.030532  6399 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.052805  6623 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:10.061918 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:10.062711 31865 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 8 to 13, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 13 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:10.067518 31936 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:10.073634 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.052374  6685 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.064594  6682 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.096956 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:10.067087  6016 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:10.112442  6798 log.cc:927] Time spent T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:20:10.112701  6685 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:10.113052 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:10.113457  6685 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.113891 32652 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:10.113729  6743 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:20:10.065274  6540 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.114861 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:10.115334 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:10.090703  6620 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:10.114230 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:10.130296 31865 catalog_manager.cc:5697] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:10.079312  6773 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.124305  6713 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:10.152842 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.101068  6745 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.156790 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.116541  6727 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.161397  6332 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.166673 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.104265  6682 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:10.114144  6685 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.184422 31968 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:10.127028  6579 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.196082 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:10.196983 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:10.201608 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:10.192201  6727 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:20:10.129477  6016 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.216099  6685 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:10.217105 31865 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 16 to 18, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 18 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:10.218520 31865 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:10.220219 31865 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 9 to 16, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 16 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:10.223579  6540 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:10.225793  6709 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:10.231912 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.232400  6216 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.234194 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.236423 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.185187  6675 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.241988 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.244545 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:10.245139 32038 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.245246 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:10.246125 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:10.194463  6684 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.177850  6773 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.247588 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.247594 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:10.250614 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.247850 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.253659 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:10.254504 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.255336 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.207276  6570 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.187079  6682 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.260620 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.258119 31972 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:10.262435 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:10.249917  6685 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:20:10.260032  6745 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.259874  6540 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:10.264851 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.247272  6675 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:10.261521 31964 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:10.270578 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.299964 31966 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:10.296352 32116 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:10.246223  6709 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:10.245301  6432 consensus_queue.cc:1243] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:10.308393  6745 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.256281  6767 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.372257 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:10.307538  6686 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.376547  6682 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.302449 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.388747  6216 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:10.390141 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 348682us
I20260709 12:20:10.396626  6745 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.299266  6540 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.256661  6579 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.396888 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:10.402007 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:10.402289 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 102758us
I20260709 12:20:10.410082 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:10.410710  6709 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:10.393246  6399 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.423695 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:10.410620  6540 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.438438 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:10.443477 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:10.419484  6724 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:10.448069  6686 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.450933  6686 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.451606  6686 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.454826  6812 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:10.455914  6812 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:10.456228  6686 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.458447  6760 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:20:10.463461  6805 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:20:10.454926 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:10.489497 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51744.6us
I20260709 12:20:10.518601 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:10.520397 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:10.525296 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.457357  6812 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.484122  6713 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:10.543380 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:10.544909 32038 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:10.422894  6332 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:10.592182 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48888.9us
I20260709 12:20:10.601016 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:20:10.589260  6805 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 12:20:10.605021 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 566343us
W20260709 12:20:10.647879 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.594215  6675 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.591125  6403 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:10.601450  6709 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:10.648301  6813 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:10.673571 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.023s	sys 0.045s
W20260709 12:20:10.683377 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.023s	sys 0.045s
W20260709 12:20:10.683712 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 605115us
I20260709 12:20:10.664584  6686 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:10.691932 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:10.680398  6184 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
I20260709 12:20:10.672403  6773 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:10.678154  6747 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:10.701290 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 451087us
I20260709 12:20:10.715265 31967 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:20:10.663594  6403 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.717300 32658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1281) took 1058 ms. Trace:
I20260709 12:20:10.687412  6684 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.681972  6813 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:10.743170 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 232276us
I20260709 12:20:10.691078  6686 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.668779  6675 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.756593 31963 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:20:10.736127 32658 rpcz_store.cc:276] 0709 12:20:09.658127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.662239 (+  4112us) service_pool.cc:224] Handling call
0709 12:20:09.662724 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:09.663157 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:20:09.663407 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 12:20:10.064153 (+400746us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.076931 (+ 12778us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.077801 (+   870us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:20:10.077848 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.077919 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:10.716271 (+638352us) raft_consensus.cc:1685] finished
0709 12:20:10.716294 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.716421 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1609,"prepare.queue_time_us":18372,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":447,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5483,"thread_start_us":400268,"threads_started":1,"wal-append.queue_time_us":583957}
I20260709 12:20:10.761512  6756 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.761683 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.762087  6756 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.764482  6756 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.753800  6684 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.767675  6684 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.768857  6684 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:10.764317  6808 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:10.774027  6580 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:10.774588  6580 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:10.775787  6580 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.777297  6580 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.699980  6184 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.781255  6814 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:10.781584  6184 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.781985  6814 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.784686  6814 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.738262  6216 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:10.791221  6808 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:10.800869 32651 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:10.808636  6618 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.808993  6813 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:10.811043 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:10.812170 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 153543us
I20260709 12:20:10.810138 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.750239  6686 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.815471 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:10.748462  6579 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.744247  6724 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:10.789052  6704 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:10.717154  6747 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.847432 32046 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 28. Current term is 29. Ops: []
W20260709 12:20:10.837296  6623 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.734s	user 0.007s	sys 0.004s
W20260709 12:20:10.849633 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 248717us
I20260709 12:20:10.850999 31963 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:10.852026 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:10.856860  6773 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:10.857358  6773 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:10.857815  6773 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.858525  6828 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.858762  6724 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.859184  6828 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.861510  6747 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.863934 31967 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:10.864948 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.865628 31972 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:10.866729  6179 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:10.848076  6618 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.869495  6016 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:10.869889 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.870056  6016 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.866918  6675 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:10.841938  6704 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:10.879184 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.880117 32084 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:10.867215  6808 consensus_queue.cc:1059] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.008s
I20260709 12:20:10.896478 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.897169 31972 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:10.895869 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.901646 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:10.908319 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.908851 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.909511 31966 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.910741 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.911366 31969 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:10.912103 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.913553 31970 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:20:10.908926 32044 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:10.914848 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:10.915822 31967 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:10.879359  6675 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.905530 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:10.902923  6713 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:10.872473  6623 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.932873 31969 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
W20260709 12:20:10.921607 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.911s (SENT) [suppressed 8 similar messages]
I20260709 12:20:10.957252 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:10.964068 31966 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:20:10.967504 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 203584us
I20260709 12:20:10.968617 32626 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:10.970711 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.971485 32044 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:10.974349  6810 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:10.977185 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:10.977823 32113 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:10.982091  6831 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.982896  6831 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:10.984730 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:10.988897  6469 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:10.989487  6469 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.989938  6469 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:10.957813  6184 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.991601  6403 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:10.992758  6540 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.963852  6814 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:10.991415  6184 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:10.953696  6817 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:10.953956  6829 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.001145 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:10.985973  6831 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.006250 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:11.010174 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:10.967818  6179 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:10.966925  6724 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.010821 32660 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:11.018038 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:11.017720 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.018362 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.020437 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.023819 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.024088 32655 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:20:11.030208  6570 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.030912  6570 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.034096 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:11.034433 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 156614us
I20260709 12:20:11.034796 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:11.035533 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:11.036382  6844 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.036870 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.037014  6844 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.037899 31967 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:11.039568  6844 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.010053  6403 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.005825  6817 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:11.044075  6831 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:11.044245 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.045593  6540 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:11.046058  6819 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:20:11.057022 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.046236 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.047976 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:10.957016  6675 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.050235 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.004760  6704 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.051674  6815 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:11.051831 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.052325  6814 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.052083  6403 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.009092  6828 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.054080 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.056789  6831 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.042680  6016 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.054396  6570 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.049281  6709 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:20:11.056285  6817 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.056485  6618 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.054843  6844 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.065493  6808 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:11.015763  6829 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.025722  6179 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.029114  6724 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.092250 32653 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
W20260709 12:20:11.096907 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60193.1us
I20260709 12:20:11.100466 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:11.098011 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 193743us
I20260709 12:20:11.125244 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.089636  6704 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.128074 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.129166 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.140421 31969 raft_consensus.cc:2393] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:11.144102 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.144294  6432 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:11.144544 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.145078 32038 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:11.107591  6815 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:11.146261  6815 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.147303 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.147938 31972 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.110186  6540 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:11.131734 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.150697 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.154016 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.124444  6709 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.146898 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:11.156147 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 149961us
I20260709 12:20:11.174865 32117 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:20:11.172222 32085 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:11.180379 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.181113 32120 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:11.184589 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.145114  6179 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.121764  6713 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:11.189821 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:11.191635 32651 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:11.195534 32117 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:11.130179  6844 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.156852  6842 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.198285 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.175796  6623 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.198669 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.206094  6844 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.176607  6184 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.171842  6540 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.213425 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.218149 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.219327 32117 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:11.228690 32115 raft_consensus.cc:2410] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:20:11.220418 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.233260 32121 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:11.232643 31971 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:11.175153  6709 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.209677  6822 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:11.244138 31972 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:11.246520 31968 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:11.218458  6848 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:11.249595 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.220443  6844 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:11.268317  6844 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.268774  6724 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.251857  6848 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.270179 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.271747  6540 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.272166  6848 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:11.254865  6843 log.cc:927] Time spent T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:20:11.274914  6686 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.276203 32120 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:11.277345  6686 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.279785  6831 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.280395  6831 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.281002 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:11.255610  6850 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 12:20:11.256012  6834 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:20:11.261991  6829 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.263652 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.263835 31971 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:20:11.287551 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.288354 31971 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:11.289505 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.290807 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.291002  6844 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.264642 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.265617 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.265986 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.292418  6403 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.292861 31972 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25. Re-sending same reply.
I20260709 12:20:11.263864  6709 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.267342  6773 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.267115  6713 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.219774  6807 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:20:11.280913  6831 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.246402  6623 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:11.254608 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 110148us
I20260709 12:20:11.264330  6846 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:11.266481 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.294701 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.303675 31972 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:20:11.294373 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 100418us
W20260709 12:20:11.250478  6825 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
W20260709 12:20:11.297186 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 107433us
I20260709 12:20:11.297850 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.300632 31966 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:11.308215 32112 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.299885  6623 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.296039  6822 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:20:11.305315 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.307230  6773 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.296823  6675 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.308652 31971 raft_consensus.cc:2376] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18. Re-sending same reply.
I20260709 12:20:11.308594 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.212760  6842 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.312425 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.313581 31966 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:11.295630  6841 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:11.311283 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.311508  6623 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.327219 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.327937 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:11.329036 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.317456 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.318337  6831 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.318917 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.319546 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.350664 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.347548 31971 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:11.325297  6822 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:11.352213  6829 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.330768 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:11.351158 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.348007 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.332311 32043 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:11.364843 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.363269 32041 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:11.370147 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.358691 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:11.359956  6724 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:20:11.372354  6817 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.373127  6618 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:11.373562  6618 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.373960  6618 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.374899 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.375972  6016 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:11.377242  6828 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:11.377619  6828 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:11.367298 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.327679  6858 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:11.378540  6828 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.379097  6469 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.380232 32652 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:11.380295  6828 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.384217 32656 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:20:11.386653 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.331800 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.363925  6829 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.368301 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.393339 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.324609  6856 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:11.325142 32114 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
W20260709 12:20:11.388290 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57577.5us
I20260709 12:20:11.397609 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:11.398999 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:11.399778  6831 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:11.400182  6831 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.400597  6831 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:11.401360 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:11.358283  6580 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.002s
I20260709 12:20:11.392505 32117 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:11.367528 32047 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:11.394862 31972 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 15: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:11.387992 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:11.395650 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.398192 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:11.364632 32038 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:11.406196 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.407760 32041 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:11.406133 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.409184 31969 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:11.409292 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:11.410182 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.396140  6469 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.398582  6817 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.402484  6675 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:11.412865 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.397126  6856 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:11.393085  6807 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:20:11.416944 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:11.417433 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:11.417774 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 105400us
I20260709 12:20:11.408844  6743 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:11.426501  6743 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.426972  6743 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:11.432727 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:11.434060  6820 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.636s	user 0.004s	sys 0.004s
I20260709 12:20:11.436043 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.438055 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.440701 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.445822 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.418215 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.412006  6713 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:11.447201  6829 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.411087  6814 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:11.457059 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.457386  6814 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.457669 32119 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:11.457799  6814 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.460521 32116 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:11.460707  6618 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.410998  6846 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.422750  6773 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:11.463071  6773 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.463178  6618 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.463537  6773 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.465600  6842 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.422950  6856 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.486234  6856 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:11.486433 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.424055 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.424539  6859 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:11.487098 32655 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:11.487579 32114 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:11.422734  6216 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:11.488059  6216 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.488404  6579 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:11.488708 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:11.447290  6713 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.489779  6713 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.410995  6709 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:11.493435  6747 raft_consensus.cc:3055] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:11.493882  6747 raft_consensus.cc:740] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:11.487625  6859 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:11.494830  6747 consensus_queue.cc:260] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.495281  6859 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.496140  6747 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:11.493903  6709 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:11.498987  6709 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.499356  6709 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:11.500267  6833 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.455s	user 0.002s	sys 0.000s
W20260709 12:20:11.506796 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:11.448971  6839 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.440s	user 0.001s	sys 0.000s
I20260709 12:20:11.449283  6831 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:11.507999 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:11.508263  6826 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.729s	user 0.001s	sys 0.002s
I20260709 12:20:11.454262 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.509104 32118 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.423211  6675 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:11.465914 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.470827  6853 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:11.510614 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2779) took 1277 ms. Trace:
W20260709 12:20:11.471364 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44991.2us
I20260709 12:20:11.511638 31938 leader_election.cc:400] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:20:11.512157 31938 leader_election.cc:403] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:20:11.511037 32042 rpcz_store.cc:276] 0709 12:20:10.233365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.239738 (+  6373us) service_pool.cc:224] Handling call
0709 12:20:10.253261 (+ 13523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.299909 (+ 46648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.299943 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.385506 (+ 85563us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.720277 (+334771us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:10.720322 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.720401 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.507506 (+787105us) spinlock_profiling.cc:243] Waited 63.9 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:11.510280 (+  2774us) raft_consensus.cc:1685] finished
0709 12:20:11.510331 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.510559 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":202,"raft.queue_time_us":9047,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":79116,"spinlock_wait_cycles":146999168,"thread_start_us":301140,"threads_started":1,"wal-append.queue_time_us":376155}
I20260709 12:20:11.513756  6853 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 17. Current term is 18.
I20260709 12:20:11.515949  6846 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:20:11.517165  6838 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.512s	user 0.001s	sys 0.000s
I20260709 12:20:11.528630 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.529771 32045 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
W20260709 12:20:11.472779  6840 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.477s	user 0.002s	sys 0.000s
I20260709 12:20:11.415783 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:11.478261  6837 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.479s	user 0.001s	sys 0.000s
I20260709 12:20:11.473551  6332 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:20:11.482190  6818 log.cc:927] Time spent T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.484s	user 0.001s	sys 0.001s
I20260709 12:20:11.488778  6579 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:11.471835  6807 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:11.485404 32651 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:11.564010  6854 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:11.565433 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:11.535117 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 120184us
I20260709 12:20:11.534783  6332 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:11.553817  6179 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:11.573433 32045 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:11.556381 31965 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2766) took 1406 ms. Trace:
I20260709 12:20:11.556845  6813 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:20:11.580121 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66748.8us
I20260709 12:20:11.557689 32042 raft_consensus.cc:1240] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 28. Current term is 29. Ops: []
W20260709 12:20:11.574019 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 108167us
I20260709 12:20:11.576442 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:11.554749  6579 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.580063  6179 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.577849 31965 rpcz_store.cc:276] 0709 12:20:10.150150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.150283 (+   133us) service_pool.cc:224] Handling call
0709 12:20:10.176310 (+ 26027us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.198200 (+ 21890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.198251 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.211875 (+ 13624us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.672020 (+460145us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:10.672059 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.672118 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.481483 (+809365us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:11.556093 (+ 74610us) raft_consensus.cc:1685] finished
0709 12:20:11.556112 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.556304 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":213303,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12247,"spinlock_wait_cycles":44680960,"thread_start_us":459749,"threads_started":1,"wal-append.queue_time_us":695854}
I20260709 12:20:11.582453  6179 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:11.583686  6743 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.584075  6216 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.585148 32119 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:11.586352  6016 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:11.586908  6016 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:11.587190  6579 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.587872  6016 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.588299 32011 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:11.591675 32118 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:11.592847 32656 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:11.595160  6332 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.595688  6675 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.589591  6827 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:11.630610  6827 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:11.631031  6827 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.632529  6709 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.602768 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.634752 32119 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.603525 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.631031 32114 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:11.605935 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.609786 32626 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:11.610913 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:11.587890  6813 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:11.615399  6831 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.456624  6623 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.622030  6858 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.611234  6724 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:20:11.637082 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.645570 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:11.653122 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 236229us
I20260709 12:20:11.654338 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.655756  6675 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.656579  6816 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:11.657100 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.658090  6724 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:11.658190  6841 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:11.641505  6813 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:11.658452 32084 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.659384  6841 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:11.659775 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.659482  6813 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.660557  6831 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:11.660857  6813 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.650301  6618 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:20:11.664224  6618 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:11.664587  6618 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:11.665458  6618 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.665827 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.666500 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.666637 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.667243 31965 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:11.667251 32046 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:11.667639  6841 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:11.649824  6685 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:11.671172 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.672708  6469 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:11.673388 32626 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:11.674566 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.675002  6835 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:20:11.675145 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71766.5us
I20260709 12:20:11.675319  6743 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.643561  6684 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:11.676178  6684 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.676429 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.676625  6684 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.658620  6724 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:11.677053 32046 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.677845  6724 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.635052  6332 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:11.679118  6724 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.647773  6828 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:11.679922  6828 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.675158 31963 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.680359  6828 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.675614  6835 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:11.683287  6835 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.676043 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:11.692950 32119 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:11.699488 32046 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:11.688097 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:11.703531  6216 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.702054 31963 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:11.706878 31965 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
W20260709 12:20:11.700071 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49529.1us
I20260709 12:20:11.714673 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:11.716029 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61818.5us
W20260709 12:20:11.718649 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69954.1us
I20260709 12:20:11.719251 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:11.719591 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:11.673113  6623 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.729390 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:11.710803  6618 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.712945  6812 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:11.735772 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:20:11.726208  6756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.024s	sys 0.019s
W20260709 12:20:11.635715  6854 consensus_queue.cc:1175] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ec7f26ea46944ea498ad253b1b1819b0" responder_term: 25 status { last_received { term: 24 index: 16 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:11.725013  6816 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:11.745530 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69582.8us
I20260709 12:20:11.746047 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.725699  6822 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:11.727761  6767 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:11.715413  6184 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:20:11.731364  6216 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:11.718278  6403 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
W20260709 12:20:11.744035  6756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.025s	sys 0.019s
I20260709 12:20:11.742264  6684 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.771988 32656 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 20: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:11.758266  6816 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.778757  6816 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.778874  6684 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.780021 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:11.780849  6403 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:11.780766  6856 consensus_queue.cc:1059] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:11.781385  6403 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:11.782330  6403 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.783684  6403 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:11.785933 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.788211  6675 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:11.747735  6709 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.742934  6831 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:11.733040  6743 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:20:11.766986  6859 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:11.799909 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.800727 32113 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.747375  6773 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.737028  6713 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.793346  6704 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.828760 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:11.829623 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.831774 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.832793 32042 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:11.832619 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.834650 32115 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:11.831676 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.836470 32120 raft_consensus.cc:2393] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:11.822337  6415 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:11.795073  6179 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.800622  6831 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:11.802071  6866 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:11.825011  6859 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:11.841931  6432 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.848676  6684 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:11.849129  6815 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:11.849819  6870 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:20:11.849905  6853 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:11.851163 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.852254  6713 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.852313 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.853365 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.853880  6773 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.854519 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.856413 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.842521 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.889021 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.889870 31969 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:11.866792 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.835273  6704 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.868793 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:11.890547 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:11.869387 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 142806us
I20260709 12:20:11.869580 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:11.891306 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.825861  6814 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:11.892489 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62056.6us
I20260709 12:20:11.865659 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.880400 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.842447  6415 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:11.885303 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:11.879257  6745 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:11.895495 32112 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:11.897884 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.895493 31966 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:11.894959 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:11.900990 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:11.906582 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:11.907243 31965 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25. Re-sending same reply.
I20260709 12:20:11.908560  6179 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:11.910825  6867 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:20:11.912034  6859 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.891559  6853 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.915802 32112 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:11.920038 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.920773 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.935643 32042 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.936355 32115 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:11.937973  6853 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:11.939348  6828 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.942924 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:11.943318  6828 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.884661  6684 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.923703 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.923332  6831 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.925114 32118 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:11.925594  6814 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:11.929590 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.935714 32045 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.952205 32117 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:11.921808 32047 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:11.936246 32040 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.945807 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:11.922726  6704 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:11.955626  6816 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.885192  6432 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:11.959671  6432 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:11.960072  6756 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:11.960387 31963 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:11.898216  6745 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:11.947685 32044 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:11.965807 32045 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
W20260709 12:20:11.968676 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.971834 32117 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.972795 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.973553 32121 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:11.970383 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.896652  6827 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.894979  6815 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:11.981850 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.960518  6756 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:11.945906  6879 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:11.985195 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:11.988852  6877 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.990265  6874 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:11.993103  6724 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:11.994663 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:11.995069 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:11.995780 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.996153  6828 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:11.997711 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.033025 32038 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:12.002008 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:11.994719 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.034180 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.004647  6815 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:12.010145  6827 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:12.010758 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:12.017982 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:12.019099 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:12.025784 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.038239 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.044646 32656 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
I20260709 12:20:12.044926 32040 raft_consensus.cc:2393] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:12.045423 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.049177 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:12.050246 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.022298  6879 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:12.055383 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 62663.3us
I20260709 12:20:12.054260 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:12.064039 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66767.9us
I20260709 12:20:12.066582  6827 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:12.068109 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.070053  6402 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:11.969223  6415 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.042532  6866 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:12.043463  6813 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:12.070016 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.005970  6745 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.094269 32046 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:20:12.048501  6815 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:12.083132 32042 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:12.097421 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.116374  6756 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.129964  6684 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:12.131419  6745 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.135720 32046 raft_consensus.cc:1240] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:12.136620 32042 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:12.052148  6432 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.053975  6816 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:12.144311 32042 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:20:12.118158  6756 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.165925  6878 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:12.168591  6857 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.009s	sys 0.012s
W20260709 12:20:12.169087  6857 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.009s	sys 0.012s
W20260709 12:20:12.170408 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73062.2us
I20260709 12:20:12.173039 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.173907  6618 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:12.175348  6618 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:12.178088  6618 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.178874  6879 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.183118 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.184026  6885 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:12.033052  6877 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.134691  6881 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.191159 32046 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 25. Current term is 26. Ops: []
W20260709 12:20:12.194268 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 126804us
I20260709 12:20:12.054780  6814 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:20:12.198221 32121 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:12.204937 32041 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.205983 32046 raft_consensus.cc:1240] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:12.209473 32114 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:12.211516 32113 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:12.212698 32046 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:12.218575  6812 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:12.220227  6812 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.225579 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.226127 32045 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:12.226974 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.206527 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:12.229843 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.231628 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:12.235683 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.239764 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.242285 32041 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:12.242038 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.245287 31965 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:12.219134  6822 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:12.264832 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58432.3us
I20260709 12:20:12.247309 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.243520 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:12.265368 32010 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.265949 32084 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:12.249380 31970 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:12.267654 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.268209 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.251724 32119 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:20:12.254529 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.270834 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.256429  6877 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.235556  6859 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.239691  6874 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:12.240072  6570 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:12.273976  6831 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:20:12.278298  6570 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:12.278750  6570 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.274170  6713 raft_consensus.cc:2804] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:12.280117  6713 raft_consensus.cc:697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:12.280907 31966 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:12.281054  6713 consensus_queue.cc:237] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.274267  6885 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:12.274693  6704 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:12.282896  6859 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.274932  6469 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:20:12.262460  6685 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:12.262811  6875 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.277742  6743 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.247613  6858 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:20:12.272094 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:12.248823  6812 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.274103 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.274343 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.275245 31972 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:12.277009 32045 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:12.273871  6841 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:12.277544 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:12.270200  6184 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.239399  6816 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:20:12.232777  6881 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:20:12.291239 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.295179 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:12.264796  6822 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:12.307446 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.296032 31971 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:12.285768 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.308615 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.309187 31963 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:12.304231 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.305637  6842 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:12.297510  6853 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.309430 31972 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:12.287204  6685 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.311350 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.312654 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:12.286818  6885 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:12.318856 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.324769 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.313154  6812 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:12.313611  6743 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:12.306906  6841 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:12.314841  6724 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:12.313934  6685 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.298547  6773 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:20:12.332295 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.315064  6866 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.314860  6432 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:12.336364 32040 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:12.322131  6875 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:12.336246  6724 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:12.331250  6469 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.343657  6856 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:12.340399  6815 consensus_queue.cc:1059] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:20:12.344471  6878 consensus_queue.cc:1059] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:12.344695  6859 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:12.343948  6724 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.345265  6580 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:12.346578 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:12.347172  6580 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:12.347574 32656 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:12.347983  6580 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.342299  6846 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:20:12.348937 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:12.349192  6580 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.331503  6812 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:12.351027  6842 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:12.352561  6842 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.352111  6822 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.354250  6842 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.354998  6822 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.356362 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.353873  6856 consensus_queue.cc:1059] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:12.358176 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.358865 31967 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:12.358961 31963 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:12.328474  6745 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:20:12.351359  6016 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:12.354398  6432 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:20:12.360893  6579 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:12.350546  6815 raft_consensus.cc:3055] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:12.380496 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.352756  6812 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.384573 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.364429  6016 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:12.391155  6016 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 21.19, Last appended by leader: 19, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.392361  6016 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.392449 31966 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:12.393565 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.394271 31966 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:12.400242 31970 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:20:12.400837 31970 pending_rounds.cc:85] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 10
I20260709 12:20:12.401119 31970 pending_rounds.cc:107] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:20:12.401458 31970 raft_consensus.cc:2889] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:12.385812 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.402648 32011 leader_election.cc:400] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
I20260709 12:20:12.402967 32011 leader_election.cc:403] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:20:12.405395  6874 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.406086 31968 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:12.408027  6570 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.427471 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.428139 31968 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.429836 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.430704 32047 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:20:12.431118 31963 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:12.431877 32045 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:12.390149  6618 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.385351  6815 raft_consensus.cc:740] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:12.411046  6469 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:12.444190 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.410286  6570 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.445369 31968 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:20:12.450981 32042 raft_consensus.cc:1240] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:20:12.456391 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.460940 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.579267 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:12.582293 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 121452us
I20260709 12:20:12.584117  6570 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.376950  6885 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.601539  6889 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.602430  6889 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.427413  6841 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.605748  6889 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:12.624969 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.471556  6831 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.452132  6469 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.660331  6684 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:12.633766  6815 consensus_queue.cc:260] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.666199  6815 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:12.667240  6876 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:12.667783  6876 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:12.674892  6853 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:12.675675  6853 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.680536  6709 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.681249  6709 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.684700  6878 raft_consensus.cc:3055] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:12.685264  6878 raft_consensus.cc:740] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:12.641772  6831 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.686305  6878 consensus_queue.cc:260] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.692097  6878 raft_consensus.cc:3060] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.695479  6709 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:12.697346  6469 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.668861  6876 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.700357  6812 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.701476  6853 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.703120  6812 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:12.705551 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:12.705955 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:12.706350 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.707871 32042 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:20:12.709916 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.712133 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.651899 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.655396 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.715696 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:12.716400 32042 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:12.718537 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.721141 32042 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:20:12.589576 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.723726 31970 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 12:20:12.725540 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73753.4us
I20260709 12:20:12.726497 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:12.728377 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:12.728520 31969 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:12.745508 32044 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:12.716029  6874 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
I20260709 12:20:12.752096 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:12.753396 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:12.753470 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:12.754998 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:12.740550  6853 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:12.758502  6812 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.759195  6812 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.762552  6773 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:12.764458  6773 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:12.764801 32047 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:20:12.764915  6773 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.765214  6807 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:12.755657 32652 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:12.748457  6831 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:12.745798  6623 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:20:12.767161  6579 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:12.777335  6773 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:12.809345 31969 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:20:12.799635  6893 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:20:12.798116  6432 consensus_queue.cc:1243] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 26.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:12.833150 32121 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:12.835016 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:12.835867 32121 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
W20260709 12:20:12.842510 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 187581us
W20260709 12:20:12.842543 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 114214us
W20260709 12:20:12.844911 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 417189us
I20260709 12:20:12.854136 32119 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:12.851967 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:12.857846 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.868060  6816 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:12.868706  6816 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:12.868417 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.869791  6816 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.871387  6835 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.872628 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:12.868037 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:12.804752  6853 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:20:12.885650 32120 raft_consensus.cc:1240] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:12.878338 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:12.855353  6685 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.510s	user 0.005s	sys 0.004s
I20260709 12:20:12.885291 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:12.889544 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:12.894899 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:12.878645  6835 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:12.901489  6897 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:20:12.903271  6906 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.906304 32113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 3073) took 1022 ms. Trace:
I20260709 12:20:12.914799 32113 rpcz_store.cc:276] 0709 12:20:11.883494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.883666 (+   172us) service_pool.cc:224] Handling call
0709 12:20:11.891184 (+  7518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:12.218333 (+327149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.906076 (+687743us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540043b2d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:12.906246 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736292224}
I20260709 12:20:12.908836  6856 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:12.916314  6856 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:12.919518  6822 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:12.919940  6822 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:12.920848  6822 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:12.925192  6580 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 12:20:12.930472 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:12.932103  6909 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:12.944411 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:12.945317 32652 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 29.
I20260709 12:20:12.946281  6841 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:20:12.949087  6901 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:20:12.959789 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.961951  6853 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.962261 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:12.962550  6675 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:12.962950  6898 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:12.963524  6898 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.966010  6876 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:12.966656 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:12.967422 32116 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:12.966027  6898 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.962694  6853 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.962962  6675 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:12.970184 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:12.970341  6675 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.971417  6675 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.972373  6853 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:12.970836 32117 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 29.
I20260709 12:20:12.894208  6685 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.976012 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:12.976608 32652 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:12.977200  6685 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:12.977980  6831 consensus_queue.cc:1175] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3836f9b5cd36488da56d4540dc9d80ec" responder_term: 22 status { last_received { term: 20 index: 13 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.14-21.14]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:12.978628 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:12.979840  6713 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:12.980259  6713 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.980647  6713 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:12.986517 31971 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:12.987497  6432 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:12.987968  6432 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
W20260709 12:20:12.988422  6724 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.000s
I20260709 12:20:12.988907  6432 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.995029  6432 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:13.001945 31970 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:12.993499 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:12.967599  6856 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.003374  6876 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:13.003890  6876 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.004346  6876 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:12.988910  6724 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:12.995937 32117 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:20:13.010461  6724 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:12.996887  6415 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:13.014070  6876 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:12.999055 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:12.992301 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:13.024019  6579 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:13.024537  6876 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:13.022197 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.024752  6580 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:13.028508  6580 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.024034  6905 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:20:13.029250  6905 consensus_queue.cc:1243] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:20:12.922050  6822 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:13.032227  6579 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:13.034463  6580 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:12.958230  6842 consensus_meta.cc:287] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.603s	user 0.000s	sys 0.007s
I20260709 12:20:13.035398  6842 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.038751 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:13.039322 32656 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:13.039494  6853 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:13.039939  6853 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:13.040848  6853 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:13.035907  6908 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.139s	user 0.000s	sys 0.003s
I20260709 12:20:13.042804 31967 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 12. (term mismatch)
I20260709 12:20:13.043228 31967 pending_rounds.cc:85] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:20:13.043435 31967 pending_rounds.cc:107] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 25.12
I20260709 12:20:13.043514  6842 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:13.043848 31967 raft_consensus.cc:2889] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:13.045468  6856 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:13.042053  6853 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:13.046376  6877 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:13.046964  6877 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.049130  6877 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:13.033764  6579 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.056231  6580 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.063167 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:13.065450 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:13.065773 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:13.067314  6402 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.067854  6402 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.068584  6885 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:13.069528 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:13.069991  6402 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.070803 31963 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:13.071084 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:13.077402 31967 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:12.997279  6713 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.080597 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:13.081676 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:13.085932 32656 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:13.178025 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:13.082475  6402 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.195258 31970 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 26: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:13.188421 32044 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:13.191160  6415 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.087332 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:13.076629 32011 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:13.190371 32625 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.191602 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.191720 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.212702 32120 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:13.193125 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 110342us
I20260709 12:20:13.194124 32117 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:13.208643 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:13.214020 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 126808us
I20260709 12:20:13.196043  6402 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.207181 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:13.219096 32118 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:13.219274 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:13.190604  6822 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:13.215467 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.217451 32112 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:13.189688 32047 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:13.220482 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:13.227026  6907 consensus_queue.cc:1059] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:13.226790  6822 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.225914 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.231601 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:13.235138 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 154623us
I20260709 12:20:13.236647 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:13.241323 31938 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:13.256506 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 179941us
I20260709 12:20:13.228018  6905 raft_consensus.cc:3055] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:13.224459  6846 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:13.219357  6216 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.195617  6773 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.237258  6907 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:13.269073  6858 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.014s	sys 0.058s
I20260709 12:20:13.244563  6743 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:13.223480  6415 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.295558 32118 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 26.
I20260709 12:20:13.273273 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:13.244604  6900 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.244180  6402 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.271742  6877 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:20:13.244550  6016 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:20:13.279246  6898 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.270792  6905 raft_consensus.cc:740] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:13.332711 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:13.332789  6905 consensus_queue.cc:260] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.333715 32652 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:13.284351  6724 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:13.334694  6724 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:13.334741 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:13.335486 32656 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:13.335871  6724 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:13.325074 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51886.7us
I20260709 12:20:13.325685  6907 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:13.339021 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:13.314863  6877 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.330603  6580 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:13.282869  6846 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:13.341619 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.331291  6713 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:13.343613 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:13.353610 32121 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:20:13.354247 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 499109us
I20260709 12:20:13.283407  6216 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.314270  6743 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:13.361433  6743 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.362766  6216 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:13.351310  6854 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.367084 31963 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:13.367594  6846 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.369721  6853 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.370361  6853 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:13.381668 31997 heartbeater.cc:646] Failed to heartbeat to 127.30.25.62:33695 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.30.25.62:33695 timed out after 15.000s (SENT)
I20260709 12:20:13.381729  6858 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.383190  6858 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.384784  6907 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:13.385949  6858 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:13.388180  6773 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:13.345477  6877 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:13.391176 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:13.391980  6856 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.392457  6856 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.394876 32656 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:13.396726  6907 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:13.403211 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:13.407649 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.414958 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:13.415680 31972 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:13.419304 31966 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:13.414569  6878 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:13.421295  6858 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:13.421860  6858 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:13.423156  6685 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.423699  6685 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.357091  6905 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:13.425690  6415 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.426434  6858 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.432551 32651 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:13.357307 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:13.434875 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:13.435494 32042 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:13.445654 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.446322 32114 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:13.447655 32045 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:13.448179 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:13.448767 32039 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:13.449317 32041 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:13.449868 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:13.434093  6907 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:13.453012 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:13.425828  6685 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:13.483702 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.470849  6432 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:13.493650 32042 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:13.485942 31972 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:13.454088  6773 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:13.470738  6856 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.458449  6907 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.499648 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:13.501363 31963 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:13.510295 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:13.478996  6842 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:20:13.514151 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:13.506050 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:13.514881 32043 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:13.516423 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:13.517280 32045 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:13.510927 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:13.513000 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:13.491110  6432 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.518339 32044 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:13.521929 31972 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:13.523917 31963 raft_consensus.cc:2410] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:13.504232 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:13.523932 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:13.530866 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.533344 31966 raft_consensus.cc:2410] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:20:13.535158 32114 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:13.510608  6853 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:13.536593  6709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.016s	sys 0.038s
I20260709 12:20:13.500324  6216 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:20:13.524034  6432 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:13.510957  6713 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:20:13.542403 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.543467 31864 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:13.540313  6841 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:13.554834  6841 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.555219  6841 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:13.559528 31865 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 233) took 15698 ms (client timeout 15000 ms). Trace:
I20260709 12:20:13.548054  6216 consensus_queue.cc:1243] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:20:13.541908  6709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.016s	sys 0.039s
I20260709 12:20:13.564318 32043 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
W20260709 12:20:13.567860 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.563843 31864 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 17 to 18, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 18 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.587563 31963 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:20:13.588858 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 370762us
W20260709 12:20:13.589501 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 333780us
I20260709 12:20:13.590081 32626 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:13.592439 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 332612us
I20260709 12:20:13.588095 31864 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 20. New cstate: current_term: 20 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.594512 31864 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:13.595753 31864 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 23 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:13.597328 31864 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 20. New cstate: current_term: 20 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.598980 31864 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 21, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 21 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:13.615551 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 355943us
I20260709 12:20:13.600410 31864 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 22, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:13.619081 31864 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 22, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.620708 31864 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 21, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 21 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.536816  6773 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:20:13.636875 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:13.637637 32118 raft_consensus.cc:2368] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
W20260709 12:20:13.561484 31865 rpcz_store.cc:269] 0709 12:19:57.861557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.861723 (+   166us) service_pool.cc:224] Handling call
0709 12:20:13.559415 (+15697692us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:11.030910 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:11.031704 (+   794us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:11.031768 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:20:11.031784 (+    16us) tablet.cc:662] Decoding operations
0709 12:20:11.033687 (+  1903us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:11.033813 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:11.033882 (+    69us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:20:11.206722 (+172840us) tablet.cc:701] Row locks acquired
0709 12:20:11.206775 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:20:11.206944 (+   169us) write_op.cc:270] Start()
0709 12:20:11.207097 (+   153us) write_op.cc:276] Timestamp: P: 1783599611206877 usec, L: 0
0709 12:20:11.207131 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:20:11.207940 (+   809us) log.cc:844] Serialized 6228 byte log entry
0709 12:20:11.789301 (+581361us) op_driver.cc:464] REPLICATION: finished
0709 12:20:11.926224 (+136923us) write_op.cc:301] APPLY: starting
0709 12:20:11.926344 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:12.150044 (+223700us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:12.150198 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:12.627550 (+477352us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:12.627945 (+   395us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:20:12.628007 (+    62us) write_op.cc:312] APPLY: finished
0709 12:20:12.628471 (+   464us) log.cc:844] Serialized 148 byte log entry
0709 12:20:12.628671 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:12.628854 (+   183us) write_op.cc:454] Released schema lock
0709 12:20:12.628934 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:12.629379 (+   445us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:12.629414 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3049107,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":136626,"apply.run_cpu_time_us":26103,"apply.run_wall_time_us":703119,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70943,"num_ops":16,"prepare.queue_time_us":245521,"prepare.run_cpu_time_us":35635,"prepare.run_wall_time_us":293722,"raft.queue_time_us":367468,"raft.run_cpu_time_us":12573,"raft.run_wall_time_us":110264,"replication_time_us":581986,"thread_start_us":1103095,"threads_started":4,"wal-append.queue_time_us":139615}]]}
I20260709 12:20:13.631102  6016 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:13.634354 32121 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:13.641649 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.643589 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:13.576061  6216 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:13.650014 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:13.656211 31864 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 19, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:13.652755 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.654999 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:13.689544 31963 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:13.659840 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.690723 32116 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:13.694691 32118 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:13.696242 31864 catalog_manager.cc:5697] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 17, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 17 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:13.698581 31864 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.700515 31864 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 20, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 20 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.702278 31864 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.615417  6831 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:13.576519  6623 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:20:13.626362  6403 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:20:13.692021  6829 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.019s	sys 0.047s
W20260709 12:20:13.688277  6579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.007s	sys 0.020s
I20260709 12:20:13.716367  6685 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.709980  6842 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.622434  6909 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.087s
I20260709 12:20:13.646270  6822 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:13.705988  6216 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.816534 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:13.820539 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.776000  6403 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:13.745051 31864 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 18, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 18 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:13.781280 32039 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
W20260709 12:20:13.764215  6829 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.019s	sys 0.048s
I20260709 12:20:13.813891  6842 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.852977 32117 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:13.853875  6842 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:13.770145  6897 log.cc:927] Time spent T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:20:13.771847  6579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.009s	sys 0.022s
W20260709 12:20:13.855250  6925 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:13.855438 31965 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:20:13.843217 31864 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 24, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 24 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.801333  6846 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.862054  6846 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.864645  6846 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:13.866189 31864 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 17 to 24. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:13.882071  6579 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.882694  6579 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.874743 31864 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 20, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 20 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.884848  6579 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.886204 31864 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:13.887485 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:13.776691  6831 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.818758  6907 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.899044 32118 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:13.888525  6846 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.900876 31864 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.902137 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:13.890516  6773 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.813355  6685 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.904070  6874 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:13.901453  6877 consensus_meta.cc:287] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
W20260709 12:20:13.917337 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 382556us
I20260709 12:20:13.925567 31864 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:13.923659 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:13.933362 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.916855  6907 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:13.929544 31864 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 18, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 18 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:13.936115 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.920688  6854 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:13.909296  6846 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.945106 31936 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:13.925981  6874 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.941923  6907 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.941787 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:13.920182  6773 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.933792  6709 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:13.931085  6877 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:13.962924 31864 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:13.951292  6216 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:13.987712 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 342641us
I20260709 12:20:13.962357  6469 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:13.995900 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:13.983342  6907 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:13.970620  6899 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:13.970148  6905 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.001827 31864 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:13.989389  6709 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.967192  6854 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:13.982559  6724 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:20:14.034380 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:14.031217 31864 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 19, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 19 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:14.032646  6709 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.018097  6469 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:14.052126  6936 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:14.060334  6899 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:14.064368  6831 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:14.012665  6618 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.074019  6469 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:14.030033  6216 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:14.031805  6905 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:14.077193  6841 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.001s
I20260709 12:20:14.085173  6854 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:13.952195  6403 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:14.087106 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48162.9us
I20260709 12:20:14.087492  6773 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:14.026098  6931 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.089432  6846 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:14.090852  6685 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:14.025722  6179 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:14.158193 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:14.158859 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:14.095969 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:14.122637 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 100440us
I20260709 12:20:14.149528 32656 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:20:14.124521  6934 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:20:14.153046 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:14.153093  6937 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:20:14.154161  6905 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:14.124819  6899 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:14.123159  6618 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.158888 32658 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:20:14.175601 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:14.165980  6432 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.001s
I20260709 12:20:14.143394  6216 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:14.166029  6179 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.188973  6179 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.143903  6841 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:14.197711 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 256080us
I20260709 12:20:14.184526  6685 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:14.201447 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3044) took 1802 ms. Trace:
W20260709 12:20:14.199323 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 168235us
W20260709 12:20:14.205286 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 170938us
I20260709 12:20:14.168823  6931 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.214493 31970 pending_rounds.cc:85] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 12
I20260709 12:20:14.214826 31970 pending_rounds.cc:107] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:20:14.215065 31970 raft_consensus.cc:2889] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:14.215613 31972 raft_consensus.cc:1275] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:14.180740  6773 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:20:14.201920 32046 rpcz_store.cc:276] 0709 12:20:12.398572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.398681 (+   109us) service_pool.cc:224] Handling call
0709 12:20:12.398999 (+   318us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:20:12.403846 (+  4847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:12.403881 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:20:12.404614 (+   733us) log.cc:844] Serialized 78 byte log entry
0709 12:20:12.717360 (+312746us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:13.202696 (+485336us) log.cc:844] Serialized 14 byte log entry
0709 12:20:13.258529 (+ 55833us) log.cc:844] Serialized 14 byte log entry
0709 12:20:13.339035 (+ 80506us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b18004b8ec0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:20:13.648611 (+309576us) log.cc:844] Serialized 14 byte log entry
0709 12:20:14.198525 (+549914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.200865 (+  2340us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:14.201129 (+   264us) raft_consensus.cc:1685] finished
0709 12:20:14.201141 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:14.201286 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117964,"prepare.queue_time_us":718583,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":102,"raft.queue_time_us":284870,"raft.run_cpu_time_us":465,"raft.run_wall_time_us":19731,"spinlock_wait_cycles":57020672,"thread_start_us":1648881,"threads_started":4,"wal-append.queue_time_us":364040}
I20260709 12:20:14.229065 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.229965 31972 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:14.204917  6685 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.239687 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.240355 31972 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:14.187415  6432 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.244293 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.244621 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.245983 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.246608 31972 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:14.251118 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:14.251653 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.252535 32040 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:20:14.252852 31972 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 15.
I20260709 12:20:14.259776 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.260474 31972 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:14.215807  6580 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.264604 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:14.229835  6874 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:14.268944 32039 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:14.229113  6469 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:14.249975  6685 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:14.273824 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:14.276492 31964 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:14.277252 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:14.332785  6957 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 12:20:14.334267  6958 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:20:14.339120  6961 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:20:14.273563  6580 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:14.350703  6960 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:20:14.542539 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:14.542817 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.543490 31966 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
W20260709 12:20:14.373076  6959 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:20:14.282624  6469 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.256913  6899 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.258601  6216 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.536151 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.540439 31963 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:14.546780  6906 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.563436 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:14.591382 32039 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:14.524698  6831 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.525100  6966 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:14.583917  6907 consensus_meta.cc:287] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.551s	user 0.005s	sys 0.012s
I20260709 12:20:14.541260  6905 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.548677  6929 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.546238  6579 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:14.555536  6967 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:14.635916  6959 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:20:14.712109  6982 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.712743  6982 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.713919  6970 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.714627  6970 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.582850  6876 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:14.581351  6415 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:14.651330 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 56847.7us
I20260709 12:20:14.705519  6841 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:14.707935  6709 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.004s
I20260709 12:20:14.635572  6907 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.708104  6685 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.711514  6967 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.629176  6618 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:14.631820  6966 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.633621  6831 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.642249  6905 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.723523  6981 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.747977 32084 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:14.704481  6909 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.636217  6179 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.750945  6877 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:14.628136  6906 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.744431  6432 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:14.731971  6876 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.721935  6579 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.644878  6929 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.750648  6978 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.732396  6415 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:14.758436 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:14.839067 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 91161us
I20260709 12:20:14.731743  6709 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.806123  6931 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:14.806015  6713 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.807689  6916 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.828537 32013 leader_election.cc:400] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23.
I20260709 12:20:14.853017 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:14.842216 32046 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:14.828248  6618 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:14.843135 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 252558us
I20260709 12:20:14.839946 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:14.814414  6933 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:14.849704 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:14.854730 32120 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:14.856364 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.858685 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:14.814234  6773 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:14.874910 31963 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 26.
I20260709 12:20:14.882632 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:14.894646 31966 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:14.851383  6920 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:14.848712  6986 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.904281 31963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:14.906050 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:14.906764 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:14.895460  6975 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:20:14.906189 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:14.879499  6846 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:14.907296 31963 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:14.935283 31963 raft_consensus.cc:614] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:14.816561  6981 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.863312  6979 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:14.938513  6979 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:14.906169  6926 log.cc:927] Time spent T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 1.283s	user 0.008s	sys 0.000s
W20260709 12:20:14.933311 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50812.1us
I20260709 12:20:14.866077  6931 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:20:14.912266  6184 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:14.885241  6950 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.901497  6841 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.862881  6713 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.903048  6469 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:14.921983 31966 raft_consensus.cc:2393] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:14.902029  6835 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.940778  6965 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.899392  6773 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.930644  6580 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:14.959986  6985 log.cc:927] Time spent T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:20:14.822485  6909 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.867817  6916 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.944707  6989 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:14.851102  6978 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.967336  6579 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:14.877894  6933 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.874248  6618 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:14.922528  6986 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:14.928217  6724 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:20:14.975528 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3153) took 2076 ms. Trace:
I20260709 12:20:15.025177  6179 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:15.027714 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 121619us
W20260709 12:20:14.962947  6974 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:20:14.944844  6016 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:14.962263  6977 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:20:14.968340  6907 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:14.968290  6979 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:14.963101  6831 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:14.970268  6432 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.028761  6743 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:15.045624  6972 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 12:20:15.024659  6990 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.050243  6623 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.075141 32042 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:15.066419 31969 rpcz_store.cc:276] 0709 12:20:12.899799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.899961 (+   162us) service_pool.cc:224] Handling call
0709 12:20:12.900717 (+   756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:12.958812 (+ 58095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:12.958845 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:12.985603 (+ 26758us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b2c000972c8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:20:12.986022 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 12:20:13.345812 (+359790us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:13.387330 (+ 41518us) log.cc:844] Serialized 14 byte log entry
0709 12:20:13.387531 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:13.387602 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:14.745809 (+1358207us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:14.975308 (+229499us) raft_consensus.cc:1685] finished
0709 12:20:14.975332 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:14.975470 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57688,"prepare.queue_time_us":40704,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":58862,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":184,"spinlock_wait_cycles":63657472,"thread_start_us":359382,"threads_started":1,"wal-append.queue_time_us":548720}
I20260709 12:20:15.025458  6841 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:15.110318 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 187751us
W20260709 12:20:15.081228  6403 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.011s	sys 0.004s
I20260709 12:20:14.974282  6950 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.063344  6992 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.063745  6685 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.043277  6835 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.093079  6970 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:15.078488 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 199158us
I20260709 12:20:15.111883 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.094789  6956 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.043910  6965 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.119831 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.132298 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.132947 32047 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:15.120128 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.107723  6876 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.121903 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.086851  6432 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.122577 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.086474  6743 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.067819  6987 consensus_queue.cc:1048] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:20:15.143025 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.143719 32039 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:15.137109 31969 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:15.107522  6967 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.148226 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.149066 32044 raft_consensus.cc:2393] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:15.109238  6016 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.134835 32045 raft_consensus.cc:2368] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24.
I20260709 12:20:15.135144 32038 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:15.054451  6989 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.138972 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.128273 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.133962 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:15.118992  6403 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.011s	sys 0.004s
I20260709 12:20:15.125911  6931 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.135339  6743 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.175884 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.159647 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:15.177543 32038 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:15.109902  6623 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.160346 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:15.161659 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.132812  6822 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.138268 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.189967 32039 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 26: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:15.127101  6846 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:15.163843 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.158749 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:15.210125 32045 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.173082 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.135847  6983 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.143342  6979 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:15.135478  6991 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.129s
I20260709 12:20:15.135965  6954 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.158629  6709 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.206763 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.172483  6415 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.158255  6929 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.209406 31970 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:15.180732  6931 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.130587  6992 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:15.216161 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52415.4us
I20260709 12:20:15.110662  6990 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.136411  6432 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.162290  6016 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:15.215504 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 185697us
I20260709 12:20:15.297143  6713 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.217177  6979 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.144004  6956 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:15.240406 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 290645us
W20260709 12:20:15.316917  6993 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
I20260709 12:20:15.207490  6822 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.299921 32045 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:15.226639  6184 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:15.304237 32038 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:15.343226 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:15.308118  7001 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:20:15.308364  7009 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:15.318099  6841 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.348544  7009 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.349356  6905 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.354107  6916 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:15.355695  6996 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:15.357110  6992 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.357784  6931 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.312484  6933 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.308444  7012 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:15.313534  6906 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.314888  6835 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:20:15.359100  7012 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.303829  6909 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.316669 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.359524  7012 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.329437  6989 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:15.363178  7013 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:20:15.363416  6965 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.368500 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.235102 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.308159  7007 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.217293  6842 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:15.218622  6983 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.217187  6846 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.323375 32651 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:20:15.228153 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 98448.9us
I20260709 12:20:15.380880 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.325609 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:15.327673  7006 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:20:15.328186  6950 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.328591  6986 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.328990  6956 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.329313  6016 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.225306  6954 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.323078  6979 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.332748 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.334554 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.404520 32118 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:15.405735 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.334895 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:15.336566 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:15.407249 32116 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:15.407522 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.307616  7005 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.344885 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.410205  7005 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.390431  7011 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.395340  6989 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:15.412602  7005 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.346798  6822 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.412652  7011 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.412388 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.413587 31971 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:15.347695  6713 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.357244  6966 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.242918  6773 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.414422  6713 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.369359 32112 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:15.416570 32120 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:15.416656  6713 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.295439  6920 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:15.377554  7007 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.418889  6920 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.384122 32652 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:15.420986  7007 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.423065 32043 raft_consensus.cc:2410] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:20:15.386916 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:15.388093 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 153095us
I20260709 12:20:15.388204 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.424495 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.378774  7009 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.376513  6967 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.436499  6967 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.439229  6967 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.400242  6846 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.400547  6983 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.388521  6909 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:15.440676  6909 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:15.441228  6909 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:15.442656 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.409554 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.443344 32047 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:15.443477  6982 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.444391 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.444906  6978 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.445648 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.447141  6981 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.447693  6981 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.448006 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.448137  6981 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.449047 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.345755 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2848) took 1014 ms. Trace:
I20260709 12:20:15.412696  6989 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.433862  7020 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:15.435750  6623 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.444350 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.452620 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.453559 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.454645 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.450060 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.456048 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.410632 32653 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:15.466776 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.467585 31967 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:15.464802 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.472985 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:15.475418  6981 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.476392 32044 raft_consensus.cc:2393] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:15.481434 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.450834 32040 rpcz_store.cc:276] 0709 12:20:14.331737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.545273 (+213536us) service_pool.cc:224] Handling call
0709 12:20:14.583254 (+ 37981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:14.631841 (+ 48587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:14.631878 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:14.646145 (+ 14267us) log.cc:844] Serialized 26 byte log entry
0709 12:20:14.852900 (+206755us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:14.852941 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.853013 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:15.316346 (+463333us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:15.345415 (+ 29069us) raft_consensus.cc:1685] finished
0709 12:20:15.345442 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:15.345687 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48121,"raft.queue_time_us":126496,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":108,"spinlock_wait_cycles":12759808,"thread_start_us":115068,"threads_started":1,"wal-append.queue_time_us":199947}
I20260709 12:20:15.485759 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.478118 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.500576 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.502934 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.503739 32119 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:15.502151 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.486352 31965 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:15.403240  6954 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.515612 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 30.
I20260709 12:20:15.470340  6989 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:15.511058 32045 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:15.470084  7020 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.489264 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.415041  7011 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.509392 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.525579 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.518105 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.516044 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.526621 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.525704 32119 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:15.520809 31966 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.531512 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.532948 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.534020 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.539845  6916 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:15.542419 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.542351  6916 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:15.543056 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:15.543629 32047 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.545610 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.546602 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.560951 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.562173 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.563509 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.565495  6990 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.565711  6579 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.566305  6579 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.568918  6579 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.524703  7020 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:15.570780  7002 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.458s	user 0.000s	sys 0.002s
I20260709 12:20:15.514650  7007 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:15.548053  6916 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:15.593281  7024 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 12:20:15.594532 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:15.595197 31964 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2817) took 2087 ms (client timeout 2555 ms). Trace:
I20260709 12:20:15.596326 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.548404 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.477576  6743 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.601101 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.553567  7009 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:15.554821  6955 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:15.556674 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.571980 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3176) took 1037 ms. Trace:
I20260709 12:20:15.578241 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.578811 32047 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:15.579818 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.612597 32045 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:15.582522 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.547328 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.554801 31967 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:20:15.584254 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.584204 32119 raft_consensus.cc:2410] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:20:15.588132 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.590436 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.599000 31970 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:15.602442 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:15.595717 31964 rpcz_store.cc:269] 0709 12:20:13.499521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.500088 (+   567us) service_pool.cc:224] Handling call
0709 12:20:15.586551 (+2086463us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b540042fed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:15.586727 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1748474240}
I20260709 12:20:15.608513 31966 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:15.613377 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:15.624850 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.625640 32656 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:15.612709 31972 rpcz_store.cc:276] 0709 12:20:14.534562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.534694 (+   132us) service_pool.cc:224] Handling call
0709 12:20:14.535048 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:14.606987 (+ 71939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:14.607022 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:14.607525 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:20:15.070713 (+463188us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:15.070758 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.070839 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:15.568479 (+497640us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b2400177428. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2288f689c4 00007f2288f66acb 00007f2289121dc9 00007f2289121a55 00007f2289120fb0 00007f228bd97488 00007f228bd97be8 00007f228bd6f398 00007f228bd6a76a 00007f228bd69bb6
0709 12:20:15.571167 (+  2688us) raft_consensus.cc:1685] finished
0709 12:20:15.571236 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:15.571919 (+   683us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49537,"raft.queue_time_us":177604,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":5537024,"thread_start_us":484256,"threads_started":2,"wal-append.queue_time_us":507880}
I20260709 12:20:15.626279 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.627192 32042 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 30: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:15.627565 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.662510 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:15.669771 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 91613.4us
I20260709 12:20:15.630126  7016 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:15.677445 32115 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:15.676496 31936 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:15.680198 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.684257 32655 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:15.631206  6916 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:15.682169  6179 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:15.689837  6179 consensus_queue.cc:1243] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:15.635233  6846 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:15.685959 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:15.660588  6899 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.016s	sys 0.012s
I20260709 12:20:15.692345 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.696762 32653 raft_consensus.cc:2410] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 23 because replica has last-logged OpId of term: 21 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 18.
I20260709 12:20:15.692442 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.679316 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:15.600472  7007 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:15.696170 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 220906us
I20260709 12:20:15.707923 32652 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 16.
I20260709 12:20:15.688653  6877 raft_consensus.cc:2764] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:20:15.713920 32116 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 30.
I20260709 12:20:15.682991  7016 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:15.696259  6899 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.016s	sys 0.012s
I20260709 12:20:15.696934  6903 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:15.737434 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:15.769498 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 93085.3us
I20260709 12:20:15.769879  6967 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:15.770573  6967 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.770967  6967 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:15.778810  7030 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:15.755717  6916 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:15.780562  6967 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.738687  7016 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:15.787326  7009 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:15.789373 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:15.706437  6920 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:15.795334 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:15.744717  6903 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.789846  6967 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.804699  6743 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:15.811335  7007 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.813078 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.815107  7009 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.756116  6216 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:15.799208  6975 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:15.821785  7012 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.803618  6903 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.854350  7020 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.861030 32118 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:20:15.818295  6989 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.857208  6846 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:15.872006 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.872494 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.866546  7020 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.873114 32115 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 25: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:15.874279 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.875200 32121 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
W20260709 12:20:15.876952 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 194707us
I20260709 12:20:15.877907 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:15.879715  7040 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:15.880152  7040 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:15.849136  6216 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.882583 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.858233  6713 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.880892 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:15.888257 32013 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:15.888788 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.889825  6713 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.890480  6885 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:20:15.890962  6885 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:15.892015  6713 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.891815  6885 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.893163  6846 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:15.894207  6846 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.899605  6618 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.885294 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.903957 32115 raft_consensus.cc:2393] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 30: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:15.904482 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.911546 32115 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:20:15.913852 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.914547 32654 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:15.915275 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.916126 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:15.917919 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.921800 32115 pending_rounds.cc:85] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 13
I20260709 12:20:15.922230 32115 pending_rounds.cc:107] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:20:15.922559 32115 raft_consensus.cc:2889] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:15.925242  6950 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:15.926091  6216 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.883793  6623 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.930998  6623 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.933655  6623 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.939080 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.940657 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:15.941515 32121 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:15.943320  6989 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.943727  7012 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.881150  6842 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:15.945039  6842 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:15.947242  6979 consensus_meta.cc:287] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.534s	user 0.005s	sys 0.005s
I20260709 12:20:15.947839  6842 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.947806  6979 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:15.949450 32113 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:15.950001  6909 consensus_meta.cc:287] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 12:20:15.950860  6979 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:15.968534 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.951675  7009 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:15.952008 32113 pending_rounds.cc:85] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 11
I20260709 12:20:15.973651 32113 pending_rounds.cc:107] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:20:15.973997 32113 raft_consensus.cc:2889] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:15.917527  6684 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:15.974697 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.880628  7040 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.869045  6920 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:15.955241 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:15.957336 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.960340 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.977413 31965 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.963188  6954 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:15.963352 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:15.963888 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47822.6us
I20260709 12:20:15.964123 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.963371 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:15.966322 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:15.950588  6909 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:15.969252 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:15.963946 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.970482 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:15.971894 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 82236.1us
I20260709 12:20:15.966626 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.913486  7007 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:15.975999 31966 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.977905 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.979830 31967 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:16.043754 32112 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:16.044483 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:16.044889 32112 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:16.041910 31968 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:16.041605 31964 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:16.040565 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.052971 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:15.970299  6967 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.055447 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.057576 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:16.059085 32121 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.062583 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.057004 32657 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
W20260709 12:20:16.046754 32040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.007s	sys 0.029s
W20260709 12:20:16.065163 32040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.007s	sys 0.029s
I20260709 12:20:16.058089 31966 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:15.979619  6954 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.046697  7007 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.095323  7007 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:16.095647  6954 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:16.090852  7020 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:16.097915  7020 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:16.038669  6743 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:20:16.098486  7020 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:16.098675  6743 consensus_queue.cc:1243] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer 38c927f0ee104b57972899f5b27aa99b log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:16.099745  6950 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.101392  6950 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:16.089570  6903 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.106519 32116 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:16.109329  6903 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.110481  6956 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:16.111083 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.110991  6956 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.112128 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.112286 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.095479 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.112967 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.089257 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.113811 32041 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:16.114286 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.113075 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.117480 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.120321 31965 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:16.124588 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.128415 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.132614 32121 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:16.133963 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.136384 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.138521  6841 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:16.138794  7019 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.137995 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.138217  6932 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:16.140916  6932 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:16.142139  6709 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:16.141793  6932 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.141801 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.151333 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.152032 32119 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:16.151942 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.152328 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:16.151286 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 185051us
I20260709 12:20:16.153678 32626 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.155999 32039 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:16.155949 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.053602  7016 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:16.160254  7035 log.cc:927] Time spent T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.341s	user 0.000s	sys 0.003s
I20260709 12:20:16.163141 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.164925 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.167148  6909 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:16.203779 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3060) took 1081 ms. Trace:
I20260709 12:20:16.204655 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.167471 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.142597  6709 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.171087  6956 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:16.171904  6920 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.126814  6903 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:16.220580 32118 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:16.175748  6992 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.183348 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.183492  7012 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:16.186676  7027 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.382s	user 0.000s	sys 0.002s
I20260709 12:20:16.187027  6835 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:16.191883 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47558 (request call id 1104) took 1032 ms. Trace:
I20260709 12:20:16.196049 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.156373  7049 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:16.204216 32046 rpcz_store.cc:276] 0709 12:20:15.122545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.156725 (+ 34180us) service_pool.cc:224] Handling call
0709 12:20:15.159387 (+  2662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.355036 (+195649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.355093 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:15.355155 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:15.355199 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.203571 (+848372us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54001a5ed4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:16.203613 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.203726 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449302272}
I20260709 12:20:16.208381 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.219151 32656 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:16.067875  6967 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.225447 32651 rpcz_store.cc:276] 0709 12:20:15.159879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.160132 (+   253us) service_pool.cc:224] Handling call
0709 12:20:16.191720 (+1031588us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:16.191826 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311442688}
I20260709 12:20:16.226536 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:16.239326 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.242094  6967 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:16.217312  6709 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:16.248165 32120 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:16.254793 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 214295us
I20260709 12:20:16.257481 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:16.255836 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 289745us
I20260709 12:20:16.262919 31935 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:16.267537 31965 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:16.270229  6179 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.222558  6773 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.277311  7053 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:16.222723  6930 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:20:16.293689  6930 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:16.294155  6930 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:16.295058  6930 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.295677  6986 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.265556  6983 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:16.296258  6986 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.296289  6930 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.296700  6983 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.299026  6983 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:16.299265 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.300271  6853 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.302775 32116 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:16.231814  6956 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:16.306480 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 43654.2us
I20260709 12:20:16.311551 31935 leader_election.cc:400] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24.
I20260709 12:20:16.311877 31935 leader_election.cc:403] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:16.302244 31938 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:16.312048 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.316964 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.320329 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.331166 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.332381 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:16.304183  6981 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:16.313951  6905 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:20:16.340255 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 104463us
I20260709 12:20:16.345515 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.346264  6846 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:16.346647 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.347769  6846 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:16.348145  6846 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.348129 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.348922  6846 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.349619 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.349898  7012 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.350584  7012 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:16.350939  7012 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:16.351176 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.351379  7012 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.350749 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.352663 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.353885  6713 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:16.353940 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:16.354398  6713 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.354820  6713 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.356185  6978 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:16.357072  6216 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:16.358800  6858 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:16.361851  6978 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:16.362828  6978 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.337513  6984 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.328558  6967 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:16.300757  6853 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.328785  6579 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:16.321328  6415 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24.
W20260709 12:20:16.379686 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48644.8us
I20260709 12:20:16.388756 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.339761  6986 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:16.343456  6905 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.395694 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.398389 32112 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
W20260709 12:20:16.401743 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 136275us
I20260709 12:20:16.401978 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.380873  6966 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:16.385265  6967 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:16.419302  6989 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:16.400141  7004 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:16.395041  6981 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.422875 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.402724  6905 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:16.438664  7016 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:16.440980  7007 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.443441  7020 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.401782  6841 raft_consensus.cc:2764] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:20:16.461931  7058 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:16.454524 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.426517  6899 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:16.433362  7040 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.486083 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.466409  6713 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.452976  7004 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:16.467758  6967 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.452765  6981 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.422801  6986 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:16.496686  6981 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.496775  6986 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:16.497617  7040 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:16.477535  6853 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:16.477573 32657 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:20:16.430872  6980 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:16.499784  6713 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:16.499984  6980 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.482601 32117 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:16.483667  7020 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:16.459385  7019 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:16.505240  7019 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.507418  7019 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.485284  6982 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:16.487607 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.453591  6216 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:16.518975 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.430461  6989 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.487402  6966 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:16.523483 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.525332 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.526633 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.529064 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.530123 31963 raft_consensus.cc:2368] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26.
I20260709 12:20:16.490622  6899 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:16.497489  6986 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.531015 32038 raft_consensus.cc:2393] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 26: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:16.533375 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:20:16.540536 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 183592us
I20260709 12:20:16.531379 32626 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.522513  6853 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:16.547264 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.548019 31966 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:16.552445  6684 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.552899  6954 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.555714 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.555960 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.556706 32112 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:16.557907 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.559566 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:16.559943 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.563120 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.563628 31966 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:16.567494 32114 raft_consensus.cc:2368] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26.
I20260709 12:20:16.568461 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.568801 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.573668 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.573922 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.575783 31971 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 12:20:16.576099 31971 pending_rounds.cc:85] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:20:16.576310 31971 pending_rounds.cc:107] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:20:16.576524 31971 raft_consensus.cc:2889] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:16.577054 31963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:16.577499 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.528657  6981 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:16.577152 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:16.541855  6415 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.582041 32046 raft_consensus.cc:2368] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:20:16.583770 31970 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:20:16.545954  6963 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.584551 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.618105 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.527973  7040 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:20:16.613857 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.489688  7058 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.586722 32623 leader_election.cc:400] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26.
I20260709 12:20:16.620693 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.623904 32623 leader_election.cc:403] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:20:16.621845 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:16.616008 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51353us
I20260709 12:20:16.628468 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.629902 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.585417  6966 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:16.638167 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:16.639180 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:16.639441 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56267.1us
I20260709 12:20:16.640447 31971 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:16.642542 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.655431  7058 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:16.656389  7059 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.071s	user 0.003s	sys 0.002s
I20260709 12:20:16.654026  6954 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:16.661188  6905 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.663239 31972 raft_consensus.cc:2368] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 26. Current term is 27.
I20260709 12:20:16.663508  6905 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.663637 31968 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:16.664721  6980 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.645922  6909 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:16.665347 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.666694 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:16.672204 32626 leader_election.cc:400] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26.
I20260709 12:20:16.629709  6216 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.623834  6963 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.696609  6963 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.585223  7004 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.701603  6989 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.639664  6966 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:16.704175  6966 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.707844 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.708297 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.722743  7007 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:16.725891  7061 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
I20260709 12:20:16.725777  7007 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:16.742885 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:16.743837 32117 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:16.732218 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.750200 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.751425 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:16.773576 32115 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:16.759341  6709 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.761802 32119 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:20:16.763743 32113 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:16.802975 31963 raft_consensus.cc:606] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.803488 31963 raft_consensus.cc:614] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:16.765393  6964 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.809031 32118 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 25. Current term is 26. Ops: []
W20260709 12:20:16.811482 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 196638us
W20260709 12:20:16.808903 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 275602us
I20260709 12:20:16.813670 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.815497 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.806237  6955 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.779702  6846 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.821630 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:16.805567 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55057.5us
I20260709 12:20:16.828063 32115 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:16.781584  7012 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:16.833920  6932 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:16.859187 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:16.859859 32657 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:16.859997 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.860250 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.862025 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.840070 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.868741 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.871479 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.822840 31938 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.843644 32041 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:16.874488 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.804242  6983 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:16.820016  6982 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:16.877137 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.811117  6618 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:20:16.814206  6402 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
W20260709 12:20:16.883263 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 60501.2us
I20260709 12:20:16.884344 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.884881 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:16.883157 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.821310  7020 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
W20260709 12:20:16.897044 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63790.5us
I20260709 12:20:16.890157 32045 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:16.891353 32117 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:16.891376 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.899219 32044 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:16.900408 32039 raft_consensus.cc:2368] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23.
I20260709 12:20:16.829102  6905 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:20:16.892211 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:16.894621 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:16.895435 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.828871  6773 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:20:16.866758  6016 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:16.910058 32043 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:16.894069 32038 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:20:16.912390 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:16.902719  6945 raft_consensus.cc:2764] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:20:16.936666 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.888619  6415 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:16.936744 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:16.889482  6684 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:16.944335  6983 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:16.901562  6858 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:16.957659 32657 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:16.961373 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:16.905511  6982 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:16.962378  6179 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.960263 31936 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:16.970746 32660 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:16.978464 32043 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:16.955747  6853 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:16.967895  6973 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:17.008674 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:17.015147 31965 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:17.016307 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:17.017638 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:16.967180  6964 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:16.969173  6982 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.939601  6016 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.190125  6982 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.169576  6973 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:17.167559  6579 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:17.214818 32040 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:20:17.240694 32040 pending_rounds.cc:85] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 11
I20260709 12:20:17.241044 32040 pending_rounds.cc:107] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:20:17.241353 32040 raft_consensus.cc:2889] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:17.215636  6966 consensus_meta.cc:287] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.008s
I20260709 12:20:17.239154 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:17.243602 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.028s	sys 0.060s
I20260709 12:20:16.999845  6899 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:17.239756 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:17.167366  6984 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:17.253391 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:17.243954 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.028s	sys 0.060s
I20260709 12:20:17.235618  7004 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:17.274410 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.275080 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.246069  6966 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.302636  6846 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:17.304332  7012 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:17.304811  6966 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.278116 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.306267 31964 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:17.309651  6986 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.751s	user 0.006s	sys 0.004s
I20260709 12:20:17.253065  6773 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:17.315598  6773 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:17.316141  6773 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.313637  6016 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:17.321493 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.333407 32651 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:20:17.334506 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:17.270916  6579 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:17.338618 32085 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:17.319748  7004 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:17.341176  6982 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:17.369228 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 461888us
W20260709 12:20:17.370028 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 408700us
I20260709 12:20:17.332777  6986 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.371675 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.343561  6579 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.377811 32113 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:20:17.366612  6415 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:17.387275  6415 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.388242  6579 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:17.389528  6909 consensus_meta.cc:287] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.001s
I20260709 12:20:17.389972  6909 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26.
I20260709 12:20:17.393975 32010 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.395902 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.397368 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.400123 31968 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:17.400300 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:17.401136 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 464469us
I20260709 12:20:17.401388 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:17.402374 32623 leader_election.cc:400] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 26. Current term is 27.
I20260709 12:20:17.402534 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:17.402652 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.402837 32623 leader_election.cc:403] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:20:17.347980  6973 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.408267 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:17.404603 31972 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:20:17.404663 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 441625us
I20260709 12:20:17.410435  6966 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:17.411157  6899 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:17.411557  7065 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:17.411996  7065 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:17.412817  7065 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.427170 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:17.414167  7065 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.400300  6964 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:17.404171  6402 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:17.406481  6909 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 26. Current term is 27.
I20260709 12:20:17.406121  6885 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:17.404516  6932 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:17.429754  6899 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:17.430145  6966 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:17.449175 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:17.452946 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.455793 32656 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.405144  6579 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:17.453987 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.457465 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:17.458441 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 465466us
I20260709 12:20:17.460144 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.441000  6973 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.463910 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:17.399626  7004 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:17.469198 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 130645us
I20260709 12:20:17.470389 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.470892 32113 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:17.488042 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.439867  7009 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.023s
I20260709 12:20:17.457736  6945 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:17.494976 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:17.495713 31964 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:17.462023  6982 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.493270 32656 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:17.517127  6920 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:17.492408 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 246197us
I20260709 12:20:17.525336 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:17.526558  7083 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:17.527129  7083 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:17.528034  7083 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.462358  6579 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.538674 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:17.539867 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 49028.6us
I20260709 12:20:17.543118 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:17.544314 32038 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:17.502481  7082 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:20:17.545574 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 88224.4us
I20260709 12:20:17.523854  6945 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:17.549474 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:17.552007 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 93380.1us
I20260709 12:20:17.532797  6982 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:17.566715  7096 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:17.567469  7078 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:20:17.548477  7088 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:17.572795  6579 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:17.573405  6982 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:17.556193  7075 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:20:17.583405  6982 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:17.585537 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2884) took 1392 ms. Trace:
I20260709 12:20:17.585994 32040 rpcz_store.cc:276] 0709 12:20:16.193164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.193947 (+   783us) service_pool.cc:224] Handling call
0709 12:20:16.194302 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.241935 (+1047633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.585269 (+343334us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003c0854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:17.585470 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2345554304}
I20260709 12:20:17.587815  6986 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.556866  7082 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:17.590289  6930 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:17.590806  6930 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:17.591686  6930 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.592926  6930 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:17.596895 32118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3315) took 1492 ms. Trace:
I20260709 12:20:17.597288 32118 rpcz_store.cc:276] 0709 12:20:16.104989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.221377 (+116388us) service_pool.cc:224] Handling call
0709 12:20:16.221640 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.810868 (+589228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.596658 (+785790us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b540012c0d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:17.596835 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350390144}
I20260709 12:20:17.590943  7082 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:17.613937 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 119017us
W20260709 12:20:17.616695 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.616695 32116 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:17.618892 32121 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:17.595752  6773 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.624150  6773 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:17.632241  7081 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:17.632870  7081 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.635075  7081 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:17.635618  7081 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.635699 32084 leader_election.cc:400] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23.
I20260709 12:20:17.636104  7081 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.638465  7081 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.604077  6967 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:17.641615  6967 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:17.642141  6967 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.649624 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.648837 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.647899 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:17.652601 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.658164 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:17.670477 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 128764us
I20260709 12:20:17.685817  6773 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:17.685055  7009 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:20:17.693641  6967 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.697189  6846 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:17.697896 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:17.698720 32118 raft_consensus.cc:2393] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:17.699020  7060 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.699528  7060 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.700117  6967 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.701937  7060 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.703044 32117 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:17.708328 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 59472.7us
I20260709 12:20:17.714614 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:17.726593 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 171138us
I20260709 12:20:17.730749 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.735157 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.738443 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:17.702451  6909 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:17.750932 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:17.719098 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:17.743630 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:17.752688 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:17.756592 31972 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:17.713502  6973 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:17.716827  6930 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.746356  6982 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:17.762475 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 58870.4us
I20260709 12:20:17.746987  7097 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.705538  6773 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:20:17.763250 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:17.765084 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.771106 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.759968  6973 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:17.765537 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46526.1us
I20260709 12:20:17.775960 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:17.762476  6858 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:17.780854 32118 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:17.762537  7086 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:17.851008 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 73802us
I20260709 12:20:17.755470  6909 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:17.852401 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 77113.5us
I20260709 12:20:17.771579  7097 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.854094 31963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46882 (request call id 37) took 1307 ms. Trace:
I20260709 12:20:17.854478 31963 rpcz_store.cc:276] 0709 12:20:16.546241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.547251 (+  1010us) service_pool.cc:224] Handling call
0709 12:20:17.853904 (+1306653us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1c9853 00007f229119760a 00007f228a71d7d4 00007f228a71d762 00007f228a71d691 00007f228a71d60c 00007f228a71c8c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:17.854014 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518116096}
I20260709 12:20:17.855063  7097 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:17.860324  7069 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.556s	user 0.002s	sys 0.000s
I20260709 12:20:17.773336  6989 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:17.773139  7099 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:17.764139  7060 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
W20260709 12:20:17.869841 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 93962.5us
I20260709 12:20:17.871398 31969 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:20:17.877111 31966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2862) took 1302 ms. Trace:
I20260709 12:20:17.778822  7088 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:17.765300  6930 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.773416  7081 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 12:20:17.877485 31966 rpcz_store.cc:276] 0709 12:20:16.574454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.574674 (+   220us) service_pool.cc:224] Handling call
0709 12:20:16.580302 (+  5628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.585083 (+  4781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:16.585130 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:16.625841 (+ 40711us) log.cc:844] Serialized 26 byte log entry
0709 12:20:16.894443 (+268602us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:16.894507 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.894602 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:17.859605 (+965003us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:17.870954 (+ 11349us) raft_consensus.cc:1685] finished
0709 12:20:17.870973 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.877047 (+  6074us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4427,"raft.queue_time_us":4466,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2822,"spinlock_wait_cycles":35257728,"thread_start_us":232220,"threads_started":1,"wal-append.queue_time_us":661998}
W20260709 12:20:17.874270 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 85724.7us
I20260709 12:20:17.775780  6216 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:20:17.851106  6684 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:17.866988  6858 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.898947  7040 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:17.899443  6858 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:17.901935  6709 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:17.870469  7090 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:17.766516  6982 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.887187 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:17.896961  6684 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.882764  7099 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:17.907480 32119 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:20:17.892221 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.870357  6954 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:17.876103  6842 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.879554  6986 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:17.917220 32656 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:20:17.885639  6930 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:17.918917 31966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:17.893824  7089 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:17.904230  6713 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
W20260709 12:20:17.936208 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84349.5us
I20260709 12:20:17.935137 31966 raft_consensus.cc:606] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:17.890899  7009 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:20:17.936820 31966 raft_consensus.cc:614] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:17.890358  7086 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:20:17.918978  7085 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:17.940797 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 161116us
I20260709 12:20:17.903982  6973 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:17.965296 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 187786us
W20260709 12:20:17.947718 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 200481us
W20260709 12:20:17.952735  7107 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:20:17.953081 31966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:17.967195 31966 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:17.967542 31966 raft_consensus.cc:614] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:17.934758  6842 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.953601  7104 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:17.970281  7099 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 21.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.970897  7104 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:17.971805  7104 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.973033  7104 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:17.933434  6954 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:17.954828  6909 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:17.949246 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 80561.3us
I20260709 12:20:17.915786  7090 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:17.969141 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:17.934006  7095 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:20:17.942066  7089 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.007694 32040 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:17.975274 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:17.976118 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:17.976310 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:17.979282 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:18.009879 31966 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.988902  6982 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:17.989535 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:17.990643  6684 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:17.991604  6877 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:17.992368  6842 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:17.992933  6909 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:17.994423 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:17.995779 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 1208) took 1439 ms. Trace:
I20260709 12:20:18.006238 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:18.011600 31963 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:18.020640 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:18.022382 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:18.034722 32043 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:18.033838 31936 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:18.031311 32112 rpcz_store.cc:276] 0709 12:20:16.556464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.557567 (+  1103us) service_pool.cc:224] Handling call
0709 12:20:16.557857 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:16.703453 (+145596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:16.703500 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:17.408003 (+704503us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:20:17.408687 (+   684us) log.cc:844] Serialized 52 byte log entry
0709 12:20:17.515620 (+106933us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:17.515667 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.995473 (+479806us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001491d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:17.995524 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:17.995569 (+    45us) raft_consensus.cc:1685] finished
0709 12:20:17.995583 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.995720 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8306,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":405373184,"thread_start_us":104682,"threads_started":1,"wal-append.queue_time_us":138244}
W20260709 12:20:18.038511 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 62707.9us
I20260709 12:20:18.047904 31966 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:17.980135  6930 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.039196 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:18.052294 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.053071 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:18.051324 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:17.980664  7081 consensus_queue.cc:1059] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:20:18.053925  7086 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:18.054428  6967 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:18.017493  6877 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:18.031076  6909 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:18.050648  7103 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.067952  7114 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:20:18.071908 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:18.073180 32040 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:18.083043  7099 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:18.092588 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:18.066009  7115 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:18.051869  7097 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
I20260709 12:20:18.057852  6930 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.086473 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:18.055657  7085 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:18.090142 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.096874  6899 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:18.095163  7106 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:20:18.104094  7128 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:18.107142  7112 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:20:18.112273 31967 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2863) took 1537 ms. Trace:
I20260709 12:20:18.070832  6986 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:20:18.095709  7086 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:20:18.091221 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.095422  6967 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.096189  7099 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.095947  7116 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:18.084316 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102656us
I20260709 12:20:18.216464 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:18.122814 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.126494  6877 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.126509 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3222) took 1461 ms. Trace:
I20260709 12:20:18.102035  7115 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.091207  7103 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.133249  6930 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.133903 32046 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:20:18.134943 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 153902us
W20260709 12:20:18.138176 32039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54754 (request call id 1004) took 1430 ms (client timeout 1616 ms). Trace:
W20260709 12:20:18.219278 32039 rpcz_store.cc:269] 0709 12:20:16.707434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.707572 (+   138us) service_pool.cc:224] Handling call
0709 12:20:18.137939 (+1430367us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400032fd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:18.138080 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435133184}
I20260709 12:20:18.116873  7117 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:18.117908  6899 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.218295 31969 rpcz_store.cc:276] 0709 12:20:16.664635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.694277 (+ 29642us) service_pool.cc:224] Handling call
0709 12:20:17.871165 (+1176888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:17.880327 (+  9162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.126325 (+245998us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000f5d50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:18.126454 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2614577920}
I20260709 12:20:18.139971 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.228669 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.223130 32038 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:18.229249 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.140089  6773 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.140865 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 128689us
I20260709 12:20:18.142939 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:18.147194 32117 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:18.148360 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56072 (request call id 2868) took 1313 ms. Trace:
I20260709 12:20:18.150301  6724 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:18.152341 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:18.154593  7072 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.794s	user 0.003s	sys 0.000s
I20260709 12:20:18.242503 32114 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
W20260709 12:20:18.159281  7079 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.794s	user 0.002s	sys 0.000s
I20260709 12:20:18.205681  6990 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.185847  6579 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.195137  7089 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.204154 32040 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:20:18.260493 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:18.261231 32040 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:18.138659 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:18.204139  6967 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.223228  7060 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:18.218647  7086 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.237183 32045 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:18.176528  7085 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.234560 31968 rpcz_store.cc:276] 0709 12:20:16.834676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.838402 (+  3726us) service_pool.cc:224] Handling call
0709 12:20:16.838668 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.404893 (+566225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.148147 (+743254us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b5400455bd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:18.148303 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1290735872}
I20260709 12:20:18.174373 31967 rpcz_store.cc:276] 0709 12:20:16.574760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.574865 (+   105us) service_pool.cc:224] Handling call
0709 12:20:16.575106 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.580671 (+  5565us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:16.802430 (+221759us) log.cc:844] Serialized 14 byte log entry
0709 12:20:16.802763 (+   333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:16.802816 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:16.802868 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.112022 (+1309154us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b5400069854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:18.112066 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.112215 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10511,"prepare.queue_time_us":316695,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3039,"raft.queue_time_us":85,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3949,"spinlock_wait_cycles":12329216,"thread_start_us":170609,"threads_started":1}
I20260709 12:20:18.242264  6724 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:18.204723  6853 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.255909 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:18.213922  7004 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.256989 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:18.267625  7095 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:20:18.269683  7074 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.926s	user 0.003s	sys 0.000s
I20260709 12:20:18.274593 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.218279  6982 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.235600  6899 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:18.296188 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:18.296290 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:18.309777 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 93437.7us
W20260709 12:20:18.309926  6981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.398s	user 0.009s	sys 0.011s
I20260709 12:20:18.311654 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:18.179342  6905 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.314208 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.345583 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.247929  7120 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:18.250698  7012 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.318135 32652 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:20:18.117659  6909 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.223346  7116 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:18.323637 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67804.5us
I20260709 12:20:18.258519  7066 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.369593  7066 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.372239  7066 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.321635  7136 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:18.329885  7138 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.256722  6709 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.343206  6842 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:18.247493  6773 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.290902  7124 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.313349 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:18.363592 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 211476us
I20260709 12:20:18.363718 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:18.343564  6981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.439s	user 0.010s	sys 0.012s
I20260709 12:20:18.276475  7101 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:20:18.184787  7128 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:18.329295  6950 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.006s	sys 0.006s
I20260709 12:20:18.333168  6954 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.277984  6990 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.317127  7090 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.276698  6579 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.343542  7135 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.383872  6724 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.597735 32041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3089) took 1703 ms. Trace:
I20260709 12:20:18.390117  7095 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:18.376396  7009 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:18.358978  7054 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.602764  7054 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.385243  7136 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.608492  7136 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:18.610463  7121 consensus_queue.cc:1175] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f05e4741d35c4cf3a31358c56e65b30c" responder_term: 22 status { last_received { term: 22 index: 17 } last_committed_idx: 17 last_received_current_leader { term: 22 index: 17 } }
I20260709 12:20:18.611843  7122 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.612614  7122 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.349084  6982 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.523178 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:18.621712 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 98592.8us
W20260709 12:20:18.623587 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.365834  7120 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:18.572912 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 165605us
I20260709 12:20:18.362795 31971 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3208) took 1778 ms. Trace:
I20260709 12:20:18.311355  7060 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.322403  7085 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.335625  7004 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:18.587266 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 255746us
I20260709 12:20:18.522454  6709 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.333389  6853 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.393780  6909 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:18.360567  6905 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.316402  7086 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.374660  7012 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.598169 32041 rpcz_store.cc:276] 0709 12:20:16.893940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.894066 (+   126us) service_pool.cc:224] Handling call
0709 12:20:16.894384 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.894587 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:16.894611 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:16.900767 (+  6156us) log.cc:844] Serialized 26 byte log entry
0709 12:20:17.183405 (+282638us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:17.183463 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.183570 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:18.158709 (+975139us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:18.590694 (+431985us) raft_consensus.cc:1685] finished
0709 12:20:18.590716 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.597668 (+  6952us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58517760,"thread_start_us":280985,"threads_started":1,"wal-append.queue_time_us":446102}
I20260709 12:20:18.394586  7138 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:18.624034 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:18.689246 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.570876 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3204) took 1989 ms. Trace:
I20260709 12:20:18.585575  6973 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:18.632366 32112 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:18.639546 31963 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:18.669081 32084 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:18.636574 31971 rpcz_store.cc:276] 0709 12:20:16.583936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.650617 (+ 66681us) service_pool.cc:224] Handling call
0709 12:20:16.650930 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.659942 (+  9012us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:16.659978 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:16.660452 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:20:16.954843 (+294391us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:16.954883 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.954966 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:18.154018 (+1199052us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b4400022b4c. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f2289008c85 00007f22890096a2 00007f228bd7b1db 00007f228bd76aaf 00007f228bd6a86e 00007f228bd69bb6 00007f228bd697d8 00007f228bd73221 00007f228bd731d9 00007f228bd73169
0709 12:20:18.350607 (+196589us) raft_consensus.cc:1685] finished
0709 12:20:18.350637 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.362726 (+ 12089us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":102,"spinlock_wait_cycles":71832448,"thread_start_us":293992,"threads_started":1,"wal-append.queue_time_us":679425}
I20260709 12:20:18.585989  6956 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:20:18.690533 31970 rpcz_store.cc:276] 0709 12:20:16.581173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.662380 (+ 81207us) service_pool.cc:224] Handling call
0709 12:20:16.662719 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.728751 (+ 66032us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:16.967932 (+239181us) log.cc:844] Serialized 14 byte log entry
0709 12:20:17.458037 (+490105us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:17.469559 (+ 11522us) log.cc:844] Serialized 26 byte log entry
0709 12:20:17.470051 (+   492us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:17.470103 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.391948 (+921845us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b540034e254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:18.392010 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:18.392064 (+    54us) raft_consensus.cc:1685] finished
0709 12:20:18.570610 (+178546us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.570806 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":245434,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":91990,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":35048,"spinlock_wait_cycles":150947712,"thread_start_us":470077,"threads_started":2,"wal-append.queue_time_us":363837,"wal-append.run_cpu_time_us":2263,"wal-append.run_wall_time_us":1022455}
I20260709 12:20:18.672350  6724 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.678548 31969 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
W20260709 12:20:18.691404 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 395269us
I20260709 12:20:18.711700 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.695439 31968 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:18.714110 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.640723  7009 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:18.584059  6950 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.313s	user 0.006s	sys 0.006s
I20260709 12:20:18.631203  6986 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.062s
W20260709 12:20:18.721814 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52843.7us
I20260709 12:20:18.722095 31971 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:20:18.729417 31971 pending_rounds.cc:85] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:20:18.729766 31971 pending_rounds.cc:107] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:20:18.730069 31971 raft_consensus.cc:2889] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:18.622220  7135 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:18.774016  7139 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
W20260709 12:20:18.782882  7141 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:20:18.734117  7095 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.788350  7054 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.789937  7128 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:18.737056  7115 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:18.790661  6967 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:18.737550  7104 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.764s	user 0.012s	sys 0.000s
I20260709 12:20:18.792404  6990 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.828758  6990 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:20:18.792765  7116 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.793951  7120 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:18.780186  7102 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 12:20:18.811125  6986 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.830771  6986 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.835749  6967 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:18.836489  6967 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:18.837993  6899 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.839084 31966 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:18.839105  7142 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:18.843758  6986 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.719331  7009 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.728546  6858 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:18.735950  7066 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.839989  6899 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.845453  7066 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.822804  7012 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.823159  6709 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.824694  6773 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:18.828032  7060 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.811007  7147 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.851056 31971 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:18.871896 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.888940 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.890107 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.846235  6982 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:18.850834  7009 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:18.890729 31971 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 18.
I20260709 12:20:18.900848 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:18.903719 32042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.105s	sys 0.156s
W20260709 12:20:18.904155 32042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.105s	sys 0.156s
I20260709 12:20:18.851895  6899 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.923834 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:18.938446 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:18.939250 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 258372us
I20260709 12:20:18.930071 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.943842 31967 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:18.948661 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:18.897095  7122 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:18.952287 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:18.953826 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:18.937755  6982 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:18.956044 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:18.840013  7142 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.958614 32040 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:20:18.957368 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:18.959775 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:18.948745  6709 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:18.967824 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:18.969168 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:18.969954 32656 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:18.895378  7147 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.939723  7102 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:20:18.957257  7012 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:18.945086  6899 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:18.972361  6709 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:18.995034 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:18.997449 31967 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
W20260709 12:20:19.000072 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48976.6us
I20260709 12:20:18.984959  7012 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.033638  7012 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:19.034847  7140 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.248s	user 0.003s	sys 0.000s
I20260709 12:20:19.034077  7009 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.037011  7009 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.001427 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:19.002261  7068 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:20:19.002797 32115 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:19.002902  7119 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:19.005622  7152 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:19.007311  7142 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.010066  7135 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.014652  6967 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:18.994697  7143 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:19.023025  7147 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.024420  7104 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:18.994144  6709 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:18.966337  6982 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.032435  7066 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.041373 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:19.113962 32112 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:19.045354 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3097) took 1040 ms. Trace:
I20260709 12:20:19.115070 32046 rpcz_store.cc:276] 0709 12:20:18.005128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.005249 (+   121us) service_pool.cc:224] Handling call
0709 12:20:18.005573 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:18.218807 (+213234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.045060 (+826253us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540037a354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:19.045288 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294349056}
W20260709 12:20:19.051018 32623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 1.616s (SENT) [suppressed 9 similar messages]
I20260709 12:20:19.052634  6932 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.052991  6899 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.054728 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.056963 31936 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:19.121017 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 64143.9us
I20260709 12:20:19.090714  7119 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:19.061198  6983 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.129668  6983 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.131793  6858 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:19.132217  6858 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:19.132682  6983 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.133136  6858 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.061829  7124 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.063479  7085 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.065876  7101 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.067884 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.073453 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.111343  6920 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.074488 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.075330 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.160377 32041 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:19.084417 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.085325 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.086656 32655 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:20:19.091800 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:19.163417 31970 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:19.099252 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.091835 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:19.082753  7120 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:19.059420  7099 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.073546 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.082181 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:19.206162 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 80292.7us
I20260709 12:20:19.207334 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:19.168708 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:19.211935 31966 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
W20260709 12:20:19.214432 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 122672us
I20260709 12:20:19.213148 32112 raft_consensus.cc:2468] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:19.204672  7120 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.159287  6858 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:19.223606 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.223673  6179 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.225636  6179 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.135582  7147 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:19.226516 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.236639 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:19.168315  7151 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.132753  6773 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:19.228582 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.117889  7104 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.229952 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.120662  6932 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.210862  6909 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.137228  7119 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:19.222246  7120 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:19.258071 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 187884us
I20260709 12:20:19.265108  6179 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.262482  7151 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.252990 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58058 (request call id 987) took 1889 ms. Trace:
I20260709 12:20:19.276602 31964 rpcz_store.cc:276] 0709 12:20:17.363501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.541725 (+178224us) service_pool.cc:224] Handling call
0709 12:20:17.541985 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.552528 (+ 10543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:17.552557 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:17.552607 (+    50us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:17.552642 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.252715 (+1700073us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5400069354. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:19.252779 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.252926 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23856384}
I20260709 12:20:19.229341  7066 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:19.283031 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.281018 32047 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:19.239823  7147 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.239586  6945 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:19.205412  6920 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.287168 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.267724  6773 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:19.292002 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 84781.3us
I20260709 12:20:19.261521  6983 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.292941  6179 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:19.212005  7101 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.158627  7124 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.288530  7066 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.292851  7147 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:19.315056 32119 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:19.273856  6909 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.303922  6980 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.317612  7066 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:19.301817  6983 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:19.332502  7161 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:20:19.334229  6983 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.335203  6932 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.326473  6980 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.339056  7101 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.339555  6980 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.342721  6920 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.351191  7104 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:19.353807  7130 log.cc:927] Time spent T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 1.057s	user 0.013s	sys 0.040s
I20260709 12:20:19.354817  7124 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:20:19.399243 32013 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:19.418612 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.420189 32117 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:19.422185 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.338127  6709 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.435900 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 994) took 1218 ms. Trace:
I20260709 12:20:19.436494 31969 rpcz_store.cc:276] 0709 12:20:18.217074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.228435 (+ 11361us) service_pool.cc:224] Handling call
0709 12:20:19.435476 (+1207041us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400312c54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:19.435680 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033132800}
I20260709 12:20:19.437996  6709 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.443631  7063 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:19.360375  7147 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.490784  7163 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:19.330461  6773 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.491874  7163 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:19.493091  6842 consensus_queue.cc:1175] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3836f9b5cd36488da56d4540dc9d80ec" responder_term: 21 status { last_received { term: 19 index: 11 } last_committed_idx: 11 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:19.476159  7151 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.496100  7128 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:19.459937  6980 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:19.497440  6980 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.498006 32042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3102) took 1158 ms. Trace:
I20260709 12:20:19.465257 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.500319  7163 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.498953 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.498694 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.467742 32114 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:19.486622  7152 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.502094  7147 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.462803 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.516273  6980 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.467998  6909 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.474189 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:19.479472  7117 raft_consensus.cc:2804] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:19.445055  7063 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.465127  7085 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.540470 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.540352 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.545137 32041 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 12:20:19.546342 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72576.4us
I20260709 12:20:19.547972 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:19.548473  6980 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:19.549023  6980 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:19.549511  6980 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:19.554076 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.555756 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.547912 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:19.556675 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:19.557682 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:19.558482 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.559741 32044 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
W20260709 12:20:19.559864 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 323158us
I20260709 12:20:19.561313 32011 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:19.539225 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.569324 32658 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:19.578238  7152 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.478826  7104 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:20:19.583652 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:19.563246  7117 raft_consensus.cc:697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:19.588776 32040 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:19.589119  6909 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.589815  6909 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.590986  7118 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:19.591297  7120 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.591717  7118 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.592341  6909 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.515192 32042 rpcz_store.cc:276] 0709 12:20:18.339021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.348895 (+  9874us) service_pool.cc:224] Handling call
0709 12:20:18.349169 (+   274us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:18.938822 (+589653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.938868 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:18.939550 (+   682us) log.cc:844] Serialized 52 byte log entry
0709 12:20:19.090842 (+151292us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:19.481608 (+390766us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b180019a200. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228fbc73c7 00007f228c1d99a7 00007f228c1e90bf
0709 12:20:19.481880 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 12:20:19.495744 (+ 13864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.497383 (+  1639us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:19.497462 (+    79us) raft_consensus.cc:1685] finished
0709 12:20:19.497605 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.497805 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":632,"prepare.queue_time_us":221350,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":40,"raft.queue_time_us":26214,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":5923,"spinlock_wait_cycles":5179136,"thread_start_us":769609,"threads_started":3,"wal-append.queue_time_us":217210}
I20260709 12:20:19.602596  7175 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:19.602697  7009 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:20:19.605815 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47552 (request call id 1242) took 1289 ms. Trace:
I20260709 12:20:19.606781  7063 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.607221 32652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 1129) took 1232 ms. Trace:
I20260709 12:20:19.610004 32652 rpcz_store.cc:276] 0709 12:20:18.375098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.375260 (+   162us) service_pool.cc:224] Handling call
0709 12:20:18.375549 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:18.614053 (+238504us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:20:18.614406 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 12:20:18.936169 (+321763us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:18.936232 (+    63us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:20:18.936275 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.606825 (+670550us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54005ea1d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:19.606882 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.607056 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7312,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":73,"raft.queue_time_us":1676,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2878,"spinlock_wait_cycles":547976576,"thread_start_us":321382,"threads_started":1,"wal-append.queue_time_us":393571}
I20260709 12:20:19.613632 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.615587  7118 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.615940  7117 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.616660  6579 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.577764  6709 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:20:19.588788 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:19.594545  7120 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.619797  7152 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.606208 32651 rpcz_store.cc:276] 0709 12:20:18.316292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.316408 (+   116us) service_pool.cc:224] Handling call
0709 12:20:18.316673 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:18.394945 (+ 78272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.394991 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:18.395047 (+    56us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:20:18.395084 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.605282 (+1210198us) spinlock_profiling.cc:243] Waited 78 ms on lock 0x7b54005eabd4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:19.605356 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.605566 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179470592}
I20260709 12:20:19.632400 32011 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:19.639045 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76318.3us
I20260709 12:20:19.640820  6964 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:19.640625 32011 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:19.642465  6618 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.643350  6618 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.645141  7004 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.648846 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:19.655445 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.659924  7178 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:19.621802  7085 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.660835  7086 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.660212  7119 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.663475  7085 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.664639  6618 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.666553  6980 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.668735  7138 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.670908  6982 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.676910 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:19.668203 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 617027us
I20260709 12:20:19.680934 32623 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:19.682463  7124 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:19.682875  7124 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.683303  7124 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:19.684353  6980 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.686642  7115 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:19.689026  6853 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:19.688601 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 46489.4us
I20260709 12:20:19.691478  6905 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.692438  6877 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.666167  7103 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.694424 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.695315 32652 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:19.695704 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:19.695992 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:19.686154  6980 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:19.698149  7063 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:19.699572  7063 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:19.699972  7063 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.700088 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:19.701552 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 126387us
I20260709 12:20:19.702183  6955 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:19.702603  6955 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:19.703047  6955 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:19.708669  7012 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.010s
I20260709 12:20:19.709254  7012 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.712097  7128 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:19.718479  7063 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.722651  7063 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:19.724637 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.725402 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.729437  7068 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.730156  6973 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.702454 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.698076 32623 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:19.740908 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.742416 32115 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:19.742133  6973 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.743774 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:19.744616 31971 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:19.744952  6973 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:19.745801 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.752797  7147 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:19.755749 32115 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:19.756317  7147 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:19.758435 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:19.763890 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.666429  7119 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:19.774044 32112 raft_consensus.cc:2393] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:19.776422  7190 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:19.777011  7190 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:19.777495  7190 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.781163 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:19.764204 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:19.793871 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:19.795104  7068 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:19.783895  6743 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:19.774363  7063 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:19.751850  7104 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.024940 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:19.740796  6909 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:19.739574  6905 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:20.012470 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.015190 32653 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.016294 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:20.033201  7066 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.684s	user 0.006s	sys 0.023s
I20260709 12:20:20.029455 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:20.034595 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 336793us
I20260709 12:20:20.036479 32115 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:20.035426 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 210002us
I20260709 12:20:20.038412  6743 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.041318  6773 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.041604  6743 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.042865 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.043951  7086 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:20.044188  7138 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.033704  7066 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.048087  7147 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.049029  7099 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:20.049747  7193 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:20:20.050767 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.059489  6877 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.035519  6841 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:20.069700 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 318695us
W20260709 12:20:20.068441 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 316120us
I20260709 12:20:20.074479 31971 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:20.027251  7124 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.059867  7138 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.093325  7086 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.094748  7138 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.094936  7147 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:20.095575  7147 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.085700 32118 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:20.096998  6930 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.097832  7147 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.086704  7065 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:20.086514  7066 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:20.098665  7065 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.099467  6983 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:20.099922  6983 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:20.101282  7065 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.100975  6983 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.086723  7085 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:20.102284  7085 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.103080  7068 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.086301 32113 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:20.104084  6835 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.073514  7099 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.089548  6955 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.090080  6877 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.081945  6905 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.074401  6743 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.091184  7117 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.091461 32114 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:20.091296  7004 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:20.067430  7097 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.139227 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.095873  7086 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.097584  6930 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.088198  6990 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.089022  6945 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.088016  6841 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.115669 32651 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:20.068184  7063 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.118148 32653 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:20.094074  7124 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.133671 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.136912 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.102221  6983 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.133577  6743 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.150636 32116 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:20.140244  7119 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.137671  7009 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:20.194223 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 47317.7us
I20260709 12:20:20.136724  6835 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.201443  6835 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.201858  6930 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:20.202734 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52461.6us
I20260709 12:20:20.203940  7063 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:20.204221 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 88416.9us
I20260709 12:20:20.204481  6990 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.205468 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:20.206856  6990 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.136591  7068 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:20.207963  7068 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.137333  6980 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:20.210739  6980 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.212972 32655 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:20.213172  6980 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.214867 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.147324  7085 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.215673  7099 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.215480 32114 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:20:20.139220  7150 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.217641  6841 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.192977 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.223558 32113 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:20.224787  6885 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:20:20.226642  6877 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.228689 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:20.230979 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:20.232033  7190 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.232977  7086 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:20.233403  7086 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:20.233819  7086 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:20.235208  6955 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.236332 31971 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:20.236220 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.237272 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:20.216924  6905 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.193434  6743 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.246587  7150 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.248602  6885 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:20.249066  6885 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:20.249287  7150 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.146728  6945 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.121047  7104 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.147203  7147 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.258455 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:20.258190 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69145.3us
I20260709 12:20:20.144441  7012 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:20.262634  7098 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:20.250084  6885 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.265997 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.266537  7007 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:20.270452  7115 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.270916  6930 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.210237  7068 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.271638 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.271839 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.272658 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.273046 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.274205 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.274920  7009 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.257964  7119 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.276995 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.277177 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:20.277891  7187 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:20.277936 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72622.5us
I20260709 12:20:20.142422  7117 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.255069  6945 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.274228 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.281384 32113 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:20.282477 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.282900 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.283035 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.283619 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.283671 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.284073 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.285354 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.287029 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.288808 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.289508 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.290408 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.291013 31966 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:20.354017 31971 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:20.293720  7104 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.294423 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.294593  7179 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:20.355062 32041 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:20.295125 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.295575 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.353404 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:20.356098 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 79019.5us
I20260709 12:20:20.296906 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.297441 31964 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 18. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:20:20.234189  7190 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.142377  7097 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.366393 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.367447 32115 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:20.370371 32043 raft_consensus.cc:2368] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:20:20.314904 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.316957 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.318186 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.318393 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:20.327042 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:20.348699 32121 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 19: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:20.363361 32660 raft_consensus.cc:1240] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:20.356698 31967 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
W20260709 12:20:20.373920 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46983.7us
I20260709 12:20:20.374532 32046 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.374804 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:20.378767 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.379396 32039 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:20.382153  7040 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:20.383280 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.383541  7040 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:20.383931 32041 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:20.383014  7103 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:20.384330 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.384819  7103 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.385329  7103 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.386554 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.387104 32045 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:20.383019  7007 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:20.387951 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:20.382797 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 64514.1us
I20260709 12:20:20.388903 32045 raft_consensus.cc:2410] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:20:20.390749 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.385905 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:20.391988 32120 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.382103 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.382455 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.382944 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.382709 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.393337 32044 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:20.363996  7068 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:20.405673 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.399106 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.390743 31968 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:20:20.400180 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.383985  7040 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:20.369405  6930 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.394146 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.380048  7104 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:20.411705 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.421408 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:20.423035 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:20.424281  6973 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:20.425464  6973 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:20.425843  6973 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:20.360142  7147 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.418198 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.428632 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.427646 32118 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:20.353458  7115 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.433910 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.435070 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.415586  7068 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:20.436508 32121 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:20.436231  7068 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.437556  7119 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:20.439442  6955 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.440027  6955 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.412604  7098 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:20:20.443572  7098 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:20.444072  7098 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:20.445147  7098 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.412784  6885 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.446389  6835 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.446897  6835 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.447289  6835 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.446724 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.451632 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.452270 32118 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:20.453154 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.453584 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:20.454669 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89424.7us
I20260709 12:20:20.459260  6842 consensus_queue.cc:1059] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:20.460379 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.460947 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.461645 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.462143  7124 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:20.467221 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.468751 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.469359 32120 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:20.462273 32113 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:20.453926 32118 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:20.454617 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.472427 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.466158 31971 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
W20260709 12:20:20.465560 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.476001 32115 raft_consensus.cc:2368] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:20:20.474112 31964 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:20.478009 32043 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:20.480298 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.477005 32013 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:20.416232  7104 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:20.485739 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.487035 31966 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:20.481854 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.411880  7187 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.491365 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:20.492458 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.460417  6842 raft_consensus.cc:3055] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:20:20.494035 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 71095.1us
I20260709 12:20:20.489955 32013 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:20.500483 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.498996 32044 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:20.499619 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.501896 32041 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.502054 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.462579  7124 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:20.511282 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.512676 32046 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:20.514616 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.514118 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.520880 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.481604  7138 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.527318 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.538190 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.533291 32114 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:20.534947 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.541020 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.541148 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.542742 32121 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:20.543560 32116 raft_consensus.cc:2410] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:20:20.500204  6842 raft_consensus.cc:740] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:20.517906  7124 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:20.530431  7098 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:20.566444  6724 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:20:20.567219  7082 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:20.574090 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:20.557860 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.551477 31964 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:20.587054 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:20.594558 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:20.595566  6773 consensus_meta.cc:287] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.554s	user 0.002s	sys 0.011s
I20260709 12:20:20.595621  7082 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.618795  7082 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:20.622758 32112 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:20:20.598385 31938 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:20.631464 31968 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.626500 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:20.628298 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.628552 31965 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:20.599843 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:20.635377 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:20.636093 31963 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.640609 32118 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:20.645975 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.582139  6877 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.647732 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:20.546864  7138 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.599901  6955 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.610446  7160 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:20.660434 31967 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:20:20.664408 31967 raft_consensus.cc:3055] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:20.664748 31967 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:20.665481 31967 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.666422 31967 raft_consensus.cc:3060] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.605491  6743 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.661183 32121 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:20.674301 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:20.677767 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 77988us
I20260709 12:20:20.542245  7086 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:20.683895 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 168928us
I20260709 12:20:20.548822  6973 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.713940 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:20.660027  7119 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:20.622251  6835 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.660058  6579 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:20.733891 32039 raft_consensus.cc:2393] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:20:20.720501 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.725682 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:20.726997 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72363.8us
I20260709 12:20:20.622416  7104 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:20.727438 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.041s	sys 0.065s
W20260709 12:20:20.734756 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.041s	sys 0.066s
I20260709 12:20:20.730630 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.731431 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:20.736851 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.742069 32044 raft_consensus.cc:2468] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
W20260709 12:20:20.743362 32119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.028s	sys 0.096s
I20260709 12:20:20.743165 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:20.745105 32119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.028s	sys 0.096s
I20260709 12:20:20.725389 31970 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:20.678635  7160 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:20.751947 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:20.676469  7187 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:20.756664 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:20.764133 32656 raft_consensus.cc:1240] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:20.747041  7065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.025s	sys 0.035s
I20260709 12:20:20.773399 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.739161  7117 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.776633 31971 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:20.777050 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.777956 31966 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:20.779677  7147 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:20.780839 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:20.755534  6853 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:20.783020 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.821548 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:20.817375 32044 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:20.726564  7119 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:20.762032  7004 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:20.830188 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:20.767730  7097 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.681625  6877 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:20.781436  7065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.026s	sys 0.035s
W20260709 12:20:20.781579  6982 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.034s	sys 0.029s
I20260709 12:20:20.757993  6983 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:20.839564 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63845.4us
I20260709 12:20:20.776227  7187 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:20.883208 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:20.828369 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59629.9us
I20260709 12:20:20.780220  6579 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:20.782670  6973 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:20.782505  6930 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.747081  6842 consensus_queue.cc:260] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:20.819491  7138 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:20.861828 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 88724.3us
I20260709 12:20:20.911181 31935 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:20.878772  7097 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:20.888374 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.891464  6920 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:20.892069 31972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.050s	sys 0.061s
W20260709 12:20:20.922873 31972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.050s	sys 0.061s
W20260709 12:20:20.878790  6982 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.034s	sys 0.029s
I20260709 12:20:20.889230 32084 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:20.842993  7119 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:20.917677 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:20.885914  6579 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.883406  7115 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:20.932417 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48285us
I20260709 12:20:20.943398 31971 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:20.945366 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:20.950587 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.951777 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.945500  7120 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.952780  7120 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:20.953265  7040 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.000s
I20260709 12:20:20.928542  7138 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:20.945591 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:20.986119 32041 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:21.001348 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.002345 32041 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:21.020227 31967 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:20:21.020920 31967 pending_rounds.cc:85] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 12
W20260709 12:20:21.047408 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 91934.2us
I20260709 12:20:20.938519  6773 raft_consensus.cc:2804] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:20.936088  6579 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:21.059365 32657 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.015s	sys 0.038s
W20260709 12:20:21.059747 32657 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.015s	sys 0.038s
I20260709 12:20:21.059576 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.060613 32041 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:20.955554  6842 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.021288 31967 pending_rounds.cc:107] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:20:21.065877 31967 raft_consensus.cc:2889] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:20.920621  6983 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:21.054934  7103 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.002s
I20260709 12:20:21.068766 31935 leader_election.cc:400] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:20:21.069067 31935 leader_election.cc:403] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:20.954147  7138 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:20.832095  6945 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:20.957141  7120 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:20.930343  6920 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:21.085623 32040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.004s	sys 0.010s
I20260709 12:20:21.058117  6743 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:21.073464 32116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.003s	sys 0.011s
W20260709 12:20:21.088743 32116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.003s	sys 0.011s
W20260709 12:20:21.090493 32040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.005s	sys 0.010s
W20260709 12:20:21.079178  6709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.002s	sys 0.011s
I20260709 12:20:20.953538  7082 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.068632  7190 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:21.100087 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:20.953722  7040 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:21.102895 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 160755us
I20260709 12:20:21.104327 31965 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:20.929307  7160 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.079834  7138 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:21.079066  7099 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.000s	sys 0.014s
I20260709 12:20:21.057952  7115 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:20:21.071210  6773 raft_consensus.cc:697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:21.101043  6709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.003s	sys 0.012s
I20260709 12:20:21.120060 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:21.120028  7197 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:20:21.126214  7040 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.128558  6835 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:21.130700 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46467.6us
W20260709 12:20:21.115954  7099 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.000s	sys 0.015s
I20260709 12:20:21.132712  6920 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:21.134411 32044 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 27: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:21.134780 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.136127 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:21.138015  6773 consensus_queue.cc:237] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.138541  7147 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:21.139114  7147 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:21.139617  7147 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:21.140991 32119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3382) took 1381 ms. Trace:
I20260709 12:20:21.142181  6905 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:21.141386 32119 rpcz_store.cc:276] 0709 12:20:19.765923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.766060 (+   137us) service_pool.cc:224] Handling call
0709 12:20:20.025457 (+259397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:20.050107 (+ 24650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:20.050142 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:20.060210 (+ 10068us) log.cc:844] Serialized 26 byte log entry
0709 12:20:20.748497 (+688287us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:20.748553 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.749649 (+  1096us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.140734 (+391085us) raft_consensus.cc:1685] finished
0709 12:20:21.140753 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.140899 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18934,"raft.queue_time_us":13235,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3142,"thread_start_us":656570,"threads_started":1,"wal-append.queue_time_us":844180}
W20260709 12:20:21.143931  7198 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:20:21.146931  7201 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:20:21.147600  6877 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.150089  6853 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:21.150701  6853 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:21.115631  7088 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.022s	sys 0.021s
W20260709 12:20:21.154038  7088 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.022s	sys 0.022s
I20260709 12:20:21.156116  7104 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.081169  7103 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.157617 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.158244 32652 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:21.159008  7103 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.126245 32047 raft_consensus.cc:2393] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 18: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:20:21.165789 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 228802us
I20260709 12:20:21.166185 32039 raft_consensus.cc:1240] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:20:21.167852 32626 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:21.169365  6909 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:20:21.170671 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:21.171566  6983 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.171566  7132 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:21.172163 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.172317  6983 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:21.172781  6983 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:21.173153  6983 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.173065  6955 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:21.173559  7132 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:21.174185  6955 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:21.179481  6955 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:21.180372  6955 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.181623  6955 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:21.181818  7200 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:20:21.183382  7009 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.091871  7086 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.126089  6945 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:21.178486 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:21.190733 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.197963  7082 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.198153  7119 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.200361  7119 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:21.201294  7203 log.cc:927] Time spent T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:20:21.202425 31963 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.000s
W20260709 12:20:21.202574 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.202802 31963 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:21.203568  6982 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
I20260709 12:20:21.204125 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:21.207340  6885 consensus_meta.cc:287] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.002s
I20260709 12:20:21.207602  7120 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:21.208027  7120 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.208350  7120 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.221037 32652 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
W20260709 12:20:21.224188 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.221978 31966 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
W20260709 12:20:21.191558 32042 consensus_meta.cc:287] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 12:20:21.225793 32045 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:21.192514 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.225793 32042 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:21.227087 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.225977 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.229351 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:21.229952  7124 consensus_meta.cc:287] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.002s
I20260709 12:20:21.230422  7124 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:21.236840 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 458983us
W20260709 12:20:21.235344 32011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.007s	sys 0.006s
W20260709 12:20:21.238443 32011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.007s	sys 0.006s
I20260709 12:20:21.235608 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:21.235198 32013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.008s	sys 0.005s
W20260709 12:20:21.249577 32013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.008s	sys 0.005s
W20260709 12:20:21.249898 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 595437us
W20260709 12:20:21.252750 31968 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.000s
I20260709 12:20:21.253224 31968 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:21.254302 32013 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:21.252677 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.232913  7104 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:21.262012 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.234833  6982 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:21.206132  6858 consensus_queue.cc:1059] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:20:21.259121 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:21.288722 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:21.289023 32038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.027s	sys 0.031s
W20260709 12:20:21.298521 32038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.028s	sys 0.032s
I20260709 12:20:21.256558  6905 raft_consensus.cc:2764] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:20:21.303227 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.323268 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.324116 31966 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
W20260709 12:20:21.320798 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61742.9us
I20260709 12:20:21.261972  7040 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:21.341341 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 398606us
I20260709 12:20:21.344976 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:21.346627 32045 raft_consensus.cc:1240] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:21.288388  7119 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:21.349025 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.350729 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.351509 32039 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:21.353295 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.360235 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.360708 32653 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:21.362439 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.304344  7132 raft_consensus.cc:2764] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:21.294919  6990 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:21.301530  7065 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:21.346241  7040 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.388866 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:21.365998 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.352566  7007 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:21.361599  6743 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:21.352401  6930 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:21.393625 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.394969 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.396667 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.373967  6858 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:21.304672  6982 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.391467  7040 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.361532  7119 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.412597 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.412130 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.416217 32657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47544 (request call id 1380) took 1179 ms. Trace:
I20260709 12:20:21.396119  6920 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:21.443656 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:21.405531  6743 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:21.445513 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:21.446462 31963 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:21.411429  7132 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:21.458880 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46812.7us
I20260709 12:20:21.459815 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:21.456570 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.383148  7065 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.418133  6945 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:21.461123 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:21.438622 32657 rpcz_store.cc:276] 0709 12:20:20.236894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.237033 (+   139us) service_pool.cc:224] Handling call
0709 12:20:20.237330 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:20.237504 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:20:21.060425 (+822921us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.414106 (+353681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:21.414162 (+    56us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:20:21.414207 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.414269 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.416161 (+  1892us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4887,"prepare.queue_time_us":809770,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":3672,"raft.queue_time_us":31231,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":7286,"thread_start_us":1167060,"threads_started":2}
I20260709 12:20:21.472151 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.447167  7004 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:21.445233  6920 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:21.454694  6743 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:21.466877  7199 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:21.464340  6773 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:21.413563  6990 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:21.499248 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84666.6us
I20260709 12:20:21.469331  7068 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:21.468466  6945 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:21.504577 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 43532.7us
I20260709 12:20:21.507660 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:21.489082  7206 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:20:21.482957  6920 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.518221 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:21.525592 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50692.4us
I20260709 12:20:21.488991  7004 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:21.527801 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:21.494486  7160 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.525350 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3242) took 1312 ms. Trace:
I20260709 12:20:21.510978  7068 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.512806  6945 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:21.544992  7214 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:21.516497  6853 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.541194  7068 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:21.547514  7211 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:20:21.547602  6853 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.548062 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.548645  6853 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:21.549495 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.550431  7049 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.550990  7049 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.552052  7160 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.553025  7049 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:21.524479  7124 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:21.555569  7205 log.cc:927] Time spent T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:20:21.556047  7213 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:20:21.556495 32118 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:21.556734  7065 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.556823 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.557147 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.564150 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.565032 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.565443 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.566848 32045 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:21.554281  7160 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.549412  7088 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:21.540637 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:21.513056  7219 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:21.582288 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.583609 32039 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
W20260709 12:20:21.584939 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44386.7us
I20260709 12:20:21.587713  6990 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:21.588060  7223 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:21.588455 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:21.589749 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:21.590054 32013 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.590510 32117 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:21.591404 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.592929  7138 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.593955 31971 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:21.594437  7088 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:21.595439  7088 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.542757  7150 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:21.596812  6858 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:21.597643  7150 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.599838  7150 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.601392 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.602614 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.603648 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:21.604318  6983 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.604734 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.607156  6842 raft_consensus.cc:3055] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:20:21.595820  6579 consensus_meta.cc:287] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
W20260709 12:20:21.577205  6973 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.002s
I20260709 12:20:21.585593 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:21.534659 31972 rpcz_store.cc:276] 0709 12:20:20.201420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.226769 (+ 25349us) service_pool.cc:224] Handling call
0709 12:20:20.227148 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:20.227413 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:20.227821 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.352018 (+1124197us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.377552 (+ 25534us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.513922 (+136370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:21.513977 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:21.514024 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.514082 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.514200 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25142,"prepare.queue_time_us":579121,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":4974,"thread_start_us":1259205,"threads_started":3,"wal-append.queue_time_us":726612}
I20260709 12:20:21.621141  7098 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:21.621560  7098 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.621937  7098 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.626683 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:21.626873 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.627336 31966 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.627499 32039 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:21.618928 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3136) took 1003 ms. Trace:
I20260709 12:20:21.615804 32117 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:21.610968 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:21.630632 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:21.631809 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.607599  6842 raft_consensus.cc:740] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:21.632189 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:21.632544 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:21.634655 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:21.635419 32117 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:21.614665  6983 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.639331 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.640146 32045 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:21.641240 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.640870  6842 consensus_queue.cc:260] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.642030 32039 raft_consensus.cc:2410] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:21.639564 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.644941 32043 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.645969 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:21.646116 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 319122us
I20260709 12:20:21.646028 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.647106 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.648847  6842 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.649058 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.597923  6858 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.610698  7124 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:21.656399 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.662437 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.648512 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:21.674140 32043 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:21.684482 31966 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:21.653678  6945 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.686256 32046 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:21.615324  6709 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:21.629016 32047 rpcz_store.cc:276] 0709 12:20:20.615896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:21.162215 (+546319us) service_pool.cc:224] Handling call
0709 12:20:21.178578 (+ 16363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:21.178769 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:20:21.434607 (+255838us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.618637 (+184030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:21.618686 (+    49us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:20:21.618723 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.618787 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.618881 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17024,"prepare.queue_time_us":304633,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":4639,"thread_start_us":409645,"threads_started":2}
I20260709 12:20:21.615458  7219 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:21.696084  7187 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:21.696929 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:21.698467 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:21.811697 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.814852 31967 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:21.699734 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.697834 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.699887 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:21.654248  7147 consensus_meta.cc:287] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.001s
I20260709 12:20:21.701118  7220 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:21.705236 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58398 (request call id 2944) took 1210 ms. Trace:
I20260709 12:20:21.707042  7004 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.710666  6982 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:21.715196  7230 log.cc:927] Time spent T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:21.715920  6858 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.659360  7086 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:21.716620  6709 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:21.826934  6709 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.654398  6973 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:21.827628 32117 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:21.828114  6973 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:21.828190 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.828855 31963 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:21.829111  6973 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.829716 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.642424  7150 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:21.719233  7119 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:21.724896  7098 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:21.660229  7210 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:20:21.628688  6579 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.726809  6724 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:21.729797 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:21.730516 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.840427 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:21.735095  6945 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.733896 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:21.745416 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:21.747452 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.586156  7120 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.748950 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.754559 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.754760  7234 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:21.755527 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:21.755561 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:21.850837 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 95350us
I20260709 12:20:21.756471  6835 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:21.758563 32114 raft_consensus.cc:2468] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:21.758082  7138 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.770757 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:21.857372 32042 raft_consensus.cc:2368] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 26. Current term is 27.
I20260709 12:20:21.772528 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.773023 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:21.855790 32626 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:21.767264  7068 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.774595 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.775550 31867 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 12 to 17, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 17 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:21.780910 31864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 247) took 10167 ms. Trace:
I20260709 12:20:21.619685  6853 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.787123 32040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2938) took 1321 ms. Trace:
I20260709 12:20:21.880179  7138 consensus_queue.cc:1059] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:21.881062  7138 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:21.881707  7138 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:21.882056  6853 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.880690 32040 rpcz_store.cc:276] 0709 12:20:20.465435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.466589 (+  1154us) service_pool.cc:224] Handling call
0709 12:20:20.488331 (+ 21742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:20.516795 (+ 28464us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:21.101704 (+584909us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.786703 (+684999us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b1800061660. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f228bd774bc 00007f22890c548b 00007f2291f1f9a9 00007f22890e169b 00007f22890df93f 00007f22890df455 00007f228c1c7060 00007f228c1d9716 00007f228c1ea287
0709 12:20:21.786809 (+   106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:21.786852 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:21.786900 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.786968 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.787071 (+   103us) inbound_call.cc:177] Queueing success response
0709 12:20:21.825187 (+ 38116us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b1800061660. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
Metrics: {"prepare.queue_time_us":888253,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":42462,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":23054336,"thread_start_us":945329,"threads_started":3,"wal-append.queue_time_us":419847}
I20260709 12:20:21.787181  6930 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:21.754607  6920 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.883054 32120 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 27: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:21.793439 32115 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:21.883101  6930 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:21.819747  7187 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.841976  6724 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:21.820196 32046 rpcz_store.cc:276] 0709 12:20:20.501360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.537608 (+ 36248us) service_pool.cc:224] Handling call
0709 12:20:21.705066 (+1167458us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400217854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:21.705163 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2502428416}
I20260709 12:20:21.885924  6920 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.788484 32119 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:21.885988 32112 raft_consensus.cc:1240] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:21.823060 31964 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:21.887544 32038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2935) took 1427 ms. Trace:
I20260709 12:20:21.821309  7147 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.887898 32038 rpcz_store.cc:276] 0709 12:20:20.459845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.462394 (+  2549us) service_pool.cc:224] Handling call
0709 12:20:20.462741 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:20.477950 (+ 15209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:20.477979 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:20.524120 (+ 46141us) log.cc:844] Serialized 26 byte log entry
0709 12:20:21.298954 (+774834us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:21.299004 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.299105 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.887323 (+588218us) raft_consensus.cc:1685] finished
0709 12:20:21.887346 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.887477 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8278,"thread_start_us":697994,"threads_started":1,"wal-append.queue_time_us":860738}
I20260709 12:20:21.824203 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:21.825626 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:21.830343 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.836879 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.840996 31965 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:21.854984 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 99539.4us
I20260709 12:20:21.891601 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:21.892171 32654 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.892325 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:21.814785  6743 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.866818  7097 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:21.869053  7068 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:21.840014  7199 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:21.874518  7219 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.854036  7234 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.877535 32121 raft_consensus.cc:2393] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:21.878595  7120 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.879431  6835 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:20:21.880014 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 146206us
I20260709 12:20:21.882486  6990 raft_consensus.cc:3055] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:21.885509  6930 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:21.821638 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:21.822520 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.885985  6724 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:21.899405 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47445.1us
W20260709 12:20:21.900317 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48593.7us
I20260709 12:20:21.876559  6579 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:21.910118 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:20:21.912534 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 138238us
I20260709 12:20:21.913426 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.914386 31972 raft_consensus.cc:2393] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 25: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:20:21.916175 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 79352.2us
I20260709 12:20:21.882124  7138 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.891189  7190 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:21.918078 31970 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:21.886631  7187 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:21.921656 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.907325  6909 raft_consensus.cc:2804] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:21.923681  7118 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:21.906970  7097 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.930089 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:21.931644 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:21.911046  7220 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:21.933297 31966 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:21.916988  7234 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.920528  6885 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:21.918026  6990 raft_consensus.cc:740] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:21.917804  6835 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:21.948231 31965 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:21.965349 32084 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.936947  7118 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:21.969540 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47972.8us
I20260709 12:20:21.971982 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:21.935678  7068 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:21.973397 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:21.938521  7097 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.001087 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:21.963055  6973 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:20:21.932209  7040 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:21.975509  7187 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:21.969386  7118 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:22.004109 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:21.990630 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 49996.8us
W20260709 12:20:21.983870 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 70324.9us
I20260709 12:20:22.009562 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.975955  7068 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:21.990967  6877 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:22.016115  6858 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:22.016558 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.026487 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.007174  7240 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:22.000609  7214 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:22.012676  7238 log.cc:927] Time spent T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:20:22.017452  7068 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:22.040242  6930 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:22.040696 32010 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:22.041129  6743 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.042389  7213 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:22.042455 31938 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:22.042997  7213 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:22.043263  6579 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:22.044003  7213 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.046507  6858 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:22.047739 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:22.048048  6980 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:22.048597  6980 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:22.049469  6980 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.049919  6773 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:22.050385  6773 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.050740  6773 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:22.051185  7218 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:22.051699  7040 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.052174  7218 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:22.052537  7218 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:22.053454 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 43966.5us
I20260709 12:20:22.031725  7103 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.054267 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:22.054497  7103 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:22.055153  7103 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:22.055408 32011 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.056391  7104 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:22.035048  7214 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.058418  7214 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.060485 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:22.061928 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:22.047503  6858 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.049796 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:22.179314 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 129599us
I20260709 12:20:22.053378  7218 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.032785  6990 consensus_queue.cc:260] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:22.033905  7244 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:20:22.089283 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 115031us
I20260709 12:20:21.861301 31864 rpcz_store.cc:276] 0709 12:20:11.616101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.616508 (+   407us) service_pool.cc:224] Handling call
0709 12:20:21.780710 (+10164202us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:14.718528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:14.719282 (+   754us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:14.719345 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:20:14.719372 (+    27us) tablet.cc:662] Decoding operations
0709 12:20:14.866383 (+147011us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:14.884634 (+ 18251us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:14.896781 (+ 12147us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:20:15.330136 (+433355us) tablet.cc:701] Row locks acquired
0709 12:20:15.330163 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:20:15.330334 (+   171us) write_op.cc:270] Start()
0709 12:20:15.330455 (+   121us) write_op.cc:276] Timestamp: P: 1783599615330266 usec, L: 0
0709 12:20:15.330476 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:20:15.331220 (+   744us) log.cc:844] Serialized 10098 byte log entry
0709 12:20:15.864934 (+533714us) op_driver.cc:464] REPLICATION: finished
0709 12:20:16.225256 (+360322us) write_op.cc:301] APPLY: starting
0709 12:20:16.237382 (+ 12126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:16.954254 (+716872us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:16.979857 (+ 25603us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:17.646190 (+666333us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:17.687729 (+ 41539us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=52,mrs_lookups=0
0709 12:20:17.687785 (+    56us) write_op.cc:312] APPLY: finished
0709 12:20:17.688331 (+   546us) log.cc:844] Serialized 228 byte log entry
0709 12:20:17.870275 (+181944us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:17.903884 (+ 33609us) write_op.cc:454] Released schema lock
0709 12:20:17.911577 (+  7693us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:17.941862 (+ 30285us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:17.942967 (+  1105us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36377,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":336465,"apply.run_cpu_time_us":55923,"apply.run_wall_time_us":1713489,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112703,"num_ops":26,"prepare.queue_time_us":514733,"prepare.run_cpu_time_us":81287,"prepare.run_wall_time_us":796430,"raft.queue_time_us":270172,"raft.run_cpu_time_us":48300,"raft.run_wall_time_us":210301,"replication_time_us":534278,"thread_start_us":775416,"threads_started":5,"wal-append.queue_time_us":413235,"wal-append.run_cpu_time_us":133518,"wal-append.run_wall_time_us":2642452}]]}
I20260709 12:20:22.004942  6909 raft_consensus.cc:697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:22.046182 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:21.983616  6955 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 12:20:22.172777 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 104903us
I20260709 12:20:22.183323 32113 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:22.058647  7237 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:22.184911 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:22.326673 32041 raft_consensus.cc:2393] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:22.189759 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:22.190606 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.063213  7240 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:22.014650  7195 consensus_queue.cc:1059] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.198s
I20260709 12:20:22.230098 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:22.226956 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50615.9us
I20260709 12:20:22.194324 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.338373 31964 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:22.229992 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.248556  7147 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.348474 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.349416 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:22.252329  7248 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:20:22.007347  7199 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:22.253495  7246 log.cc:927] Time spent T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:20:22.365885 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:22.259232  7247 log.cc:927] Time spent T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:20:22.264094  7118 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.266918  7098 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:22.267465  7138 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.189280  6835 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.270015  6990 raft_consensus.cc:3060] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:22.290879  6983 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:22.261689  7234 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.293726  6773 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.302600  7160 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:22.192094  7012 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:22.308802 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:22.313650 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:22.419546  7254 raft_consensus.cc:3055] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:22.328859 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:22.334901 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 288748us
I20260709 12:20:22.336084 31966 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:22.336884 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.350524 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:22.354179 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.447207 31969 raft_consensus.cc:2393] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 27: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:22.359197 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:22.350274 31971 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:22.442178 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:22.451295 31938 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:22.452826 31938 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:22.442168 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 113433us
I20260709 12:20:22.455700  7245 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:22.456637  7245 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:22.457470  7245 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:22.459125 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:22.257697  6724 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.363783 32045 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:22.364087 31935 leader_election.cc:400] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 26. Current term is 27.
I20260709 12:20:22.460317 31935 leader_election.cc:403] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
W20260709 12:20:22.460904 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 108223us
I20260709 12:20:22.461758  6724 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.366679 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.349731  6905 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:22.463927  7245 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:22.326063  7237 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:22.464506  7237 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.351184  7240 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.407842 32046 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:22.409154  7118 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.382442  7009 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:20:22.318985 32654 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:22.472996 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:22.243681  7190 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:22.479310  7190 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.366062  6909 consensus_queue.cc:237] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.462671 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:22.506855  6905 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:22.507099 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44516.2us
I20260709 12:20:22.462798  7232 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 26. Current term is 27.
I20260709 12:20:22.510162 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.369176 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:22.515308 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56300.2us
I20260709 12:20:22.518181 31966 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:22.518293 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.518755 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.519068 31938 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:22.519991  6905 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:22.520459  6905 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:22.441102  7254 raft_consensus.cc:740] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:22.520426 31938 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:22.542238 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 69355.8us
I20260709 12:20:22.545240 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:22.558092 32113 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:22.559324 31938 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:22.560904 31971 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:22.391378  6983 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:22.541610  6905 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:22.552300  6877 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:22.551718  7012 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:22.575183  7118 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.575527  7254 consensus_queue.cc:260] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.569531  7245 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:22.590787  7241 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:22.592317 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:22.596917  7138 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.442641  7160 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.440804  7218 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:22.607975  6773 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:22.567008  7098 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:22.610603  7234 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.618052  6835 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:22.621246  7068 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 12:20:22.622017  6983 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:22.632912 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 77136.8us
W20260709 12:20:22.637900 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 273866us
I20260709 12:20:22.691506 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:22.643085 32626 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.035s
I20260709 12:20:22.632331  6982 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:22.692495 32010 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:22.693032 32626 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.000s	sys 0.035s
W20260709 12:20:22.632871  7220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.113s	sys 0.203s
I20260709 12:20:22.702768 32113 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:22.651631  7241 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:22.703976 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 111785us
I20260709 12:20:22.706585 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.707492 31938 leader_election.cc:400] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 19 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20.
W20260709 12:20:22.709726 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 326902us
I20260709 12:20:22.709961 32121 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:22.711149 31938 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:22.645081  7187 consensus_meta.cc:287] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.627s	user 0.005s	sys 0.005s
W20260709 12:20:22.715438 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:22.703434  7220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.121s	sys 0.206s
I20260709 12:20:22.651206  7118 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.699028  7132 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:22.699707  7254 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.715792 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:22.732789 32623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.698553  7098 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:22.705109  7245 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.710093  7241 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.710456  6773 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.726359  6709 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:22.719769  7234 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.727630  7187 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:22.737764  7243 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:22.696839  7068 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.730984  6885 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:22.727913  7220 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:22.772881 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:22.716106  6982 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.731887  6945 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:20:22.783018 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:22.784541 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.784915 32113 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:22.740972  6579 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:20:22.740624  7132 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.789130  6709 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:22.789417  7220 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.737161  7124 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:20:22.796542  7124 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:22.797049  7124 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:22.792191 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:22.749552  6858 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:20:22.789968  7234 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:22.805485 32116 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:22.807258 32113 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:22.810743 32121 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:22.789810  7237 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.812376 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.816680 32013 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:22.746970  7012 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.821393 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.827535 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:22.819988  6945 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:22.812043  6885 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:22.832286 32047 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:22.833483 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.833981 32047 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:22.811385  7104 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:22.834996 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:22.835243 32044 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:22.833773 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.830657 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.837210  7082 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:22.837718  7082 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:22.837846 31972 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:22.838627  7082 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.839331  7150 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:22.840075 32045 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:22.839932  7150 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.828527  6945 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:22.842588  6945 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.843912  6945 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:22.844966  7118 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.845553  7118 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:22.846010  7118 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:22.846402  7275 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:22.793430  7240 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.822366  6773 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:22.824599  7253 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:22.851158 32038 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:22.860108 32045 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:22.846925  7118 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.877321  7150 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:22.884985 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 157710us
I20260709 12:20:22.826939  7256 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:22.831146  7195 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:22.828398  7265 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:22.905236 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61977.9us
I20260709 12:20:22.914361 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.915205 32045 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:22.835932  7120 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:22.920845  7274 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:22.824321  7098 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.927852 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:22.929623 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:22.870169  7104 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.932466 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:22.940574 31970 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:22.933967 32045 raft_consensus.cc:1240] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:22.905745  7160 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:22.946110 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:22.946663 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:22.848912  6885 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.925765  7241 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.901417  7245 consensus_queue.cc:1059] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:20:22.915235  7265 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:22.948958 32652 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:22.947173 31964 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
W20260709 12:20:22.962970  7263 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 12:20:22.964705  7269 log.cc:927] Time spent T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:20:22.964237  7124 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:22.965574  6990 consensus_meta.cc:287] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.002s
W20260709 12:20:22.965777  7262 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:20:22.965909  7124 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:22.967377  7268 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:20:22.967903  7264 log.cc:927] Time spent T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:20:22.979031  7132 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:22.968041  7187 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:22.972882  6905 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:22.973824  7068 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:22.974829 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:22.980160  7187 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:22.980610  7240 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:22.981137  7187 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:22.982506  7187 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:22.982733  7241 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:22.984905  6842 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:22.985350  6842 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:23.011854 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:23.024717 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 145565us
I20260709 12:20:23.025684 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:23.025750 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:23.029764 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55780.4us
I20260709 12:20:22.941638  7274 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:20:23.032636 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:22.956336  7265 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:23.036620 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:22.982740  6905 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:22.918758  7195 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.055724 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:23.048300 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.028923  6842 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.032413  7234 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.051276  6885 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.059970  6842 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:23.062408  7097 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 1.056s	user 0.006s	sys 0.007s
I20260709 12:20:23.062836  7097 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.060340  7234 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.075820 31966 raft_consensus.cc:1240] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:23.097357  6982 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.098513 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.099313 32652 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:23.104382  7195 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:23.105955  7272 log.cc:927] Time spent T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.095s	user 0.001s	sys 0.001s
W20260709 12:20:23.113018  7281 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:20:23.047473  7120 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:20:23.122205 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74005.7us
I20260709 12:20:23.088973  7276 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:23.123330 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.117003  6858 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.133453  7276 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.164217  7276 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:23.167752  7283 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.168398  7283 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.171936  7234 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.173345  7104 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.174826  7237 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.130525  6920 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:20:23.180724 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.181190 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.181437 32120 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:23.135735  6945 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:23.196745 32654 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:23.199050 32047 raft_consensus.cc:2468] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
W20260709 12:20:23.199275 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:23.200639 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 269647us
I20260709 12:20:23.201141 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.201397 32038 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:23.207336 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:23.207669 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.207881 32047 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:23.210142 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.213474 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:23.182266  7097 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.210533 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.219780 32047 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:23.213069 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.154002  7049 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:20:23.233326 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.209080  7237 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.238752 31968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56072 (request call id 2929) took 1615 ms. Trace:
I20260709 12:20:23.208743  7104 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.234879 32084 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:23.247846  7283 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.252439 31968 rpcz_store.cc:276] 0709 12:20:21.623001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:21.633213 (+ 10212us) service_pool.cc:224] Handling call
0709 12:20:23.238553 (+1605340us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54004560d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:23.238694 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2838105856}
I20260709 12:20:23.259806  7104 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.260177  7082 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.262588  7082 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.264603  7082 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.226814  6842 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 12:20:23.255698  7218 consensus_meta.cc:287] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 12:20:23.271456  6955 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:23.271983  6955 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.202533  7290 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:20:23.277068 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:23.281132 31970 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:23.234627  6858 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:23.262135  7104 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.294660 31964 raft_consensus.cc:1275] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:23.297199  7265 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.298014  6990 raft_consensus.cc:3055] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:23.300479  6990 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:23.301679 31963 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:23.301375  6990 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.281359  6773 raft_consensus.cc:2804] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:23.302474  6990 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:23.302749  6773 raft_consensus.cc:697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:23.303467  7290 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:23.303915  7290 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:23.303668  6773 consensus_queue.cc:237] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.304736  7290 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.274163  6955 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.357808 32121 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:20:23.299352  7265 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:23.359612 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.255308  7237 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.371399 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.362154 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:23.335098  7292 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:20:23.363571 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.375808 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.377017 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.379134 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.383947 32112 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:23.387015 32113 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:20:23.382396 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.365094 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.376962 32117 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 27: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:23.391793 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.392166 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.394016 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.393419 32114 raft_consensus.cc:2393] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:23.393213 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.400282 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.401321 32117 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:23.401983 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.398133 32112 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:23.403398 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.404253 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.402076 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.404579 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.402930 32114 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:23.405473 32119 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:23.405851 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.407249  7261 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:23.360543  7004 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:23.416777 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.377624  6920 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.382542  7232 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.419786 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.388509  7278 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.432379 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:23.425797 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.404158  7097 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:23.444440 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.445765 31970 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:23.433205 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.436244 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:23.439424 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.440872  7280 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:23.442492  7254 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:23.442282 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50207.7us
W20260709 12:20:23.477077 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.477983 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.446897  7271 consensus_queue.cc:1048] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:23.410426  7049 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:20:23.420306  7214 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.408385  7253 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:20:23.451309  7256 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:23.451488 32113 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:23.430326 32044 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:23.451808  7232 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.452296  7278 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.458952  6983 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:23.432322  6920 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.428920  7088 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:20:23.467504 32013 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.491009 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.469012  7294 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:23.472190 32114 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: [20.11-21.12]
W20260709 12:20:23.479809 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 46676.8us
I20260709 12:20:23.480799 32121 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:23.496171 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.418256  7265 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:23.407819  7190 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 12:20:23.496708 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.500718 32121 raft_consensus.cc:2393] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 19: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:23.492689 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.504578  7294 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.484103  7254 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.491560  7097 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:23.510872  7254 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.512173  7265 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.514281  7265 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.515466 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:23.517653  7124 consensus_meta.cc:287] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.000s
I20260709 12:20:23.429067  7004 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.520239  7004 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.522452  6945 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:23.523180  6945 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.484628  7120 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:23.527560  6945 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:23.528106 31936 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:23.528383  6930 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.528940  6930 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:23.529340  6930 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.472479  7276 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.532019  7276 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.533869 32657 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:23.534210  7237 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.536615  7232 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.537317 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.538443  6920 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:23.539033 32115 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:23.540035  7124 consensus_queue.cc:1059] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:23.541092  7124 raft_consensus.cc:3055] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:23.541401  7124 raft_consensus.cc:740] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:23.542287  7124 consensus_queue.cc:260] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.543411  7213 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:23.545599 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 67727.6us
I20260709 12:20:23.543663  7278 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.504604 32120 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:23.567569 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:23.550163 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.568357 32121 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:20:23.554970 31967 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.569254 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:23.547669 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.560456 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.480502  7280 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:23.576800 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48783.1us
I20260709 12:20:23.564193 32115 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:23.486131  7256 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:23.560356 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:23.580400 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.561443 32044 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:23.567674 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.570777 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.572649 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:23.556995 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.563012 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:23.578718 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.579212 32114 raft_consensus.cc:2393] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:23.584374 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.593235 32121 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 20: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:23.595952 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.565882 32117 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:23.597594  7234 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:23.597769 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:23.598691  7120 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:23.599576  7280 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:23.600664 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.600967 32040 raft_consensus.cc:3060] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.601320 32042 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 18.
I20260709 12:20:23.543648  7124 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:23.671034 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 72511.2us
I20260709 12:20:23.598685 31971 raft_consensus.cc:2393] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:23.673520 32042 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:20:23.670691 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 74845.3us
I20260709 12:20:23.557770  7271 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:23.674208 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.669947 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:23.669932 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.711010 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:23.704268 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:23.679347 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 116403us
I20260709 12:20:23.715261 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:23.716948 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 114145us
I20260709 12:20:23.563710  7237 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.562506  7283 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:23.734117 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.723557 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.730944 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.741307 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.741514 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:23.751370 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.756217 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:23.669734  7213 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:23.757761 31938 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:23.676422  6920 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:23.680373  7190 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:23.679796  6982 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.739333  7232 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:23.744525  7214 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:23.768536 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 134818us
I20260709 12:20:23.808620 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.766511 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.679546  7234 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:23.725176  6858 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.680207  7063 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:23.570636  7265 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.191s
I20260709 12:20:23.764317 31967 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:23.773088 32120 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:23.814787 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48403.6us
I20260709 12:20:23.767143  7190 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.774220  6973 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.740667  7271 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.771127  6982 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:23.750797  7237 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.771216  7245 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:23.837752  7234 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.838477  7245 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:23.839452  7245 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.774391  7279 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:23.842878  7279 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:23.843931  7279 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.857043  7290 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:23.868885 32115 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:23.871584 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.872664 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.873152 32045 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:23.873879 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.820984  6858 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.875577 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:23.876616 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:23.876853  6858 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:23.880674  6930 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:23.804771  7086 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:20:23.883159  7214 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:23.883531  7214 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:23.884322  7214 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:23.887071 32119 raft_consensus.cc:1240] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:23.888159 31936 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.896284 32040 raft_consensus.cc:1275] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:20:23.829222  7190 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:23.823731  7253 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:20:23.903724 32042 raft_consensus.cc:1240] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:23.904642 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:23.835812  6982 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.925386  6930 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:23.930505  7004 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:23.932771 32120 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:23.900851  7283 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.935789 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:23.936599 32657 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:20:23.936779  7303 log.cc:927] Time spent T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.099s	user 0.000s	sys 0.002s
I20260709 12:20:23.893100  7237 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:23.947744  7004 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:23.949625 32657 raft_consensus.cc:1240] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:23.830101  7195 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.088s
I20260709 12:20:23.913708  7097 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:23.956224 32118 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:23.957960 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.969419  7271 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:23.972146 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 203311us
I20260709 12:20:23.982542  6579 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:23.969975  7302 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
W20260709 12:20:23.987891 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.983142  6579 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:23.989841 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:23.988283 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:23.963209  7004 consensus_queue.cc:1243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:20:23.970237  7097 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:24.002398 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:24.021699 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 255659us
I20260709 12:20:24.027040  7256 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.028746  7256 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.964789  7232 consensus_queue.cc:1059] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:20:24.034778  7232 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.035260  7232 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.036207  7213 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:24.037261  7232 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:24.037853  7232 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:24.037990  6983 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.000s
I20260709 12:20:23.994458  6579 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.047441 32044 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:20:24.111068 32010 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:23.992949  6773 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 12:20:23.999851  6990 consensus_queue.cc:1059] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:20:24.124049  6773 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:24.124567  6773 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:24.124771  7308 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:20:24.125360  6773 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.126681  6773 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:24.126870  7304 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:20:24.128165  7097 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.132855  6579 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.135915  6579 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:24.118424  7276 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:20:24.142109 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.062018  7124 consensus_queue.cc:1048] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.081s
I20260709 12:20:24.148633  6990 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:24.149266  6990 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:24.149505  6579 raft_consensus.cc:3055] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:24.149961  6579 raft_consensus.cc:740] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:24.149729  6982 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.150128  6990 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.150866  6579 consensus_queue.cc:260] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.151499  6990 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:24.152060  6579 raft_consensus.cc:3060] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:24.152719  6982 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.162577  7293 consensus_queue.cc:1059] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:24.163820  7293 raft_consensus.cc:3055] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:24.164273  7293 raft_consensus.cc:740] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:24.165262  7293 consensus_queue.cc:260] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:24.167375  7310 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:20:24.053867  7237 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.098812  7195 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.170343  7195 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.192512 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.193362 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:24.099113  7232 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:24.198096  7232 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.173947  6973 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:24.177574  7190 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.215047  7190 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.168263  7237 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.211220  7195 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.292168 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:24.293035 32039 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:24.288452  7138 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:24.297149  7293 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.303562  7232 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.318397  7294 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:24.321022 32046 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:24.325742 32039 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:24.327394 32623 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:24.333941 31972 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3305) took 1410 ms. Trace:
I20260709 12:20:24.338676 31972 rpcz_store.cc:276] 0709 12:20:22.923170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.923322 (+   152us) service_pool.cc:224] Handling call
0709 12:20:22.923608 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:23.340267 (+416659us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:23.373325 (+ 33058us) log.cc:844] Serialized 14 byte log entry
0709 12:20:23.528307 (+154982us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b18004687c0. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:20:23.752677 (+224370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:23.752745 (+    68us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:23.752803 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:24.333652 (+580849us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540034e254. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:24.333711 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:24.333857 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11695,"prepare.queue_time_us":34730,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":244,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":16074,"spinlock_wait_cycles":974691456,"thread_start_us":321057,"threads_started":1,"wal-append.queue_time_us":523711}
W20260709 12:20:24.345613 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.349244 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.359508  7213 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:24.364416  7214 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:24.375031 32116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3453) took 1195 ms. Trace:
I20260709 12:20:24.381691 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:24.342295  7195 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:20:24.388283 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 398491us
I20260709 12:20:24.391633 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:24.392141  7311 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.396955  7311 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.399297  7311 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:24.404654  7104 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:24.405478  7104 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.410485 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.411533 32113 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:24.413396  7104 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.375581 32116 rpcz_store.cc:276] 0709 12:20:23.180096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:23.182247 (+  2151us) service_pool.cc:224] Handling call
0709 12:20:23.182580 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:23.182994 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:23.390121 (+207127us) log.cc:844] Serialized 14 byte log entry
0709 12:20:23.745722 (+355601us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:24.351365 (+605643us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:20:24.351772 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 12:20:24.351947 (+   175us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:24.351998 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:24.352084 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:24.370644 (+ 18560us) raft_consensus.cc:1685] finished
0709 12:20:24.374762 (+  4118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:24.374957 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3372,"prepare.queue_time_us":300074,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":17511,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":24750,"spinlock_wait_cycles":60009216,"thread_start_us":503244,"threads_started":2,"wal-append.queue_time_us":540654}
I20260709 12:20:24.422423 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:24.423998 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.424875 31966 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:24.430676 31938 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:24.434667  6579 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:24.440032 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.440828 31964 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:24.450727 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.469861  7321 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:24.470763  7321 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:24.471951  7321 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.488597 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.489948 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.491317 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.495044 31967 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:24.500253 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.501250 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.501077 31965 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:24.489527 31966 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:24.509479 31966 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:20:24.510367 31966 pending_rounds.cc:85] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:20:24.510763 31966 pending_rounds.cc:107] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:20:24.511134 31966 raft_consensus.cc:2889] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:24.432557  7290 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.000s
I20260709 12:20:24.517385 31971 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:24.382864  7294 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.122s
I20260709 12:20:24.384344  6973 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.129s
I20260709 12:20:24.431933  7195 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:24.561828 32112 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:24.560657  6773 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.433852  7279 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:24.484076  7232 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:24.567597 31972 raft_consensus.cc:2410] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:20:24.571625 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:24.509292  6930 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:24.579504 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:24.570181 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:24.580890 31969 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:24.581630 32656 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:20:24.579018 31936 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.580814  7256 consensus_meta.cc:287] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.001s
I20260709 12:20:24.586656 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.575201  7232 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.594632 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.596464 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:24.597203 31965 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:24.602301 31967 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:24.603896 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.582454  6930 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.606254 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.606732  6930 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.607359 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:24.608032 32043 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:24.577435  6773 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.610028 32118 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:24.610816 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:24.592546  7232 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.610908 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:24.612876 32043 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:24.580469  7279 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.614835 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:24.616219 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.618472 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.620536 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:24.621218 31965 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:24.620360 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:24.623349 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:24.626286 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:24.616832  7104 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:20:24.617238  7315 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 12:20:24.633306  7274 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:24.617619  6973 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:24.640188 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.645821 32043 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:24.624107  7283 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.656674 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.657459 31964 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:24.643409  7274 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.611006  7195 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.671253 31964 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:24.642385  6973 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:24.643157  6990 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:24.713222  7120 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.716527  7330 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:24.717295  7330 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.670902  6877 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.675400  7274 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.734182 32117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 3469) took 1286 ms. Trace:
I20260709 12:20:24.743755 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.682796  6973 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:24.744648 32116 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:24.743094 32117 rpcz_store.cc:276] 0709 12:20:23.448075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:23.879708 (+431633us) service_pool.cc:224] Handling call
0709 12:20:23.879963 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:23.880325 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:23.880363 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:23.880861 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:20:24.108738 (+227877us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:24.108794 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:24.108879 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:24.733938 (+625059us) raft_consensus.cc:1685] finished
0709 12:20:24.733966 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:24.734113 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16392,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":12229,"thread_start_us":187319,"threads_started":1,"wal-append.queue_time_us":423990}
W20260709 12:20:24.751672 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 282244us
W20260709 12:20:24.747128 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 120897us
I20260709 12:20:24.756634  7330 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:24.762185 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:24.762887 32043 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:24.764055 31969 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:24.764894 31965 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [23.12-24.13]
I20260709 12:20:24.766132 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:24.766541 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.766852 31969 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:24.767750  7330 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:24.767953 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:24.772495 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 148141us
I20260709 12:20:24.773684 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:24.769680 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:24.726938  6990 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.781517 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.782460 32040 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 19.
I20260709 12:20:24.783339  7331 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:24.783955  7331 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:24.783470 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.786140  7326 log.cc:927] Time spent T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:20:24.786186  7331 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.786583  7124 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:24.787045  7124 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:24.787441  7124 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:24.791492 32047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58398 (request call id 2985) took 1502 ms. Trace:
I20260709 12:20:24.752740  7294 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:24.793263 32039 raft_consensus.cc:1275] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:24.792007 32047 rpcz_store.cc:276] 0709 12:20:23.289347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:23.289487 (+   140us) service_pool.cc:224] Handling call
0709 12:20:23.289853 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:23.415710 (+125857us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:23.415741 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:23.416183 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:20:23.757351 (+341168us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:23.757405 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:24.791002 (+1033597us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001950d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:24.791086 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:24.791151 (+    65us) raft_consensus.cc:1685] finished
0709 12:20:24.791171 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:24.791423 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38659,"raft.queue_time_us":31151,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":19096,"spinlock_wait_cycles":247596416,"thread_start_us":325194,"threads_started":1,"wal-append.queue_time_us":523963}
I20260709 12:20:24.804912 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.774451  7086 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:24.745252  6877 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.821734 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.801723  7331 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:24.813958 31935 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.2:40293 timed out after 2.088s (SENT) [suppressed 5 similar messages]
W20260709 12:20:24.876063 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 81438us
I20260709 12:20:24.801656  6983 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:20:24.881814 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 67884.9us
I20260709 12:20:24.879711  6983 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:24.883514  6983 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:24.884367  6990 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.812377  7294 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.886847  7294 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.887951  7294 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:24.888499  7294 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.889204  7280 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:24.891599  6877 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.892206  6877 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.892654  6877 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.894469 32043 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
W20260709 12:20:24.897096 32046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58404 (request call id 3154) took 1985 ms (client timeout 2088 ms). Trace:
W20260709 12:20:24.897488 32046 rpcz_store.cc:269] 0709 12:20:22.911529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.912667 (+  1138us) service_pool.cc:224] Handling call
0709 12:20:24.896954 (+1984287us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540013ced4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:24.897046 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3145352960}
W20260709 12:20:24.899458  7328 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.303s	user 0.003s	sys 0.000s
I20260709 12:20:24.902357 31967 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 12:20:24.902710 31967 pending_rounds.cc:85] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 17
I20260709 12:20:24.902921 31967 pending_rounds.cc:107] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 12:20:24.903127 31967 raft_consensus.cc:2889] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:24.910569 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:24.889786  7280 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.815593  6773 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:24.917426  7293 consensus_meta.cc:287] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.607s	user 0.007s	sys 0.003s
I20260709 12:20:24.901957  7283 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:20:24.921133  7280 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:24.944533  7279 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:24.944799  7124 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.946381 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47766 (request call id 3178) took 1570 ms. Trace:
I20260709 12:20:24.946857 32112 rpcz_store.cc:276] 0709 12:20:23.365276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:23.404035 (+ 38759us) service_pool.cc:224] Handling call
0709 12:20:24.934301 (+1530266us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001234d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:24.935876 (+  1575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659989248}
W20260709 12:20:24.955964 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 149874us
I20260709 12:20:24.957043 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:24.960508 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.961864 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.961956 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.962549 32120 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:24.962800 32117 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:24.965130 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:24.912031  7331 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.981221 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:24.970105  6877 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.989208 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:24.951789  7294 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:24.921811  7232 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:24.972579  6982 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:24.989037  7334 log.cc:927] Time spent T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:20:25.005036  6982 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:24.985332  7097 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.009465 32043 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:25.010040  7213 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:20:25.006292  7234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.165s	user 0.010s	sys 0.019s
I20260709 12:20:25.005818  7338 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:24.987268 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 195940us
I20260709 12:20:25.016682  7213 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:25.018595  7338 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.025512  7097 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.028937  7097 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.029806 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:25.030665 32039 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.995962  6930 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:25.033763  6930 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.036190 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.034230  6930 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:25.039175  6982 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:25.033985  7124 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:20:25.039536  7338 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:25.041699  7237 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:25.045845  7324 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.448s	user 0.001s	sys 0.001s
W20260709 12:20:25.048254  7332 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 12:20:25.048974  7213 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.050285  7213 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:25.051009  6983 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.053047 32043 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:25.055397 32039 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
W20260709 12:20:25.024627  7234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.010s	sys 0.019s
I20260709 12:20:25.063865  7338 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.018949 31966 raft_consensus.cc:2393] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
W20260709 12:20:25.033344  7214 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.668s	user 0.004s	sys 0.007s
I20260709 12:20:25.034192  7120 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.056026  7274 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.058714  7271 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.060299 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:25.061647 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.063416 32112 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:25.063467 32121 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:25.098126 32118 raft_consensus.cc:2393] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:25.098971 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.067030 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.100111 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:25.100658 32039 raft_consensus.cc:2376] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24. Re-sending same reply.
I20260709 12:20:25.068141 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.023084  7232 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:25.072842 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 280923us
I20260709 12:20:25.075022 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:25.104689 31935 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:24.994854  7329 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:25.097272 31970 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:25.098881 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:25.068749 32115 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:25.103229 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:25.053051  6983 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.100566 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.113966 32117 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:25.115185 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:25.117333 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 240142us
I20260709 12:20:25.116758 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:25.120523 32044 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:25.150588 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.120918 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:25.149752 32656 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:20:25.151669 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:25.126991 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:25.122097 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:25.152734 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:25.126302 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:25.130860 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:25.154388 32117 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:25.106976  7329 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.153599 32659 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:20:25.158056 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.158990 32120 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:25.155265 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:25.159909 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.150383  7256 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:25.171190  7256 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.116207  6990 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:25.106907  7336 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:25.103955  7234 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:25.108496  6930 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.106678  7271 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.099869  7124 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.056s
I20260709 12:20:25.175779 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.123807  7097 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.169955  7190 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:25.120146  7350 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:25.155220  6773 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:25.197314 32120 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:25.160542  7232 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:25.166522  7329 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:25.169562  7321 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:25.121125  7214 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:20:25.181809  7293 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:25.221890 32114 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:25.222038  6579 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:25.116008  6973 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.180994  7277 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.177767  7344 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:25.124197  7120 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:20:25.151037  6982 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
W20260709 12:20:25.209652 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59155.1us
I20260709 12:20:25.212347 31964 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:25.193256  7195 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:25.223210 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 101190us
I20260709 12:20:25.192350 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:25.204171  6773 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.231364 32010 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.231400 32084 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:25.226641 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 242622us
I20260709 12:20:25.174162  7341 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:20:25.209836  7337 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:25.236809 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.196475  7234 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.240325 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.231026  7344 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.248162 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:25.236522  6773 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.255038  7363 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:25.218443  7190 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.259141  7088 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:25.259250  7275 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:25.244779  7293 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:20:25.261224  6973 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:25.263346  7271 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:25.264668  6930 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.234998  6990 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:20:25.267367  7338 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.236591  7337 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.271147 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.272126 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:25.272325 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.257684  7256 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:25.275081 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 93270.1us
I20260709 12:20:25.242803  7237 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.273299  7330 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.193984  7336 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.249804  7344 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:25.285634 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 54279.7us
I20260709 12:20:25.236362  7277 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.235562  7195 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.287495 32117 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:25.213291  7350 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:25.290829 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 100500us
I20260709 12:20:25.322731 31935 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:25.296651 32114 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:25.302945 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:25.319046 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46827.3us
I20260709 12:20:25.282682  7280 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 12:20:25.324133 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.301751 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:25.319311 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.297091  7274 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.297549  7337 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:25.304576 32654 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:20:25.400775 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78129.2us
W20260709 12:20:25.397920 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 79625.1us
I20260709 12:20:25.300004  7351 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:20:25.402987 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 78940.6us
I20260709 12:20:25.403715 32010 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:25.324205  7276 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.011s	sys 0.012s
I20260709 12:20:25.322824  7237 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:20:25.408329 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 89122.6us
W20260709 12:20:25.413296 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 165223us
I20260709 12:20:25.415733 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:25.445967 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.400170  7234 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.404556  7351 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.530922  7351 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:25.534587  7372 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.534803  7256 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:25.535990  7256 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.538509  7256 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:25.539768 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 128409us
I20260709 12:20:25.541518 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.541839 32114 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
W20260709 12:20:25.408425  7276 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.011s	sys 0.013s
I20260709 12:20:25.543973 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.316339  7330 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.409862  7348 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:25.408381  7321 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:25.453183  7278 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.103s
W20260709 12:20:25.546504  7361 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:20:25.413323  6973 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:20:25.560508  7213 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260709 12:20:25.563763  7331 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.551792  7237 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:25.569434  7336 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.576475  7368 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:25.580763  7350 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.585407  7329 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.552481  7120 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.587134 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:25.587708  7190 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.587888 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:25.590159  6877 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:25.648936 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61897.7us
I20260709 12:20:25.595085  7330 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.598750  7195 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.605100  7321 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:25.551008  7371 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:20:25.549891  7088 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.619234 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.564850 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.620254  6973 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:25.620604 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:25.592905  7337 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.651155 32042 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:25.654361 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.696430 31967 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:25.656118 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.653587 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.658532 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:25.688565 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.681588 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:25.688580 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.698355 31968 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:25.689325 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.699276 32045 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:25.681988 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.699765 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:25.690971 32046 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:25.700326 31972 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:25.704221 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45740.6us
I20260709 12:20:25.657904  7336 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.707671 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.710706 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:25.662127  7097 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:25.711411 32039 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:25.694150  7361 log.cc:927] Time spent T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:20:25.729508 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:25.730474 31963 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:20:25.734807 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.734968 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:25.702713  7088 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.663360  7120 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.713507  7280 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:25.665673  6877 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.659574  7368 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.729298  7097 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:25.758056 32041 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:20:25.754904 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.762799 32118 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:25.765440 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:25.766033 31971 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:25.766362 32039 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:25.767006 32041 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:25.725323  7336 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.747819  7280 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.779088  6773 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.794920  7371 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:25.795534  7371 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.797739  7371 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.794800 32120 raft_consensus.cc:1240] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:20:25.712275  7321 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.801633 31971 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:25.804145  7336 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.793622 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.807340 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.807365 32045 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:25.808028 32042 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:25.805286 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.811547 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.813200  7088 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.814849 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.816771  7337 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.817791 31963 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:25.819011  6773 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:25.820359  6858 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:25.820892  6858 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.822962  6858 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.823204  7342 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:25.823947  7369 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.075s
I20260709 12:20:25.824576  7342 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:25.824954  7342 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:25.825735  7342 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.827986  7342 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:25.830719 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:25.724709  6973 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.755239  7214 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.773674  7280 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:25.887355 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:25.889017 32651 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:25.890718 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.891853 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:25.897795 32654 raft_consensus.cc:1240] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:25.805825  7368 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:25.899591 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:25.903229 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.904359 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.904937 32038 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:25.909658 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.911895 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.912520 32038 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:25.913545 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.910238 32042 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:20:25.914813  6773 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:25.794798  7377 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.143s
I20260709 12:20:25.918237 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:25.884588  7120 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:25.772456  7351 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:25.941547  7277 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:25.900449  7138 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:25.947312 32042 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:25.948393 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:25.948802 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:25.949422 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:25.954245 31972 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:25.964097 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:25.964511 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:25.965440 31972 raft_consensus.cc:2410] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:20:25.894039  7214 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:25.900331  7311 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:25.801928  7345 consensus_queue.cc:1048] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.253s
I20260709 12:20:25.912988  7336 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.893690  7337 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:20:25.959393 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:25.968145 32038 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 23: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:25.970999 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.028851 31972 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:25.965929 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.029762 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:26.030481 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64652.6us
W20260709 12:20:26.023380 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 72212.9us
I20260709 12:20:25.968500  7086 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:26.031397 32623 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:25.969235  7344 consensus_meta.cc:287] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.000s
I20260709 12:20:26.051466 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.054530 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.051604 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:26.054936 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:25.970038  7330 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:26.056967 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:26.058179 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 98877us
I20260709 12:20:25.969681  7331 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:26.031665  7190 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:26.059212 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:25.978417  7234 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.905638  7097 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.061628  7366 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:26.056911 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:25.962001  7138 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:26.077172  7138 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.061300 31972 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:26.063815 32010 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:26.079438 31972 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:26.024205  7120 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:25.969904  7354 consensus_queue.cc:1059] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 12:20:26.079769 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:25.968743  7346 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.104s
I20260709 12:20:26.083139 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.061441  7363 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:26.056422  7362 raft_consensus.cc:2804] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.092500 32010 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:26.061546  7331 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:26.080008  7377 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:26.066722  7104 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:26.087345  7356 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:26.079447  7214 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.085196  7371 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:26.058763  7086 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.087729  7343 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:26.030752  7345 consensus_queue.cc:1243] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer f05e4741d35c4cf3a31358c56e65b30c log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:26.101446  7213 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:26.096597  7195 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:26.095992  7283 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.096391  7363 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.059970  7344 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.100294  7330 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:26.083045  7234 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.098748  7331 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:26.032192  7336 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.097536  7313 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:26.129279 31938 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.132771 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.133023 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.100068  7377 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.095651  7120 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.138167  6990 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:26.138540  7086 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:26.101481  6973 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:20:26.103039  7362 raft_consensus.cc:697] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:26.092272  7237 consensus_queue.cc:1059] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:20:26.117074  7371 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.149103  7283 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:26.146420 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:26.146148 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45204.1us
I20260709 12:20:26.118952  7104 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:26.150132 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:26.130276  7343 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:26.134135  7213 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.134424  7195 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.152704 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.136092  7363 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.142495  7330 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:26.147857  7374 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:20:26.141077  7384 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:20:26.155042  7377 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:26.170585  7351 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:26.172891 32117 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:20:26.180022  7385 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:20:26.160760  7245 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:26.177614  7195 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:26.201166  7311 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:26.201628  7311 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:26.159603  7356 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:26.166705  7371 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.174853  7343 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.204583 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:26.177117  7213 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.159974  7280 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:26.207134 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 54507.5us
I20260709 12:20:26.213498 31938 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.202587  7104 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.211716  7311 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:26.229449 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.221276  7283 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.153414  7313 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.235616  7313 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:26.220534  7375 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.667s	user 0.003s	sys 0.003s
I20260709 12:20:26.238037  7120 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.238498 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:26.239308  7362 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.241042  7351 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.241585  7351 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:26.246757  7395 log.cc:927] Time spent T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:20:26.249140  7336 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.187767  7374 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.251273  6877 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.253613 32046 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58404 (request call id 3178) took 1247 ms. Trace:
I20260709 12:20:26.255975  7374 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:26.217315  7330 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.259452  7330 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:26.262910  7344 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.264760 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:26.265076 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:26.257400  6983 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:26.246037 31970 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56088 (request call id 3326) took 1750 ms. Trace:
I20260709 12:20:26.269033 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.272646 32038 raft_consensus.cc:2393] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:26.243633  7351 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.271715 31970 rpcz_store.cc:276] 0709 12:20:24.496026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:24.496595 (+   569us) service_pool.cc:224] Handling call
0709 12:20:24.496952 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:25.105453 (+608501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.245805 (+1140352us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54000f7654. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:26.245967 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379795200}
I20260709 12:20:26.253984 32046 rpcz_store.cc:276] 0709 12:20:25.005852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.006119 (+   267us) service_pool.cc:224] Handling call
0709 12:20:25.007871 (+  1752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:25.700643 (+692772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.253396 (+552753us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b5400450954. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:26.253558 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1570415488}
I20260709 12:20:26.276420 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.281714 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.287914 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.223270  7399 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:26.297037 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:26.271510  6983 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:26.310588 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.290360  7277 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:26.313843  7271 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.316006  7138 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.333274  7213 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:26.337801 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 113881us
I20260709 12:20:26.300186  7351 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.305022  6858 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:26.219357  7356 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.385608 31971 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:26.390846  7138 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.375407  7256 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:20:26.405468 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.306150  7399 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.349093  7271 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.373740  7195 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.408773  7256 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.414691  7280 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.436959  7195 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.408739  6858 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.437597  6858 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.438659 32654 raft_consensus.cc:2368] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:26.439765  6858 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.416488  7399 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:26.397195  6983 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.441326  6983 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.436983  7256 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.444134 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:26.445576  7379 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:26.448837  7331 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.418378 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:26.410426  7120 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.380762  7351 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.427541  7213 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.388499  7350 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:20:26.432662  7271 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:26.434417 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:26.460971 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.456935 32043 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 25: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:26.457386 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.404520  7371 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.464458 32120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 1253) took 1394 ms. Trace:
I20260709 12:20:26.464907 32120 rpcz_store.cc:276] 0709 12:20:25.070529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.204724 (+134195us) service_pool.cc:224] Handling call
0709 12:20:25.205155 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:25.798794 (+593639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.464193 (+665399us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54001487d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:20:26.464398 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1355407232}
I20260709 12:20:26.465476  7351 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.460733  7120 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.467139  7377 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.468003  7120 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.468665 32113 raft_consensus.cc:1240] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:20:26.469298  7377 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.472270  7343 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.486835  7343 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.493752  7371 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:26.495385  7345 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.496305 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.496873  7088 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.497407  7088 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.498692 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.499558 32115 raft_consensus.cc:2410] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:26.502005  7331 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.496104  7345 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.504649  7311 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:26.510130 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:26.510612 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:26.513191 32043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:26.513983 32043 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.514287 32043 raft_consensus.cc:614] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:26.514964  7088 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.498170 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.493553  6973 raft_consensus.cc:3055] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:26.495426  7371 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.505084  7345 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.519307 32112 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:26.527599 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67260.9us
I20260709 12:20:26.531869 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.520545  6973 raft_consensus.cc:740] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:26.537869 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.540081 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.541213 32112 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:26.535837 32119 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:26.543711 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:26.544272 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.544714 32045 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:26.545723 32117 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:26.547910 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:26.520942  7397 log.cc:927] Time spent T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.326s	user 0.003s	sys 0.000s
I20260709 12:20:26.552078 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.552753 32116 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.554190 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.556025 32084 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:26.557331 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.561348 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.562013 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:26.562721 32040 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.557936 32044 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:26.561269 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:26.562129 32042 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:26.567008 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.567186 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:26.575994 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.576769 32039 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:26.580073  7379 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:26.542462  7371 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.599089 32116 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:26.600383 32040 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
W20260709 12:20:26.606822  7290 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.360s	user 0.006s	sys 0.013s
I20260709 12:20:26.596689  7369 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:26.518298  7363 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.628636 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.630795 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.628232 32045 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:26.570250  7271 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:20:26.654520  7120 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:26.655012  7120 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:26.639653  6973 consensus_queue.cc:260] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.656015  7120 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.657728  7120 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.659018  7371 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:26.661018  7234 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.668792  7329 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.642122 32623 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:26.674960  7346 raft_consensus.cc:2804] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:26.675504  7346 raft_consensus.cc:697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:26.676362  7346 consensus_queue.cc:237] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.680576 31969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 3350) took 1024 ms. Trace:
I20260709 12:20:26.681027 31969 rpcz_store.cc:276] 0709 12:20:25.655788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.655895 (+   107us) service_pool.cc:224] Handling call
0709 12:20:26.680391 (+1024496us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400313b54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:26.680520 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410292608}
I20260709 12:20:26.641853 32045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:26.641927 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.639837  7195 consensus_queue.cc:1059] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:20:26.605732  7256 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:20:26.653292  7337 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.684255 32045 raft_consensus.cc:606] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.688956 32045 raft_consensus.cc:614] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:26.689146 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47399.4us
W20260709 12:20:26.641170  7290 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.006s	sys 0.013s
I20260709 12:20:26.692533  7338 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:26.695622  7396 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:26.696404 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.698650  6990 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:26.698822 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 485360us
W20260709 12:20:26.699482 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 444383us
I20260709 12:20:26.699610 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:26.700802 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:26.701495 31936 leader_election.cc:400] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:26.701848 31936 leader_election.cc:403] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:26.702385  7278 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:26.702778  7278 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:26.703001  7279 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:26.703183  7278 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:26.703372  7313 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:26.704887  7313 raft_consensus.cc:3060] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:26.702639 31938 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.707799 31938 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:26.714761 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.716353 31968 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:26.724783 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:26.726764 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.727391 31968 raft_consensus.cc:3060] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.738205 32651 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:26.726047 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:26.741250  6773 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:26.742117  6773 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.742596  6773 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:26.744999 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.745877 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.741251 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:26.749712 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:26.750754  7367 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:26.753953 31969 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:26.754527 31970 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:26.751257  7367 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.756168 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.756858 32656 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:26.759085  7367 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.755762  7232 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:26.773928 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 135098us
I20260709 12:20:26.778209  6773 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.755721  7417 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:26.805199 31968 raft_consensus.cc:2468] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:26.753149  7377 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:20:26.758062  7415 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
W20260709 12:20:26.761720  7418 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
W20260709 12:20:26.813972 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.752559  7234 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:26.870182 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:26.871428 32038 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:26.751948  7086 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:26.818683 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 70563.4us
I20260709 12:20:26.844570  6773 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:26.846377 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 138665us
I20260709 12:20:26.810779  7374 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:26.888764 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.815742  7335 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:26.844412  6990 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:26.856259 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.926218 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.780753  7368 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:26.780597  7097 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.886474  6579 consensus_queue.cc:1059] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:26.887271 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.847244  7377 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.846462 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:26.901458 32624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.918697 31965 raft_consensus.cc:3060] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.892289 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.759086  7004 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:26.923816 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.924151 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.890033 31972 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:20:26.859302  7341 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.748128  7337 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.935390 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:26.953001 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:26.860920  7417 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:26.937067 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:26.962153 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:26.963392 32045 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:26.859412  7369 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:26.948637 32651 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:20:26.818616  7293 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 12:20:26.977662 32040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:26.978329 32040 raft_consensus.cc:606] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:26.978653 32040 raft_consensus.cc:614] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:26.958734 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 112551us
I20260709 12:20:26.980144 32013 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:26.943082  7414 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:20:26.974942 31965 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:26.845716  7408 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.150s
I20260709 12:20:26.983896 32013 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:26.986989 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.949296  7377 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:26.990785 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.000547 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.008291 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:27.009624 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:27.011523 32118 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28. Ops: [27.18-27.18]
W20260709 12:20:27.020954 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:27.021355 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:26.919795  6990 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.023170 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:26.877095  7232 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.909700  7086 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:27.054401 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 263498us
I20260709 12:20:26.973307  7403 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:26.998783  7423 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:20:26.888401  7234 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:26.919703  7335 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:27.075642 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 190932us
I20260709 12:20:27.067250 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.054365  7412 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:20:27.067567 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 182431us
I20260709 12:20:27.017287  7343 raft_consensus.cc:2804] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.962129  7341 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.950860  7004 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:26.956734  7097 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.059285  7271 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.992744  7348 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.082s
I20260709 12:20:27.095898 31963 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 16.
I20260709 12:20:27.101933 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:27.114286  7097 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:27.116395  7097 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:27.117043  7097 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.120349  6990 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:26.988914  7369 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.125818 32120 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:27.089696  7412 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.113955  7004 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:27.137101 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 121542us
I20260709 12:20:27.069608  7424 consensus_queue.cc:1059] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.111s
I20260709 12:20:27.106490  7363 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:27.144166 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 121954us
I20260709 12:20:27.109545  7343 raft_consensus.cc:697] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:27.120791  7195 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:27.109987  7271 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:27.141167  7419 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 12:20:27.162868 32113 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:20:27.167639 32113 pending_rounds.cc:85] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 11
I20260709 12:20:27.167951 32113 pending_rounds.cc:107] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:20:27.168179 32113 raft_consensus.cc:2889] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.168344 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.168891 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.142227  7412 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:27.174088  7271 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:27.178766  7420 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:20:27.179821  7399 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:27.181123  7232 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.187255  7086 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.187604  7430 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 12:20:27.201309 32120 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:27.203233 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.225301 32121 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:27.208719  7311 consensus_meta.cc:287] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.000s
W20260709 12:20:27.213344  7120 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.002s
I20260709 12:20:27.191004  7428 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:27.248392 31972 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:20:27.247215 32121 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:20:27.269860 32121 pending_rounds.cc:85] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:20:27.238286 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.270123 32121 pending_rounds.cc:107] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:20:27.270398 32121 raft_consensus.cc:2889] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.274281 31938 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:27.275666 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:27.276939 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:27.279032  7086 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:27.279654  7086 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.280140  7429 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:27.280859  7429 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.281903  7086 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.282485  7232 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:27.282584  7396 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:27.283176  7232 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:27.283567  7372 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:27.283996  7372 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:27.284837  7372 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.285935  7372 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:27.290983 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:27.291427 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53258.3us
I20260709 12:20:27.292102 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.291271 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.297134 32117 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:27.295437 32121 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:27.304540 32038 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:27.308211 32115 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:27.310971 31963 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:27.098078  7417 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.315565 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:27.316465 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.323527 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.295588  7428 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.318054 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.253362  6973 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:27.337695  7278 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.000s
I20260709 12:20:27.338541  7329 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:27.365736 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:27.338874  7335 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:27.340309  7097 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.282608  7341 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:27.342592  7234 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:27.343431  7337 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:27.345366  7256 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:20:27.348968  7367 consensus_meta.cc:287] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:20:27.348675  7343 consensus_queue.cc:237] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.314291  7366 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:20:27.351599 32041 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:20:27.354393 32114 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:20:27.353587 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.353595 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:27.311968  7276 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:27.364972 32013 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.372247 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:27.387601 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 320401us
I20260709 12:20:27.431546 31972 pending_rounds.cc:85] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 11
I20260709 12:20:27.355574 32084 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:27.441941 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.481673 32045 raft_consensus.cc:2393] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:20:27.481652 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 116760us
I20260709 12:20:27.483510 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.486745 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.486975 32040 raft_consensus.cc:2468] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:27.529942 32623 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:27.530400 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:27.534085 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.487623 31972 pending_rounds.cc:107] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 22.12
W20260709 12:20:27.492113 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 138594us
W20260709 12:20:27.507372 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 151892us
I20260709 12:20:27.589507 31972 raft_consensus.cc:2889] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.590287 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.532578  7335 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:20:27.573648 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 279185us
I20260709 12:20:27.549036  7341 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:27.373405  7329 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.615505  7362 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:27.631505  7329 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:27.545231 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:27.631925  7274 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:27.554679 31967 raft_consensus.cc:1240] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:20:27.634686  7097 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:27.634852 31867 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 232) took 58495 ms (client timeout 15000 ms). Trace:
I20260709 12:20:27.635288  7097 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.640172  6982 raft_consensus.cc:3055] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:27.640668  6982 raft_consensus.cc:740] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:27.644125  6990 raft_consensus.cc:3055] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:27.644557  6990 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:27.458679  7367 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.542897  7344 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:27.371713  7278 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.623268 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.677768 31970 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
W20260709 12:20:27.627624 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:27.678619 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:27.618544 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:27.626695  7429 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.648859  7280 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:27.637435  7097 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.522560  7369 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:27.621898  7313 consensus_meta.cc:287] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.917s	user 0.012s	sys 0.000s
W20260709 12:20:27.685247 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 66763.1us
I20260709 12:20:27.563361  7237 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.096s
W20260709 12:20:27.635396 31867 rpcz_store.cc:269] 0709 12:19:29.153898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.208092 (+ 54194us) service_pool.cc:224] Handling call
0709 12:20:27.634744 (+58426652us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:22.393138 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:22.393807 (+   669us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:22.393866 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:20:22.393887 (+    21us) tablet.cc:662] Decoding operations
0709 12:20:22.394285 (+   398us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:22.394399 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:22.394432 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:22.394756 (+   324us) tablet.cc:701] Row locks acquired
0709 12:20:22.394807 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:20:22.394964 (+   157us) write_op.cc:270] Start()
0709 12:20:22.395127 (+   163us) write_op.cc:276] Timestamp: P: 1783599622394901 usec, L: 0
0709 12:20:22.395168 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:20:22.396085 (+   917us) log.cc:844] Serialized 530 byte log entry
0709 12:20:23.163336 (+767251us) op_driver.cc:464] REPLICATION: finished
0709 12:20:23.221931 (+ 58595us) write_op.cc:301] APPLY: starting
0709 12:20:23.222061 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:23.224153 (+  2092us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:23.224296 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:23.225255 (+   959us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:23.225608 (+   353us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:23.225693 (+    85us) write_op.cc:312] APPLY: finished
0709 12:20:23.291891 (+ 66198us) log.cc:844] Serialized 25 byte log entry
0709 12:20:23.359852 (+ 67961us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:23.371205 (+ 11353us) write_op.cc:454] Released schema lock
0709 12:20:23.383481 (+ 12276us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:23.407509 (+ 24028us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:23.407542 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1322093,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":58238,"apply.run_cpu_time_us":5354,"apply.run_wall_time_us":185568,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8303,"num_ops":1,"prepare.queue_time_us":360514,"prepare.run_cpu_time_us":32686,"prepare.run_wall_time_us":375682,"raft.queue_time_us":197960,"raft.run_cpu_time_us":3973,"raft.run_wall_time_us":58544,"replication_time_us":767965,"thread_start_us":868945,"threads_started":4,"wal-append.queue_time_us":452369,"wal-append.run_cpu_time_us":81968,"wal-append.run_wall_time_us":1575067}]]}
I20260709 12:20:27.686616 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.721761 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.726779 31970 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:27.727824 32120 raft_consensus.cc:1240] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:20:27.722687 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.723529 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.725060 31964 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:27.731288 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.757278 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.758260 31970 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:27.641697  6982 consensus_queue.cc:260] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.687992  7280 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:27.731053  7356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.019s	sys 0.019s
I20260709 12:20:27.769462 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:27.755980  7377 consensus_meta.cc:287] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.000s
I20260709 12:20:27.653352  7437 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:27.715162  7346 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:27.732589  7313 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
W20260709 12:20:27.815939 32624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.038s
W20260709 12:20:27.797011  7356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.020s	sys 0.020s
I20260709 12:20:27.685933  7341 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:27.730887  7274 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:20:27.822715 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:27.811336  7427 log.cc:927] Time spent T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:27.685578  7195 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:27.814136  7374 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:27.828253 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 294804us
W20260709 12:20:27.823971 32624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.008s	sys 0.038s
I20260709 12:20:27.689669  7256 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:27.830693 31969 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:27.844183  7280 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.825420  7442 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:27.818969  7377 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.645439  6990 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.816792  7429 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:27.813331  7396 consensus_queue.cc:1059] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
W20260709 12:20:27.850103 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 320246us
I20260709 12:20:27.866627 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:27.851341 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:27.867797 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.868419 31963 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:20:27.684875  7338 raft_consensus.cc:3055] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:27.895139  7338 raft_consensus.cc:740] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:27.896540 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:27.857744  7280 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:27.899951 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.901464 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.847191  7256 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.904951  7256 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:27.907208  7372 consensus_meta.cc:287] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.000s
I20260709 12:20:27.908016 32116 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:20:27.908371 32116 pending_rounds.cc:85] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 13
I20260709 12:20:27.908573 32116 pending_rounds.cc:107] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:20:27.827876  7346 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.908800 32116 raft_consensus.cc:2889] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.857865  7442 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.878477 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:27.878589 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:27.879238 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:27.879630 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:27.880806 32659 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 25. Current term is 26. Ops: []
W20260709 12:20:27.883066  6973 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.000s
I20260709 12:20:27.844225  7195 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:27.855134  7374 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:27.887104  7445 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 12:20:27.888003  7446 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:20:27.888648  7439 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:20:27.891261 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:27.891585  7348 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:27.893908  6983 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:27.896572  7341 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:27.958739 32660 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:27.959836  7374 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.960716  6983 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.961547  7195 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:27.962978  6983 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:27.830591  7138 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:27.963649  7337 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:27.964969  7367 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:27.848047  7428 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:27.954668 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:27.966559 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:27.896288  7338 consensus_queue.cc:260] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:27.968408 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:27.911752 32117 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:27.930641 31967 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:27.973326 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:27.974118 31967 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:27.898202 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.976292 31965 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:27.911492 32042 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:27.912273 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.997484 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.998265 31963 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:27.965857  7138 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:28.004040 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.005260 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.011050 32047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:28.016454 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.017659 32047 raft_consensus.cc:606] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:27.950554  6990 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:27.950282  7442 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:28.006037 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:27.966249  7429 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:28.018097 32047 raft_consensus.cc:614] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:28.005899 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.018636 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.014714 32045 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 27: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
W20260709 12:20:28.004977  7412 consensus_meta.cc:287] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.740s	user 0.009s	sys 0.001s
I20260709 12:20:27.920853 32116 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:28.026859 31964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56088 (request call id 3371) took 1318 ms. Trace:
I20260709 12:20:28.016937 32040 raft_consensus.cc:2393] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:28.083889 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 165933us
I20260709 12:20:28.030215 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.017215  7138 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:28.060297 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:28.016685 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 62224.7us
I20260709 12:20:28.095579 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.099037 31967 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:28.100574 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.086607  7346 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.102037 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.102787 32120 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 28: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:28.103734  7363 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.104280  7363 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.082880 31964 rpcz_store.cc:276] 0709 12:20:26.708104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.744770 (+ 36666us) service_pool.cc:224] Handling call
0709 12:20:28.026713 (+1281943us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54000f5d54. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:28.026807 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252413312}
W20260709 12:20:28.087512 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52932.3us
I20260709 12:20:28.018338  7362 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.107563  7363 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.108913  7377 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:27.998011  7348 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.116154  7348 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.116626  7348 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.117113  7348 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.119709  7195 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.120767 32656 raft_consensus.cc:1240] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:28.025131  7278 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:28.085978  7446 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:20:28.107918  7362 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.094735 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:28.182078 32651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.013s
W20260709 12:20:28.191839 32651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.000s	sys 0.013s
W20260709 12:20:28.194170 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 328669us
W20260709 12:20:28.195211 32010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.020s	sys 0.003s
W20260709 12:20:28.195590 32010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.020s	sys 0.003s
W20260709 12:20:28.195927 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 542198us
I20260709 12:20:28.198343 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:28.199072 32047 raft_consensus.cc:2393] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:28.198100 32040 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:28.208585 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.208969 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.211598 32113 raft_consensus.cc:2393] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 26: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:28.212379 32112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.008s	sys 0.011s
W20260709 12:20:28.214226 32115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.004s
W20260709 12:20:28.214581 32115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.012s	sys 0.004s
W20260709 12:20:28.212723 32112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.008s	sys 0.011s
I20260709 12:20:28.215193 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:28.221915  7442 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:20:28.225386 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 85628.5us
W20260709 12:20:28.226356 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.188732  7280 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:28.192135  7279 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.095161  7313 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 12:20:28.194969  7335 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:28.092475  7412 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:28.244410 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 149748us
I20260709 12:20:28.211881  7344 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:20:28.246274  7234 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:28.264891 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 249372us
I20260709 12:20:28.236979  7245 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:20:28.246721  7279 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.280443 31936 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:28.281518 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.257404  7367 consensus_queue.cc:1059] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:20:28.275519 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:28.276175 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71081.2us
I20260709 12:20:28.308679  7367 raft_consensus.cc:3055] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:28.309058  7367 raft_consensus.cc:740] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:28.309878  7367 consensus_queue.cc:260] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.277043 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.311106  7367 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:28.311257 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.243884  7280 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.315016 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.281734 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.272115 32116 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:28.284264  7412 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:28.284379  7454 log.cc:927] Time spent T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:20:28.285681  7452 log.cc:927] Time spent T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:20:28.287638  7335 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.297993  7348 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:28.300130  7279 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:28.301831  7372 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.302727  7195 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.303664  7362 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:28.307686 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.245147  7429 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.318647 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:28.319728 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.315768  7280 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:28.348932 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.326166  7438 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:28.324401  7363 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:28.349766 32112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 3225) took 1361 ms. Trace:
I20260709 12:20:28.350473  7329 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.363644 32112 rpcz_store.cc:276] 0709 12:20:26.988352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.989667 (+  1315us) service_pool.cc:224] Handling call
0709 12:20:26.990030 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:27.002797 (+ 12767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:27.002838 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:27.644489 (+641651us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b2c0010db98. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228aac8a11 00007f228aad8372 00007f228c20aa01 00007f228c1cf43a 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2
0709 12:20:27.645013 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:20:28.216736 (+571723us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:20:28.217126 (+   390us) log.cc:844] Serialized 14 byte log entry
0709 12:20:28.220328 (+  3202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:28.220397 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:28.349451 (+129054us) raft_consensus.cc:1685] finished
0709 12:20:28.349540 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.349703 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":201,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":2731,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":7354752,"thread_start_us":571273,"threads_started":1,"wal-append.queue_time_us":635480}
I20260709 12:20:28.372893  7329 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.368999  7437 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:28.360762  7372 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.390638  7329 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.370561  7442 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:20:28.400285 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.401219 32112 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:28.402873 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:28.384403  7437 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:28.388358  7346 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:28.423192  6982 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:20:28.423964 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.424687  7398 raft_consensus.cc:3055] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:28.425112  7398 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:28.425953  7398 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.426990  7398 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:28.437914  7372 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:28.411388  7437 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.413301  7329 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:28.459157 32115 raft_consensus.cc:2468] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 17.
I20260709 12:20:28.437422  7138 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:28.460214 32119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.029s	sys 0.035s
W20260709 12:20:28.463595 32119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.029s	sys 0.035s
I20260709 12:20:28.454514  7329 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.466305 31970 raft_consensus.cc:2410] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
W20260709 12:20:28.468123 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 105858us
I20260709 12:20:28.474812 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.465472  7441 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.476629 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.476927 32040 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:28.470265  7329 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:28.479615  7441 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.486070 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.487691  7237 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.488314  7237 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.485229  7363 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.490885  7363 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.492129  7441 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:28.493461  7363 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.494586  7372 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.495332 32114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47766 (request call id 3224) took 1507 ms. Trace:
I20260709 12:20:28.495132  7372 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.495704 32114 rpcz_store.cc:276] 0709 12:20:26.987985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.989268 (+  1283us) service_pool.cc:224] Handling call
0709 12:20:27.354142 (+364874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:27.482296 (+128154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:28.495139 (+1012843us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400138b50. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1ce8ad 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:28.495273 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838376832}
W20260709 12:20:28.497071 31969 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58058 (request call id 1029) took 1571 ms (client timeout 1502 ms). Trace:
W20260709 12:20:28.497563 31969 rpcz_store.cc:269] 0709 12:20:26.925729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.925893 (+   164us) service_pool.cc:224] Handling call
0709 12:20:28.496907 (+1571014us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400414454. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d32fa 00007f2291194b17 00007f228a70f6d4 00007f228a70f662 00007f228a70f591 00007f228a70f50c 00007f228a70e7c2 00007f228a50a48c 00007f228a5098c6 00007f228a50926f
0709 12:20:28.497007 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077385984}
I20260709 12:20:28.497160  7372 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:28.499097 31971 raft_consensus.cc:1275] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:28.504089  7138 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.506748  7377 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:28.507472  7377 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.508028 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.508692 32119 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 26.
I20260709 12:20:28.509773 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:28.510588  7363 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:28.510970  7363 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.511296  7363 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:28.511163  7377 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:28.479650 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:28.516616 32084 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.001s	sys 0.017s
W20260709 12:20:28.516978 32084 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.001s	sys 0.017s
W20260709 12:20:28.517263 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 837934us
I20260709 12:20:28.525499 32041 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:28.528106 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:28.532138 32651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47558 (request call id 1198) took 1004 ms. Trace:
I20260709 12:20:28.532327 31969 raft_consensus.cc:2393] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 21: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:28.535375 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:28.536047 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 28.
I20260709 12:20:28.538115 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:28.538921 31969 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.532554 32651 rpcz_store.cc:276] 0709 12:20:27.528214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.528398 (+   184us) service_pool.cc:224] Handling call
0709 12:20:27.528767 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:28.192687 (+663920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:28.192716 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:28.193197 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:20:28.465262 (+272065us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:20:28.465317 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:28.465735 (+   418us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:28.531933 (+ 66198us) raft_consensus.cc:1685] finished
0709 12:20:28.531952 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.532081 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147732,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":8008,"thread_start_us":935184,"threads_started":2,"wal-append.queue_time_us":336212}
I20260709 12:20:28.543406 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.544130 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.544988 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.545897 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:28.546619 32045 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.547030 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.548976 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.544101 31972 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:20:28.526858  7356 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:28.554905 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.555900 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.558418 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.501196  7195 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:20:28.567339 32047 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:28.562338 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.569100  7280 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:20:28.577087  7280 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:28.559576  7372 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:28.569418  7356 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.607092  7363 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:28.614753 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.616114 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.616772 32119 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:28.618075 32045 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:28.621135 31969 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:28.618952  7363 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.631336  7356 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.636051  7338 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:28.639058 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:28.639992  7476 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:28.642691  7329 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.644979  7329 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.647466  7363 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.647540  7429 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.648093  7363 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.648342  7429 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.649850 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.650563  7363 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.581600  7437 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.652108 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.653735 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.656196 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.657785 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.661365 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:28.662283 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.663321 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.665396 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.665705 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.666179 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:28.666615 31972 raft_consensus.cc:2468] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:20:28.666873 31965 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 26.
I20260709 12:20:28.667824 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:28.667831 32119 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 27.
W20260709 12:20:28.671767 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 109512us
I20260709 12:20:28.671789 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:28.676913 32626 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:28.678596 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:28.681159 31938 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:28.681537 31938 leader_election.cc:403] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:28.661988  7398 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.673905  7476 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:28.685403 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:28.678717  7346 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:28.697875  7346 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:28.698256  7346 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.698653  7346 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:28.672760  7363 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:28.687419 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.702374  7341 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:28.702795 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.703651  7341 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:20:28.690178 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.704476 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:28.675990  7138 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:28.706875  7138 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:28.707782 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:28.707985 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.685531  6990 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:28.708561  7377 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:28.709017  7377 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.709420  7377 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:28.706903 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:28.696617  7437 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.694980  7429 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:28.713688  7341 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:28.714104  7341 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.714682  7398 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.716144  7341 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:28.717198  7398 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:28.682497  7195 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:28.718241  7195 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:28.719278  7195 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:28.722707 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.671898  7329 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.725167 31969 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:20:28.727319 32651 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:28.725942 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.727900 32117 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:28.737282 31938 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:28.728207 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.728967 32623 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.730428 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.733067 32626 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:28.742630 32657 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:28.750541 31936 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:28.746908  7429 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:28.776799 32119 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:20:28.747728  7366 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:28.767066  7429 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:28.826720 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.748549  7372 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.138s
I20260709 12:20:28.707785  7138 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.747493  7329 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.828924 32042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:28.771593  7278 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:28.818495  6579 consensus_queue.cc:1059] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.123s
W20260709 12:20:28.789417 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.768608  7321 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.942281 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.814639 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.813047 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:28.815671 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:28.875079 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:28.815949 32118 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:28.770226  7437 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:28.854365 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.895190 32653 raft_consensus.cc:2410] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:20:28.827965 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.815101 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:28.974730 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:28.990278 32046 raft_consensus.cc:2368] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:28.988999 32119 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:28.990756 32659 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:28.991348 32039 raft_consensus.cc:2368] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:28.990339 32112 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:28.828325 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:28.988476 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 236526us
I20260709 12:20:28.993743 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:28.994529 32047 raft_consensus.cc:2368] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28.
I20260709 12:20:29.001468 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:29.002631  6579 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:29.003130 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53636us
I20260709 12:20:29.003162  6579 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.876269  7396 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:29.011267 32047 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:29.010272 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:28.834537  7279 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:29.023078 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 207488us
I20260709 12:20:28.807937  7237 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.019392 32043 raft_consensus.cc:1240] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:20:28.781430  7441 raft_consensus.cc:2804] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:28.846514  7274 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:28.886118  6982 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:28.867329  7313 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.033053 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:28.822770  7374 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.023830 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:28.968863  7278 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.019702  7396 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:28.873554  7366 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.853329 32044 raft_consensus.cc:1240] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:29.000622  7321 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.915225  7280 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:28.901566  7377 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.048491  6579 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.049991 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.029521  7487 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:29.034482  7441 raft_consensus.cc:697] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:29.048588  7278 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:29.050542  6973 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:29.050266  7362 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:29.075014 32118 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:29.027163  7279 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.919010  7429 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:28.774611  7363 consensus_queue.cc:1048] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.296s
I20260709 12:20:29.034855  7274 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.060449  7487 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:29.035684  6982 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.091965 32042 raft_consensus.cc:606] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.050961  7374 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.073256  6973 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.046746  7313 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.112171 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.062602  7344 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:20:29.079222  7362 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.063833  7488 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:29.105532 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.063155  7280 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.062772  7346 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.097462  6579 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.125029  6579 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.122432 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.093299  7438 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.119992  7441 consensus_queue.cc:237] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:29.137271  7484 log.cc:927] Time spent T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:20:29.138978  7483 log.cc:927] Time spent T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:20:29.141134  7493 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.141719  7493 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.143944  7493 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.147886  7274 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.148546 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.149220 31966 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:29.154266  7195 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.154677 32659 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:29.154929  7195 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.155759  7346 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:29.157173  7195 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.158157  7377 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:29.159142  7377 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.159622  7377 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.155313  7321 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.098690  7487 raft_consensus.cc:3060] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:29.163712 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.164409 32116 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 20.
I20260709 12:20:29.166131 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:29.113446  6973 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:29.171983 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.187865 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:29.116066  7488 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:29.119463  7362 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:29.189311 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:29.201455 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:29.203097 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.167588  7195 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:29.218530  7195 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.219015  7195 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:29.224079 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:29.224994 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.225687  7329 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.226244 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.227375 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.225041 32654 raft_consensus.cc:2393] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:29.228497 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.230540  7377 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:29.231871  7490 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:29.232407  7490 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.238657  7313 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.235735  7362 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.205685 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.247233 32040 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:29.247570 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.168632  7398 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.092031  7396 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.211437  6982 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.201092  7379 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.012s
I20260709 12:20:29.188777  7438 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:29.214000 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.224120 32114 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:29.253762 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:29.251101 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:29.247426 31966 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:29.255739 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:29.211117  6579 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.256919 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:29.258558 32112 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:20:29.262193 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:29.266750 32117 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:29.268131 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:29.294782 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.295049 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:29.297689 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 44024us
I20260709 12:20:29.227602  7280 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.209960  7374 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.306181 31938 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:29.307891 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.313184 31967 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:29.308864 32040 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:29.308175 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:20:29.296387  7341 consensus_meta.cc:287] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 12:20:29.320971 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:29.322294 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 328766us
I20260709 12:20:29.321674 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:29.211748  7488 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.351025 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:29.307124  7466 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:29.307389  7437 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:29.274605  7398 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.350822 32084 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:29.356240 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.290743  7476 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:20:29.315934  7329 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.312072  7213 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.360296 31972 raft_consensus.cc:1275] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:29.359722 31963 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:20:29.320190  7490 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:29.304826  7379 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:20:29.372296 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:29.357378  7457 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:29.378971 31972 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:20:29.349359  6983 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:20:29.360611  7362 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:29.397716 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.385749 31972 pending_rounds.cc:85] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 12
I20260709 12:20:29.333875  7341 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.363452  7321 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.382874  7488 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:29.375041  7366 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.404770 31972 pending_rounds.cc:107] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:20:29.400859 31935 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.452033 31972 raft_consensus.cc:2889] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:29.452536 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51802.9us
I20260709 12:20:29.453193 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.379060  7493 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:29.368956  7346 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:20:29.378899  7502 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.401718  7457 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:29.405589 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:29.456380 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:29.392433  7377 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:29.459467 32119 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:29.376498  7213 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.373111  7329 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.453289 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.402144  7279 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.396018  7466 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:29.473606 32656 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:29.478155  7501 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:29.481979  7496 log.cc:927] Time spent T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:20:29.482517  7438 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.485074  7341 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:29.486580  7274 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:29.487248  7274 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.491005  7283 consensus_queue.cc:1059] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:20:29.492362  7283 raft_consensus.cc:3055] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:29.492776  7283 raft_consensus.cc:740] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:29.496625  7457 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.489959  7274 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.474866  7490 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.476413 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.393013  7278 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:29.474668 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 69165.8us
I20260709 12:20:29.511221  7278 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.511579 32084 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:29.512017 31971 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 24: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:29.512712 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:29.513682 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:29.514482  7490 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.515192  7490 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:29.515559  7490 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.515913  7490 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:29.519593  7213 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.520294 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:29.523720 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:29.525007  7329 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.507658 32656 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:29.527354  7329 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:29.528221  7398 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.528544  7341 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:29.467023  7377 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:29.529846  7377 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:29.505081  7195 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.536437  7195 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.493759  7283 consensus_queue.cc:260] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.537770  7283 raft_consensus.cc:3060] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:29.540947  7487 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.546437 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:29.547492  7492 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:29.548110  7492 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:29.549209  7492 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.553033  7336 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:29.553547  7336 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:29.553889 32084 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:29.564214 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.566337 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.566942 32117 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:29.565794 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:29.570246 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:29.604210 32087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.672031 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:29.673036 32120 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:29.675251 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 121456us
I20260709 12:20:29.676347 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:29.677516 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 201169us
I20260709 12:20:29.678514 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.679141 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:29.679209 32119 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:20:29.678525 31935 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.683770 31963 raft_consensus.cc:1240] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:29.688055 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 170232us
I20260709 12:20:29.690620 32119 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:29.546494  6982 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:29.724501 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:29.725778 32084 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:29.581384  7379 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:29.463636  7493 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.675036  7451 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.696375  7278 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:29.740865 32120 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:29.745164 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:29.752605 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 74197.8us
I20260709 12:20:29.753540  7280 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:29.754367 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:29.701367  7372 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:29.700361  7329 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:29.756932 32626 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:20:29.752503 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 59272.7us
I20260709 12:20:29.754793 31935 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.704587  6982 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:29.751849 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:29.593288  7466 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.739252  7494 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.767277 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:29.748704  7379 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.769524  7512 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:29.771181 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.674477  7336 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.739564  7469 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:29.771688  7466 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.783262 32045 raft_consensus.cc:2468] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:20:29.768267 32114 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:29.771697  6982 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:29.755079  7374 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.791064  7278 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:29.821738  7441 raft_consensus.cc:2804] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:20:29.877249 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 120828us
I20260709 12:20:29.831614  7336 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:29.756541  7451 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.792793  7329 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.819585  7469 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:29.902637  7487 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:29.907166  7441 raft_consensus.cc:697] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:20:29.939256  7517 log.cc:927] Time spent T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:20:29.942250  7493 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:29.935328  7518 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:20:29.933343  7490 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.945695  7490 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:29.981371 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:29.985949 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:29.988502 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 221182us
I20260709 12:20:29.997931 32084 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:29.999854 32040 raft_consensus.cc:3060] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:30.003108  7379 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.002907 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:30.005302  7441 consensus_queue.cc:237] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.007480  7138 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:30.008858  7138 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.010087  6990 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:30.011390  6990 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.014662  7451 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.016909  7329 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.020982  7469 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:29.880714  7278 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.024212  7490 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:29.914554  6973 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.003s
I20260709 12:20:30.030692  6990 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.009644  7138 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:29.990085 31935 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:30.075131 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:30.077705 32654 raft_consensus.cc:1240] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:30.080821 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:30.086638 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 89097.6us
I20260709 12:20:30.088032  7457 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:30.089146  7457 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.069113 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:30.030136  6973 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.090590  7480 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:30.091547  7480 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.093077  6973 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.091847 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.102497  7278 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.105690  7138 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.108968 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:30.110128 32656 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:30.113425  7529 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:30.114456  7529 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.117110  7529 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:30.117420 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.119416  7480 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.117594  7463 consensus_queue.cc:1059] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.004s
W20260709 12:20:30.156445 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 166649us
I20260709 12:20:30.157548 31935 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:30.158866  7532 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:30.159633  7532 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:30.161307  7532 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.143144  7512 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:30.164319  7510 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:30.167855  7457 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.176363 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:30.177454 32653 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
W20260709 12:20:30.177656 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 86043.3us
I20260709 12:20:30.181507  7396 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:30.193763 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:30.184341 31935 leader_election.cc:400] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:30.189028 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:30.189229 32115 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:30.191984 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:30.192224 32045 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:20:30.184373  7400 raft_consensus.cc:3055] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:30.196293 32040 raft_consensus.cc:2468] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:20:30.196054  7400 raft_consensus.cc:740] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:30.202138 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.205138 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.206207 32114 raft_consensus.cc:2393] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 17: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:30.199496  7341 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:30.203157  7400 consensus_queue.cc:260] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.211092  7400 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:30.218837 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:30.220803 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 469276us
I20260709 12:20:30.227342 32114 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:30.208824 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:30.231001 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.251573 32116 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:30.253392 32656 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:30.259879 32011 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:30.261114  7336 raft_consensus.cc:2804] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:30.261693  7336 raft_consensus.cc:697] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:30.263056  7336 consensus_queue.cc:237] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.227674 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:30.231977 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48101.1us
I20260709 12:20:30.242730 31938 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:30.231629 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:30.263288  6982 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.274758 32116 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:20:30.284062 32115 raft_consensus.cc:2393] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 28: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
W20260709 12:20:30.287200 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.287403 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:30.285660 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58176us
I20260709 12:20:30.291029  7321 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:30.289857  7379 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:30.291898 31935 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:30.284739  7437 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:20:30.293900  7321 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.304286  7321 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:30.304206 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:30.307502 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:30.299294  7379 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:30.320905  7438 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:30.321868 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.322798 32045 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:30.331075  6982 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.314363  7379 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:30.337757 32040 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:30.343422 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:30.354990  7377 consensus_meta.cc:287] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.821s	user 0.011s	sys 0.011s
I20260709 12:20:30.355804  7377 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.357231 31938 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:30.358943  7377 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:30.338779  7321 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.362573  7438 raft_consensus.cc:3060] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:30.364954 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:20:30.367779 32010 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.622s (SENT) [suppressed 7 similar messages]
W20260709 12:20:30.377177 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103184us
I20260709 12:20:30.379803 31935 leader_election.cc:400] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28.
I20260709 12:20:30.387977 31935 leader_election.cc:400] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:30.392331 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:30.368363  6982 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:20:30.395473 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.418161 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:30.418490  7501 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:20:30.421774 31964 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:30.422082 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:30.429878  7237 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.430675 31965 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:30.430624  7237 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.433784  7237 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.433894  7438 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:30.429898 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:30.417146  7512 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 12:20:30.437920 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 154733us
W20260709 12:20:30.442759 31937 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.454512 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:30.455688 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:30.457911  7490 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:30.417146  6982 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:30.470278 32045 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:30.470431 32623 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:30.471067 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:30.471112 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:30.483999  6982 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:30.494733 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:30.494426 31867 catalog_manager.cc:5697] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 21, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 21 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:30.495422 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:30.496658 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.496591 31867 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:30.497344  7367 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:30.497906  7367 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.498342  7367 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:30.498606  7321 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.498123 31867 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:30.502140 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 159863us
I20260709 12:20:30.512800 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:30.491557  6982 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:30.499797 31867 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 24 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:30.517949 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:30.527004 31867 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:20:30.528180 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 101878us
I20260709 12:20:30.546423  7377 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:30.547067  7377 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:30.547986  7377 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.549338  7341 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:30.549999  7341 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.550530 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:30.552368  7341 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:30.552459 31867 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 21 to 24, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.557602  7400 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:30.559860  7336 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.560187  7541 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:30.561326  7541 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:30.559871 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:20:30.574769 31869 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 245) took 84582 ms (client timeout 15000 ms). Trace:
W20260709 12:20:30.597317 31869 rpcz_store.cc:269] 0709 12:19:05.992063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.993224 (+  1161us) service_pool.cc:224] Handling call
0709 12:19:06.506260 (+513036us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890ab0b2 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:19:06.887696 (+381436us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aae9c 00007f22890ab0f6 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:20:30.574519 (+83686823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255380,"reported_tablets":51,"spinlock_wait_cycles":133282944}
W20260709 12:20:30.575539  7547 log.cc:927] Time spent T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:20:30.604203  7313 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:30.604913  7313 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.610121 32043 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:20:30.582469 31870 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 152) took 74336 ms (client timeout 15000 ms). Trace:
I20260709 12:20:30.594533 31971 raft_consensus.cc:1275] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:20:30.565922  7437 raft_consensus.cc:2804] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:20:30.603611 32011 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:30.613067 31870 rpcz_store.cc:269] 0709 12:19:16.204457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.204699 (+   242us) service_pool.cc:224] Handling call
0709 12:20:30.531069 (+74326370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":899079,"reported_tablets":27}
W20260709 12:20:30.618831 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 49252.5us
W20260709 12:20:30.622457 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 60743.4us
I20260709 12:20:30.579370  7396 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:30.562417  7400 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.627211  7396 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.627856  7372 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:30.628422  7372 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.629113  7400 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.571214  7541 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:30.631062 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71364.4us
I20260709 12:20:30.636991  7362 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:30.637653  7531 consensus_queue.cc:1048] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:30.631716 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:30.630560  7541 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.639519  7437 raft_consensus.cc:697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:30.640442  7541 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:30.640733  7437 consensus_queue.cc:237] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.645426  7545 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:30.637692  7362 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.631126  6579 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:30.585357  7138 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:30.669833 31964 raft_consensus.cc:2468] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 28.
I20260709 12:20:30.676031  7138 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:30.684289  7138 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.630254  7396 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.658696  7400 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:30.661015  7438 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.630983  7372 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:30.692488 31867 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 24, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.696848 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:30.703701 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 44722us
I20260709 12:20:30.634434  7379 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.704644 32623 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:30.709434 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.701347 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:30.689735  7531 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:30.694856  7400 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.722718  7429 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.723271  7429 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.725445  7429 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.725565  6982 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.731289  6982 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:30.726985  7313 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.732493  7363 raft_consensus.cc:2804] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:30.733044  7363 raft_consensus.cc:697] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:30.734046  7363 consensus_queue.cc:237] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.738535 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:30.739372 32656 raft_consensus.cc:2393] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 23: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:30.741485 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:30.742429  7379 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.742909 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.744360 32084 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.745680  7492 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.746978  7341 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:30.751138  7400 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:30.712996  7372 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:20:30.750026  7531 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:30.790953 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 59029.2us
I20260709 12:20:30.721884  7438 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.798734 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.798966 32084 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.801513 31938 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:30.796864 31867 catalog_manager.cc:5697] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 19 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:30.807489 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 59421.5us
I20260709 12:20:30.809770 32121 raft_consensus.cc:1275] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:20:30.778807  7367 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.819283 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.827250 32115 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:30.829020 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.830147 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.832369 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.833189 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.835203 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:30.838243 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59310.3us
I20260709 12:20:30.839138 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.840078  7490 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:30.839737 31867 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 21, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 21 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.806854  7545 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.807729  7532 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:30.844575 32120 raft_consensus.cc:2368] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21.
I20260709 12:20:30.849977 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.815801  7138 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:20:30.857605 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.866590 32112 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:30.873847  7438 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.876406  7400 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.876969 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.877604 32040 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:30.878619  7569 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:30.880246  7531 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.882124  7367 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:30.881446 31867 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:30.883823  7400 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.886701  7545 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:30.890770 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49900.7us
I20260709 12:20:30.891978  7367 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:30.892472 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:30.892824  7367 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:30.893334  7362 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:30.893806 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:30.899120 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 97721.6us
I20260709 12:20:30.893898  7362 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.895440 32045 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:20:30.883404 31867 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.917286 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:30.923651 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46803.2us
I20260709 12:20:30.920487 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.919299  7429 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:30.894765  7564 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:30.938957  7341 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:30.945569 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:30.945919  7429 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.947890  7278 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.943495 31867 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 25, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 25 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:30.948660  7429 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.948539  7278 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.960724  7278 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.965600 31969 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:20:30.967249 32121 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:30.967891 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.968413  7313 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:30.971235  7313 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.971928  7379 raft_consensus.cc:993] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:30.972608  7379 raft_consensus.cc:1081] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:30.969189 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.978366  7313 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:30.981441  7437 consensus_queue.cc:1048] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:30.920358  7362 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:30.984553  7362 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:30.982427 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.985473 32120 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:30.986512 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:30.986931 31867 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:30.988358 32120 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:30.894345  7367 consensus_queue.cc:1059] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 12:20:30.950937  7510 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:30.998733  7510 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:30.968267 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.002677  7510 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:31.003000 32121 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:30.940416  7434 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.004078 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.004954 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.005656 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.007403 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.008361 32113 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28. Ops: [27.14-27.14]
I20260709 12:20:31.008464 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.008738 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.009327 32041 raft_consensus.cc:2393] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate ec7f26ea46944ea498ad253b1b1819b0 in this term.
I20260709 12:20:30.997296 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.997893 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:30.989344 31867 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 22, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.006242 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.011842 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:31.006268 32121 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
W20260709 12:20:31.016031 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 59721.6us
I20260709 12:20:31.018494 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.020306 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:31.019464 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.024173 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.023484 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.025128  6579 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:31.025527  6579 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.025840  6579 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:31.020175 31867 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.019464 32121 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:30.943254  7545 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:31.024580 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.031621 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.033725 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:31.023962  7434 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:31.034719  7434 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:31.035732  7434 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.020260  7438 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:31.038772  7438 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.040921 32116 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:31.041702  7438 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:31.047655 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.027742  7429 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:31.048518  7429 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.048944  7429 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:31.050590 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:31.051905 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.052201  7490 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:31.052676  7490 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:31.058454  7563 log.cc:927] Time spent T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.335s	user 0.002s	sys 0.000s
W20260709 12:20:31.058738 32013 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.061033 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.061733 31964 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 26.
I20260709 12:20:31.062729 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.063671  7463 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:20:31.064234  7574 log.cc:927] Time spent T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:20:31.064358  7463 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.066959  7569 consensus_queue.cc:1048] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:31.068843  7429 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.070458  6579 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:31.071130  7553 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.320s	user 0.003s	sys 0.000s
I20260709 12:20:31.072602  6579 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:31.078965 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 62758.2us
I20260709 12:20:31.080646  7568 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:31.080961  7331 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:31.081506  7331 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.081907  7331 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.082387  7379 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:31.082787  7379 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:31.082918 32039 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:31.064363  7442 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:31.084606  7442 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.084995  7442 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:31.085302  7455 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:31.086551 32010 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.091095  7331 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.083724  7379 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.104264 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.106428 31867 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 30, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 30 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:31.107003  7237 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.111841 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.112809  7331 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:31.113313 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.113646 32040 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:31.114742 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.116227 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.118777 31969 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 23 index: 20. Preceding OpId from leader: term: 24 index: 21. (index mismatch)
I20260709 12:20:31.119011 32045 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:31.094964  7336 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.123410 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.124581 31964 raft_consensus.cc:2393] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:31.127324 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.127825 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.134974 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.135661 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.129735 32117 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:31.139154 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.140986 31965 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 26: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:31.141489 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:31.142110 32115 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:31.142683 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:31.146638 32120 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.145933 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.148856 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.149829 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.149999 31965 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 14. (term mismatch)
I20260709 12:20:31.150398 31965 pending_rounds.cc:85] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 13
I20260709 12:20:31.151497 31966 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:31.151680  7429 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:31.152227  7461 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.011s
I20260709 12:20:31.148499  7336 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.155332 32043 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:31.159054 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.159796 31967 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:31.158479 31867 catalog_manager.cc:5697] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.164314  7442 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.164638 31867 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 22 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:31.166719 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.167502 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
W20260709 12:20:31.171072 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84650.4us
I20260709 12:20:31.177548 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.178186 31967 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:31.178215 31966 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:20:31.182688  7367 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.183207  7367 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.185101 32045 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:31.188985  7367 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.190200  7463 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:31.190982  7336 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:31.191365 31867 catalog_manager.cc:5697] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 12 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.193172  7461 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:31.196430 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:31.195884 31867 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 17 to 27, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.197027 32112 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:31.197561 31867 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.150673 31965 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 27.14
I20260709 12:20:31.199184 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.200241 31966 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 28.
I20260709 12:20:31.200815 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:31.201463  7572 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.242s	user 0.002s	sys 0.000s
I20260709 12:20:31.202728 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.204054 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.210687 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.213135 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:31.214538 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 136566us
I20260709 12:20:31.279640 32112 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:31.222664  7564 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:31.249063 32115 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:31.224323 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:31.225507 31965 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:31.285522 32119 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 22: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:31.236459 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.225462 32120 raft_consensus.cc:2468] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:20:31.284269 31938 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.283289 31967 raft_consensus.cc:2468] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:20:31.225947 31971 raft_consensus.cc:3060] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:31.294617 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.295930 32013 leader_election.cc:400] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21.
I20260709 12:20:31.287664 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.228175 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53855.6us
I20260709 12:20:31.298781 32013 leader_election.cc:403] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:31.236496 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.225977 32623 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.216321 32043 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
W20260709 12:20:31.335572 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51393.2us
I20260709 12:20:31.341307 31938 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.342446  6579 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:31.342890  6579 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:31.340127 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.340909 31972 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
W20260709 12:20:31.341661 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 115784us
I20260709 12:20:31.282366  7442 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.354240 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.325567 31867 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.355127 31966 raft_consensus.cc:2393] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 25: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:31.349401 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.261016  7372 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:31.355647 31867 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 24. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:31.359118 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68241us
I20260709 12:20:31.351308  6579 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.363624 32013 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:31.370814 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.343719  7434 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:31.374064 32041 raft_consensus.cc:2393] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 22: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:31.374589 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.378948 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.376634 31867 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 21 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.380940 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:31.267859  7492 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:31.352811  7545 raft_consensus.cc:2804] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:31.383279 32120 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:20:31.358697  7336 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21.
I20260709 12:20:31.387472 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:31.389015 31867 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 21 to 22, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:31.395026 31867 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:20:31.400159 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47566.5us
I20260709 12:20:31.402906 31966 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.412783 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.416458 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.389331  7434 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:31.413283 31867 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 20, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 20 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:31.413569 31970 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:20:31.398998  7237 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:31.399359  7545 raft_consensus.cc:697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:31.425901 31867 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.430361 32010 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.393442  7372 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.283406  7564 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.442497  7545 consensus_queue.cc:237] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.443797  7372 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.444356  7564 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.444800  7434 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.431777  7400 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:31.436317  7575 log.cc:927] Time spent T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.273s	user 0.011s	sys 0.000s
I20260709 12:20:31.438506  7578 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:31.431356 32113 raft_consensus.cc:1240] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22. Ops: []
W20260709 12:20:31.454828 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62516.9us
I20260709 12:20:31.439960  7138 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.408340  7492 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.451994 31867 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 20, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 20 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.451007  7336 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:31.467463 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.452966  7578 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.503468 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.513525 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.514614 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 47243.2us
I20260709 12:20:31.514860 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:31.494529 32112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:31.516637 32112 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:31.517024 32112 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:31.518538  7379 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:31.519146 31867 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 23, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:31.501292  7336 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:31.523782 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.491832  7578 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:31.543609 32657 raft_consensus.cc:1240] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:31.545526  7455 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:20:31.545867 31966 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:31.546345 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.544384 32112 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:31.602205 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 226726us
I20260709 12:20:31.622083 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:31.622141 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.627188 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 103483us
W20260709 12:20:31.628875 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 114431us
I20260709 12:20:31.546633 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:31.629963 32626 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.631412 32112 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.633602 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:31.639245 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.528985  7591 consensus_queue.cc:1059] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 12:20:31.641824 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:31.629029  7589 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.645726 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.635843  7585 consensus_queue.cc:1059] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:20:31.640019  7362 raft_consensus.cc:2804] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:31.639878  7331 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:31.656818 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.658469 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.659766 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.654400 32042 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:31.646243  7532 raft_consensus.cc:2764] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:20:31.634455  7564 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:20:31.679014 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.684839 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.671873  7396 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:31.687058 32042 raft_consensus.cc:2393] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:31.662863  7331 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:31.688961 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.689522 32047 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 27
W20260709 12:20:31.686457 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 50748.9us
I20260709 12:20:31.678232  7362 raft_consensus.cc:697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:20:31.690166  7331 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:31.696933 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.698210  7597 log.cc:927] Time spent T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:20:31.698437  7586 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:31.698917  7586 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:31.698865  7362 consensus_queue.cc:237] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:31.699869  7580 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.221s	user 0.001s	sys 0.001s
I20260709 12:20:31.700977 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.698069 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.701526 32038 raft_consensus.cc:3060] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:31.701673 32042 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:31.702564 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.705363  7581 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 12:20:31.699534  7586 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.708256 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.652190  7589 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.712472  7589 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:31.713975 32042 raft_consensus.cc:1240] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 25. Current term is 26. Ops: []
W20260709 12:20:31.709893 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87891.5us
I20260709 12:20:31.718986 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.719846 31935 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:31.722736 32039 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:31.724890 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.725490 31966 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:31.729092 32117 raft_consensus.cc:1275] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:20:31.730489  7579 log.cc:927] Time spent T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 12:20:31.728241 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:31.730271  7331 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.732331 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:31.690872  7532 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:31.733011  7512 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:31.733028  7341 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:31.734841 32038 raft_consensus.cc:2468] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 28.
I20260709 12:20:31.737275 32047 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:31.738407  7396 consensus_queue.cc:1059] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:31.754828 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.755420 31964 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:31.746491 31963 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:31.744134 32042 raft_consensus.cc:1275] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 23 index: 20. Preceding OpId from leader: term: 24 index: 21. (index mismatch)
I20260709 12:20:31.753010  7532 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.745921 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.762202 31966 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:31.769425 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:31.770988  7600 consensus_queue.cc:1048] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:20:31.774482  7564 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:31.774859 32623 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:31.777552 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45340.1us
I20260709 12:20:31.778396 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:31.780826  7490 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.782662  7396 raft_consensus.cc:3055] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:31.783252  7396 raft_consensus.cc:740] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:31.783526  7510 consensus_queue.cc:1059] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:31.784451  7510 raft_consensus.cc:3055] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:31.784266  7396 consensus_queue.cc:260] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.784788  7510 raft_consensus.cc:740] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:31.785465  7396 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:31.787392 31935 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:31.788686  7429 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:31.785566  7510 consensus_queue.cc:260] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:31.782590  7598 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:31.793697  7577 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.394s	user 0.001s	sys 0.001s
I20260709 12:20:31.794507 32656 raft_consensus.cc:1240] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:31.794952  7331 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.758440  7321 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:31.733057  7400 raft_consensus.cc:3055] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:31.796259  7492 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.757189  7532 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:31.803567 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:31.803567 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:31.823995 31938 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.799714  7510 raft_consensus.cc:3060] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.887359 32041 raft_consensus.cc:1240] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:31.829216 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:31.815864  7595 raft_consensus.cc:3055] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:31.833618  7593 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:31.819023  7450 consensus_queue.cc:1048] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.016s
I20260709 12:20:31.913120 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:31.813201  7400 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:31.927407 32118 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:31.927994  7400 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.929032  7400 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:31.908692  7595 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:31.847631 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.848193 32042 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:31.848905 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:31.854043 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:31.854751 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 93970.2us
I20260709 12:20:31.932739 32046 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:31.856315  7586 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:31.936066 32653 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:31.937543  7586 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:31.857930 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:31.858517 32044 raft_consensus.cc:1240] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
W20260709 12:20:31.859432 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 48016.7us
I20260709 12:20:31.938920 32113 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:31.865720 31971 consensus_meta.cc:287] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.003s
I20260709 12:20:31.933125 32046 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:31.939409 31971 raft_consensus.cc:2468] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:31.868824 32114 raft_consensus.cc:1240] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:31.871320  7490 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:31.874219 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.876761 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:31.837339  7601 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:31.930717 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 143409us
I20260709 12:20:31.944657 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.951522 31935 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:31.955749 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:31.957291 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 80654.4us
I20260709 12:20:31.940037  7434 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:31.947551  7463 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:20:31.966895 32046 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:31.975961 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 156365us
I20260709 12:20:31.979898 31965 raft_consensus.cc:1275] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:31.978811 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:31.982781 32046 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:31.984558 32084 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:31.961256  7586 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:31.960986  7237 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:31.993866 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:32.005365 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:32.005971 31970 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:32.006913 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.979732  7341 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:31.973680  7578 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:32.013410 32086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.013871  7434 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:31.995008  7138 consensus_queue.cc:1059] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:20:31.995277  7372 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:32.000483  7586 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:32.020666  7341 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:31.951017  7601 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:32.037366 32045 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:31.990535  7595 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.040262  7595 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:32.041674  7578 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:32.018951  7532 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.043231 32119 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 28. Current term is 29. Ops: [28.14-28.14]
I20260709 12:20:32.052704 31970 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:32.035526  7586 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:32.055032 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58133.6us
W20260709 12:20:32.061136 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 109721us
I20260709 12:20:32.036818  7492 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:32.036618  7341 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:32.038718  7372 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:32.062641 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:32.066648 32046 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:32.082520 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:32.087532  7605 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:32.090453 31935 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.110726  7372 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.100489  7593 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:32.115135  7532 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:32.116313  7601 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:32.119326 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:32.123829  7594 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:32.127664 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 127855us
I20260709 12:20:32.130638 31935 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.136304 31935 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.139766  7593 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:32.141448 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.142601  7532 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:32.143909  7532 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.147562  7620 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:20:32.152112 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:32.168895 32120 raft_consensus.cc:2393] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 26: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:32.163928 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:32.207253 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 63609.9us
I20260709 12:20:32.168569  7510 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:20:32.212373 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 48583.1us
I20260709 12:20:32.218482  6579 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:32.219594 31935 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:32.220441  6579 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.222468 31935 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:32.218956 31857 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b83e7300b3fe4d21a97d941dee98fa81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 38c927f0ee104b57972899f5b27aa99b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:32.224098 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.225296 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.225373 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
W20260709 12:20:32.282410 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 47506.2us
I20260709 12:20:32.282899 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:32.283560 31970 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:32.170600  7568 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 12:20:32.318034 31857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 106708us
I20260709 12:20:32.319605 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:32.318157 32010 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Timed out: RequestConsensusVote RPC to 127.30.25.1:40903 timed out after 1.622s (SENT)
I20260709 12:20:32.321199 32010 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.318681  6579 consensus_queue.cc:1059] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:20:32.228199  7594 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:32.328356  7429 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:32.328836  7429 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:32.329208  7429 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:32.329509  7512 raft_consensus.cc:3055] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:32.329946  7512 raft_consensus.cc:740] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.290158  7593 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:32.331053  7593 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:32.331585  7593 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:32.330993  7512 consensus_queue.cc:260] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:32.335786  7568 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:32.335846  7512 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:32.336388  7568 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:32.337102  7614 log.cc:927] Time spent T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:20:32.337344  7568 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 28.14, Last appended by leader: 14, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:32.312223  7396 consensus_meta.cc:287] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: Time spent flushing consensus metadata: real 0.526s	user 0.017s	sys 0.001s
I20260709 12:20:32.282946  7619 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:32.344779  7601 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:32.330897  6579 raft_consensus.cc:3055] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:32.348029  6579 raft_consensus.cc:740] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.282241  7510 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.358259 31935 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.318066  7590 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:32.396610 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.396556 32010 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.400306  6579 consensus_queue.cc:260] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.402909 32045 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:32.403654 32010 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.406515  6579 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:32.420210 32045 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:20:32.402963 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 51206.2us
I20260709 12:20:32.430452  7590 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.432771  7596 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:32.433496 32044 raft_consensus.cc:1240] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:32.435954  7592 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:32.436956  7592 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:32.397073  7586 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.447963  7586 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:32.448938  7510 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.433391  7590 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:32.455026 32120 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:32.402788  7568 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:20:32.471370 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67779.5us
I20260709 12:20:32.477156 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:32.480580 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:32.481838 31972 raft_consensus.cc:2410] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:32.483004 31970 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 29.
W20260709 12:20:32.492265 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 140505us
I20260709 12:20:32.494659 31967 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:32.494789 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.475868  7590 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:32.496735 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65690.4us
I20260709 12:20:32.500167 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:32.505761 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.483424  7596 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.490656  7377 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:32.511086 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:32.483320  7591 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:20:32.520459 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:32.511696  7532 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:32.528062 31967 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:32.530403 31972 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:20:32.530720 31972 pending_rounds.cc:85] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting all ops after (but not including) 15
I20260709 12:20:32.530877 31972 pending_rounds.cc:107] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:20:32.531011 31972 raft_consensus.cc:2889] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:32.532758 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
W20260709 12:20:32.538533 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64904.2us
I20260709 12:20:32.509649  7590 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.543241 32010 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.541973 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:32.541509  7321 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:20:32.528787  7600 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:20:32.550019  7400 consensus_meta.cc:287] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.000s
W20260709 12:20:32.545789  7582 log.cc:927] Time spent T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:20:32.527552  7377 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:32.574147 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.535859  7532 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:20:32.593216 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64723.6us
W20260709 12:20:32.603962 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64686.5us
I20260709 12:20:32.605878  7617 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:32.606359  7617 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:20:32.606848  7624 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:20:32.607321  7617 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.608660  7617 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:32.606292  7643 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:20:32.594008  7619 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:32.614923  7619 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.566700  7438 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:20:32.570328  7501 consensus_queue.cc:1059] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:20:32.621452  7532 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.622653  7501 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:32.623293  7501 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.624440  7501 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.581943  7591 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:32.594142  7621 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.628968  7621 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.630450 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:32.631727  7621 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:32.635957 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 114552us
I20260709 12:20:32.636212 32010 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.656201 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.637512 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:32.687019 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:32.693115 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:32.679540 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:32.735723 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.736315 31938 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.736361 32118 raft_consensus.cc:1275] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:32.744395  7570 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.745079  7570 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:32.747737  7570 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:32.781267 32113 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:32.675117  7501 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:32.628532  7429 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.787969 32112 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:32.681172  7590 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:32.683216  7619 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:32.750492  7568 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.796690 31938 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.772908  7438 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:32.810767 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 154642us
I20260709 12:20:32.804081  7434 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.812706  7396 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:20:32.642766  7377 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.812806  7594 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:20:32.793303  7593 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.831370 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:32.833170 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.602408  7596 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.840152  7620 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:32.808792  7512 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.810464  7570 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:32.820948  7341 consensus_meta.cc:287] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.001s
I20260709 12:20:32.813807  7643 consensus_queue.cc:1048] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:20:32.862752  7341 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.866957  7341 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:32.868109  7429 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:32.869161 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:32.871760  7570 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:32.887274 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 75797.6us
I20260709 12:20:32.896098  7620 consensus_queue.cc:1048] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:20:32.897941  7593 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:32.898280  7570 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:32.907070 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.908932  7568 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:32.895932 32038 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
W20260709 12:20:32.910470 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 113115us
I20260709 12:20:32.912778 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:32.913656 32114 raft_consensus.cc:2393] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:32.915860 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:32.916730 32117 raft_consensus.cc:2393] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 26: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:32.916947 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:32.923970 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:32.910887 32044 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:32.933853 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:32.935891  7274 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:32.936966 31935 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.940771 31935 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:32.941075  7596 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:32.941715  7596 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.942057  7501 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:32.942624  7501 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:32.947881 32044 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:32.948922 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.951910 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.953403 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.956070 32112 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:32.963421 31966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:32.964270 31966 raft_consensus.cc:2376] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27. Re-sending same reply.
I20260709 12:20:32.974721 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.975967 31967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.976902 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:32.975381 31968 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:20:32.966490  7455 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:32.977744  7434 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:20:32.989509 32044 raft_consensus.cc:1275] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:20:32.982909  7278 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.980065  7570 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.990957  7278 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.984129  7591 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.994177  7593 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:20:33.002784  7570 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:32.943804  7501 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.040681 32117 raft_consensus.cc:1240] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:33.061434  7541 raft_consensus.cc:3055] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:33.094247  7633 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.125310  7237 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:33.163113  7577 log.cc:927] Time spent T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.359s	user 0.005s	sys 0.000s
I20260709 12:20:33.087194 31968 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:33.096150 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.099807 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:33.086853 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.092411 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.017757 32010 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:33.657884 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 640185us
I20260709 12:20:33.659324 32010 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:33.139326 32046 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:33.097700 32084 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:33.305564  7660 log.cc:927] Time spent T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.304s	user 0.000s	sys 0.000s
I20260709 12:20:33.312242 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:33.298022 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 297957us
I20260709 12:20:33.309500  7237 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.713843  7237 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.296680  7541 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:33.301070  7570 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:33.297004  7591 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.725770  7591 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.304806  7596 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.308092  7633 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.732795  7633 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:33.380672  7501 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:33.404882 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:33.434471 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:33.450142 32041 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:33.473614  7379 consensus_queue.cc:1059] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 26.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:33.581349 32114 raft_consensus.cc:1240] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer 3836f9b5cd36488da56d4540dc9d80ec for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:33.657871  7455 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:33.127948 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:33.783044 32044 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:20:33.783958  7379 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.784538  7379 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:33.785499  7591 raft_consensus.cc:3055] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:33.785929  7591 raft_consensus.cc:740] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:33.126068 32045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:33.671806  7618 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.864584 32045 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:33.865017 32045 raft_consensus.cc:614] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:33.864730  7618 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.866250 32113 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:20:33.866673 32113 pending_rounds.cc:85] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:20:33.866887 32113 pending_rounds.cc:107] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:20:33.867101 32113 raft_consensus.cc:2889] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:33.867228  7618 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:33.670086 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:20:33.868384 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 198364us
I20260709 12:20:33.869058 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:33.671202  7599 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:33.686013 32039 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:33.870867  7599 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.871435 31935 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:33.686177 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 591873us
I20260709 12:20:33.688504  7442 consensus_queue.cc:1048] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:20:33.870110 32010 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:33.693293 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.873293 32119 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 22.
I20260709 12:20:33.873656  7237 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:33.722481  7654 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:33.715538 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.726383  7570 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:33.740379 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.753887  7512 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.761412  7568 consensus_queue.cc:1059] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:20:33.876078 32118 raft_consensus.cc:2410] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:20:33.743384  7501 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.747985  7622 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:20:33.801831 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51540.7us
I20260709 12:20:33.808362  7591 consensus_queue.cc:260] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.875337  7570 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.878062  7501 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.748162  7438 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:33.878675  7591 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:33.813371 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:33.879881  7570 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:33.811694  7379 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:33.846446  7455 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:33.881238  7455 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:33.881717  7541 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.883778  7455 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:33.886459 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:33.887060 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:33.860563 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 61861.1us
I20260709 12:20:33.871569  7599 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:33.302546  7278 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.873342  7596 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:33.874464  7654 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:33.890743  7596 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.888312 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:33.888437  7599 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:33.890910  7541 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:33.875823  7512 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:33.889957 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.873402 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:33.844377 32112 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:33.894207  7654 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.906556  7237 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.928844  7596 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:33.929731  7512 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:33.929867 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:33.930672 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:33.930900 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 57570.5us
I20260709 12:20:33.933391  7455 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:33.933775 32654 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 20.
I20260709 12:20:33.934262  7599 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:33.934397 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:33.935141 31936 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:33.931367 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:33.936355 32044 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 19.
I20260709 12:20:33.932613 32013 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:33.903692  7438 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:20:33.910305 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.905709 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.942233  7379 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:33.941512  7591 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:33.942531  7501 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.945683  7501 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.947961  7501 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.928036  7237 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.943727 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:33.950860  7396 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:33.951299  7396 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.951639  7396 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:33.950856  7628 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:33.962411 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.961581  7237 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:33.969604  7278 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.975095 32045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:33.975788 32045 raft_consensus.cc:606] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:33.975847  7628 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:33.976176 32045 raft_consensus.cc:614] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:33.976437  7628 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:33.980367  7628 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:20:33.984229 32044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:33.984825 32044 raft_consensus.cc:606] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:33.985162 32044 raft_consensus.cc:614] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:33.986675  7594 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:33.987835 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:33.986704 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:33.989732 32039 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:33.990625 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.993553 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.993862 31970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:33.985069  7596 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:33.994422 31969 raft_consensus.cc:2410] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:20:33.996032 32045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
W20260709 12:20:33.996770 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 123952us
I20260709 12:20:33.997025 32045 raft_consensus.cc:606] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:33.988500 32113 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:33.997440 32045 raft_consensus.cc:614] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:33.998678 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:33.999370 32118 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:33.996011  7596 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.008647 32045 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:34.014027 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.038020 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.040534  7591 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.011227 31938 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.048616 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.049144 32084 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.047089 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.050901 32039 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:34.052597 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.045329  7593 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:34.012346  7501 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.051788 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 248) took 5459 ms. Trace:
I20260709 12:20:34.055848 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.056506 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.057125 31968 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:34.043092  7596 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.069245 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.070641 31972 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:20:34.071760 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.072372 32038 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:34.076331 32041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:34.076961 32041 raft_consensus.cc:606] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.077272 32041 raft_consensus.cc:614] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:34.080724 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.083984 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.085672 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.086671 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.055157  7568 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:34.042862  7599 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.093580 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.095242 32042 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 29 index: 14. (term mismatch)
I20260709 12:20:34.101193 32042 pending_rounds.cc:85] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting all ops after (but not including) 13
I20260709 12:20:34.101451 32042 pending_rounds.cc:107] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting uncommitted NO_OP operation due to leader change: 28.14
I20260709 12:20:34.101698 32042 raft_consensus.cc:2889] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:34.096292 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.096725 32045 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:34.098414 32118 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:34.060910  7512 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:34.053349  7570 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:20:34.100729 32041 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
W20260709 12:20:34.103328 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57033.4us
I20260709 12:20:34.108021 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:34.050676  7591 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:34.057652  7593 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.082998  7274 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:34.109776 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:34.079439  7681 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:34.060127  7501 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.092767  7438 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.104403  7512 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:34.118515 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62741us
I20260709 12:20:34.075323  7596 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.132608  7541 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.133219  7541 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.133944  7438 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.134114  7684 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:34.055428 31867 rpcz_store.cc:276] 0709 12:20:28.591996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.593332 (+  1336us) service_pool.cc:224] Handling call
0709 12:20:34.051721 (+5458389us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:32.042907 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:32.045543 (+  2636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:32.045589 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:20:32.045621 (+    32us) tablet.cc:662] Decoding operations
0709 12:20:32.051423 (+  5802us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:32.051511 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:32.051535 (+    24us) tablet.cc:685] Acquiring locks for 30 operations
0709 12:20:32.123699 (+ 72164us) tablet.cc:701] Row locks acquired
0709 12:20:32.123735 (+    36us) write_op.cc:260] PREPARE: finished
0709 12:20:32.123919 (+   184us) write_op.cc:270] Start()
0709 12:20:32.124048 (+   129us) write_op.cc:276] Timestamp: P: 1783599632123837 usec, L: 0
0709 12:20:32.124078 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:20:32.323089 (+199011us) log.cc:844] Serialized 11677 byte log entry
0709 12:20:32.802688 (+479599us) op_driver.cc:464] REPLICATION: finished
0709 12:20:32.906697 (+104009us) write_op.cc:301] APPLY: starting
0709 12:20:32.906813 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:33.325875 (+419062us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:33.325981 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:33.368043 (+ 42062us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:33.368619 (+   576us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=60,mrs_lookups=0
0709 12:20:33.368686 (+    67us) write_op.cc:312] APPLY: finished
0709 12:20:33.369447 (+   761us) log.cc:844] Serialized 260 byte log entry
0709 12:20:33.369628 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:33.369884 (+   256us) write_op.cc:454] Released schema lock
0709 12:20:33.369950 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:33.370390 (+   440us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:33.370430 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":109,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":97531,"apply.run_cpu_time_us":41126,"apply.run_wall_time_us":463696,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129407,"num_ops":30,"prepare.queue_time_us":259750,"prepare.run_cpu_time_us":25876,"prepare.run_wall_time_us":472318,"raft.queue_time_us":224210,"raft.run_cpu_time_us":6091,"raft.run_wall_time_us":75758,"replication_time_us":671027,"thread_start_us":455146,"threads_started":4,"wal-append.queue_time_us":178123}]]}
I20260709 12:20:34.137404 31971 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:34.127023  7512 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:34.135696 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76626.9us
I20260709 12:20:34.140255 31938 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.143563  7396 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:34.145073  7438 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.145787  7396 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:34.146008  7596 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.155604  7593 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.157320  7591 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:34.159578 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.160373  7501 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.164019  7599 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.165093  7541 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.165426 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.173718 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:34.178174 32039 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 23.
I20260709 12:20:34.179705 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:34.186905 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 75652.3us
I20260709 12:20:34.189147 32010 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.189316 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:34.187029  7596 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:34.192693 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.193519  7541 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:34.194235  7541 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:34.194890  7541 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:34.195253  7541 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.195607  7541 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:34.197132 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.198096  7596 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.198633 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.199020  7596 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.203279  7628 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:34.205250 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.205885 32042 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:20:34.206725 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.184571  7438 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.207340 32041 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:34.207479  7438 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.208274 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.209653  7438 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.210641 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.211225 32038 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:34.213133 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.218379 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.224587 32038 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:34.217527  7512 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.228381  7512 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.237651  7541 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.241062 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.242967 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.244378 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.247282 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.251380 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.252187 32045 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 29.
I20260709 12:20:34.268790 32045 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:34.270305 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.271004 32044 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.268273 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:34.273737 32112 raft_consensus.cc:2393] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 21: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:34.271718 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.286079 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:34.282863 31935 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.286861 31964 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:34.288565 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:34.292706 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.293318 31971 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 29.
I20260709 12:20:34.290656 31935 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.301157 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:34.270591  7541 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.301878 32654 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:34.273005  7501 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.304558 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.289106  7596 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:34.297708  7438 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:34.314229  7690 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:34.319410  7690 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.309356  7501 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.323211  7690 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:34.324643  7501 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.326506  7649 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:20:34.322162  7396 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:34.326927  7649 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.327091  7510 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:34.327314  7396 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.328413 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.328341  7396 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:34.329116 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.329391  7541 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:34.329728 32038 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:34.329874 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.330775  7541 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:20:34.331161  7541 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.331492  7541 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:34.314394 32044 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:34.312426  7596 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.338692 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.343196 31938 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.342967 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.344544 32112 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:34.336817  7596 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:34.327435  7649 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:34.345397 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.346299 32116 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.315778  7438 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:20:34.350132 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080010bde0 after lost signal to thread 30823
W20260709 12:20:34.362569 31901 debug-util.cc:398] Leaking SignalData structure 0x7b08002efae0 after lost signal to thread 31904
W20260709 12:20:34.363936 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080033b320 after lost signal to thread 31996
W20260709 12:20:34.364246 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080033b300 after lost signal to thread 31997
I20260709 12:20:34.322016 31964 raft_consensus.cc:2468] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
W20260709 12:20:34.366420 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080003c020 after lost signal to thread 32071
I20260709 12:20:34.357923 32654 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:34.358579  7512 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:34.380492 32116 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:34.359247 31935 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.381661  7512 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:34.359755  7510 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:34.382179  7512 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.382501  7510 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.382947  7510 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:34.383111  7512 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.362944  7438 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:34.357573 31938 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:20:34.393267 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 45882.5us
I20260709 12:20:34.393646 32118 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:34.375389  7649 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:34.379642 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080003c000 after lost signal to thread 32072
W20260709 12:20:34.405069 31901 debug-util.cc:398] Leaking SignalData structure 0x7b08002e8f60 after lost signal to thread 32145
I20260709 12:20:34.400167  7541 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.411777  7501 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:34.412107  7541 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.399498 31863 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 22 to 25, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:34.403080 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.414245 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.415434 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.416311 32010 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.418370 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.419188  7690 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:34.419611  7690 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.419950  7690 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.416955  7618 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:34.401860  7510 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:34.406484 31901 debug-util.cc:398] Leaking SignalData structure 0x7b0800220260 after lost signal to thread 32684
W20260709 12:20:34.410226 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52773.3us
I20260709 12:20:34.420681  7570 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:34.422868 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:34.422737 31863 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.423588  7570 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:34.424018  7570 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:34.424412 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:34.424419  7570 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:34.426105  7274 raft_consensus.cc:2804] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:34.426646  7274 raft_consensus.cc:697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:34.427551  7681 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:34.427464  7274 consensus_queue.cc:237] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.427973  7681 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.428369  7681 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:34.399602  7695 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:34.431262  7695 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:34.425076  7618 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:20:34.434188 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:34.439680 31964 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:20:34.439966  7596 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.424762 31863 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 20 to 26. New cstate: current_term: 26 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:34.442549  7596 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.444485 31863 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 19 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.402288  7438 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.446185  7690 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.455828 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.464366 32039 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:34.465458 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.459232 31965 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec in current term 30: Already voted for candidate f05e4741d35c4cf3a31358c56e65b30c in this term.
I20260709 12:20:34.463196 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.448601  7570 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.469534 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:20:34.475740 31857 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b83e7300b3fe4d21a97d941dee98fa81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 38c927f0ee104b57972899f5b27aa99b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:34.484789 32115 raft_consensus.cc:2368] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22.
I20260709 12:20:34.484653  7570 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:34.485942 32116 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:34.466902  7429 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:20:34.491524  7649 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 30 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.493639 31938 leader_election.cc:400] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22.
I20260709 12:20:34.494040 31938 leader_election.cc:403] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:20:34.494468 32010 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.494959  7649 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.496747 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.497983 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 30 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.499415 32041 raft_consensus.cc:2393] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 30: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:34.503501  7690 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:34.506560 31935 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.509850 32039 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:34.511106 32044 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 21.
I20260709 12:20:34.520605 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.522948 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.525032 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.549144 32038 raft_consensus.cc:2393] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b in current term 26: Already voted for candidate 3836f9b5cd36488da56d4540dc9d80ec in this term.
I20260709 12:20:34.528519  7596 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:34.550343  7596 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.550923 32084 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.551144  7618 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:34.551848  7618 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:34.552551  7596 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.553030  7618 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:34.554328  7618 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:34.532902  7619 consensus_queue.cc:1059] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:34.554935 31963 raft_consensus.cc:2368] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 26. Current term is 27.
I20260709 12:20:34.555473  7619 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:34.555861  7619 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.556690  7619 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.527137  7690 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:20:34.453683  7438 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.519599  7379 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:20:34.446259 31863 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:34.558034  7619 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:34.560992 31863 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 13 to 16, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 16 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.548629 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.563360 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.564095 32116 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 27: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:34.530496  7512 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:20:34.579169 32084 leader_election.cc:400] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 26. Current term is 27.
I20260709 12:20:34.580289 32084 leader_election.cc:403] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:34.580960  7570 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:34.583300 32038 raft_consensus.cc:1240] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:20:34.584182 32044 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 29. Current term is 30. Ops: [29.14-29.14]
I20260709 12:20:34.585439 32039 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:20:34.588773 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.589316 32039 raft_consensus.cc:3060] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.590180 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.453210  7681 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.596817 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.574631  7649 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 21. Current term is 22.
I20260709 12:20:34.600523 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:34.576267  7438 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:34.599742 32038 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:34.592094 31863 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.604696 32115 raft_consensus.cc:1240] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Rejecting Update request from peer f05e4741d35c4cf3a31358c56e65b30c for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:20:34.605895 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:34.609068 32039 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:34.615614 31863 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 22 to 23, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.619464  7377 raft_consensus.cc:3055] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:34.619863  7377 raft_consensus.cc:740] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:34.622022  7689 consensus_queue.cc:1059] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:34.606887 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.626784 31963 raft_consensus.cc:2468] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:34.623886  7429 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.630565  7429 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.631596  7681 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:34.633944 32084 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:34.642210 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.644333 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.644646 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:34.645252 32039 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:34.645404 32038 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:34.648592 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:34.607345  7622 consensus_queue.cc:1059] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:20:34.632632  7429 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.645534  7512 consensus_queue.cc:1059] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:20:34.628921  7438 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.617687 31863 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 14 to 24, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 24 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:34.655200  7593 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:34.658880  7593 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.657454  7618 consensus_queue.cc:1059] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:20:34.659452  7593 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.659106 31863 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 18 to 19, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 19 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.660193  7512 raft_consensus.cc:3055] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:34.660576  7512 raft_consensus.cc:740] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.660835 31863 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 13 to 18, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 18 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.661391  7512 consensus_queue.cc:260] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.662940  7512 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:34.666682  7681 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:34.667085  7681 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:34.667940  7593 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.672065  7593 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.640831  7379 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.674742 31969 raft_consensus.cc:1240] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 30 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 29. Current term is 30. Ops: [29.14-29.14]
I20260709 12:20:34.674984  7379 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.677070 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:34.677440  7379 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.646487  7278 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:34.678102  7510 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.683948 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:34.693249  7689 raft_consensus.cc:2804] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:34.693781  7622 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:20:34.694326  7622 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:34.667937  7681 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.695549  7681 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:34.695226  7622 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.14, Last appended by leader: 14, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.692229  7438 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.677704 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.697199 32045 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.693846  7689 raft_consensus.cc:697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:34.678871 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:34.678804 31968 raft_consensus.cc:1240] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:34.703233 32039 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:34.706185 31935 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
W20260709 12:20:34.709246 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56591.7us
I20260709 12:20:34.715682 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.691385 31863 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.711432  7593 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:34.735052  7679 consensus_queue.cc:1059] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:34.736040 31863 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 12 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.736621  7679 raft_consensus.cc:3055] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:34.737241  7679 raft_consensus.cc:740] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:34.740006 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:34.699131  7278 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:34.705089  7618 consensus_queue.cc:1243] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer 3836f9b5cd36488da56d4540dc9d80ec log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:20:34.707834  7622 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:34.747997 31863 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.750422 31863 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 19 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:34.745352  7679 consensus_queue.cc:260] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:34.758247 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 116635us
I20260709 12:20:34.767736 32045 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 26.
I20260709 12:20:34.768986 31935 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:34.743067  7278 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:34.744954  7593 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.773373 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.774909  7396 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:34.775488  7396 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:34.786710 32013 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:34.787065  7689 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.788100  7455 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:34.788169  7377 consensus_queue.cc:260] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.797654  7396 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.806813 32115 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:34.808218  7570 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903) with higher term. Message: Invalid argument: T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for earlier term 26. Current term is 27.
I20260709 12:20:34.809844  7377 raft_consensus.cc:3060] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:34.810642  7501 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:34.810859  7274 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.816952  7274 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.817991 32113 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:34.819036  7274 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.820658  7679 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.826216 31963 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:34.827806 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:34.829957 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:34.839107 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60918.2us
I20260709 12:20:34.850080 32038 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:34.878221  7596 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:20:34.880267 32654 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:20:34.887874  7274 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:20:34.911154 31965 raft_consensus.cc:1240] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:20:34.907120  7455 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:34.912894  7455 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:34.915031  7455 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:34.916023 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:20:34.928136  7715 log.cc:927] Time spent T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:20:34.875764  7593 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:34.935385 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 66878us
I20260709 12:20:34.937987 32045 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:34.945580 31964 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:20:34.948769 32038 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:34.888561  7694 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
W20260709 12:20:34.967669 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84771.7us
I20260709 12:20:34.987962 32039 raft_consensus.cc:1240] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:34.990259 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:34.999058 32010 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.000691  7455 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:35.011309  7438 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:35.017307 32115 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:35.024467 32118 raft_consensus.cc:1275] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 12:20:35.024887 32118 pending_rounds.cc:85] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:20:35.025115 32118 pending_rounds.cc:107] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:20:34.998646  7633 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:35.025344 32118 raft_consensus.cc:2889] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:35.025703  7633 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:35.027004 31969 raft_consensus.cc:2410] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 38c927f0ee104b57972899f5b27aa99b for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 13.
I20260709 12:20:35.026625  7633 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.028086  7274 consensus_queue.cc:1048] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.035509  7379 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:35.041729  7278 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.044200 32656 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:35.047036 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:35.049973 32118 raft_consensus.cc:2393] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 27: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:35.060292 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:35.050292  7278 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:35.069052 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:35.069027  7568 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:35.074481 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.075273 32045 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:20:35.077117 32010 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.081475  7274 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:35.082108  7689 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:35.083063 32044 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:35.083647  7596 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:35.092365 31935 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:35.112164 32118 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:35.116044  7396 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:35.106581  7596 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:35.108788  7622 consensus_queue.cc:1059] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:20:35.129477  7622 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:35.129877  7622 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:35.130699  7622 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.131836  7622 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:35.134987  7707 consensus_queue.cc:1175] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f05e4741d35c4cf3a31358c56e65b30c" responder_term: 22 status { last_received { term: 22 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 22 index: 11 } }
I20260709 12:20:35.146791  7596 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:35.196691  7596 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.209048  7455 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 3 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:35.214982 32654 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 3 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:35.227653 32118 raft_consensus.cc:2955] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 3 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:35.249924 32044 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:35.252394  7618 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.260360 32044 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:35.269826  7593 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:20:35.298869 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b83e7300b3fe4d21a97d941dee98fa81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 4)
I20260709 12:20:35.302210 31963 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:35.303812  7618 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.308131 32118 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:20:35.308578 32118 pending_rounds.cc:85] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Aborting all ops after (but not including) 12
I20260709 12:20:35.308866 32118 pending_rounds.cc:107] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:20:35.309168 32118 raft_consensus.cc:2889] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:35.341259 32118 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:20:35.345019  7725 log.cc:927] Time spent T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
I20260709 12:20:35.352962  7596 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:35.331028  7593 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:20:35.363826 32045 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:35.366081  7438 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.451813 32039 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:35.453086  7716 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.453684  7716 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.459115  7278 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:35.460060  7716 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:35.461838 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:35.462495 32118 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:20:35.463269 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:35.466558 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.467770  7455 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:35.468146  7455 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.468492  7455 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:35.471138  7278 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:35.471709  7278 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.476231 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:35.476490 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:35.476919 32118 raft_consensus.cc:2393] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c in current term 24: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:20:35.478190  7278 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:35.481242  7455 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.482295 32039 raft_consensus.cc:2368] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:35.487838 31935 leader_election.cc:400] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:35.488157 31935 leader_election.cc:403] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:35.489480  7438 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:35.503826  7455 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:35.506963 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:35.510635 31965 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:35.518453 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:35.519203 32118 raft_consensus.cc:3060] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:35.519563  7377 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:35.538913 32113 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:35.540661  7278 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:35.541178  7740 log.cc:927] Time spent T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.050s	user 0.002s	sys 0.000s
I20260709 12:20:35.562427 32118 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:35.569613 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:35.568495  7438 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 24. Current term is 25.
I20260709 12:20:35.579996 32045 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:35.623541 31963 raft_consensus.cc:2468] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:35.623274 32013 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:35.624678  7716 raft_consensus.cc:2804] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:35.625177  7716 raft_consensus.cc:697] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:35.629937 32045 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:35.637086  7377 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:20:35.625988  7716 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.751595 32044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:35.752501 32044 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.752942 32044 raft_consensus.cc:614] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.755831 32044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:35.756580 32044 raft_consensus.cc:606] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.756994 32044 raft_consensus.cc:614] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.772745 32038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:35.773344 32038 raft_consensus.cc:606] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.773766 32038 raft_consensus.cc:614] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.792804 32044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:35.793440 32044 raft_consensus.cc:606] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.793778 32044 raft_consensus.cc:614] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.798820 32038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:35.799444 32038 raft_consensus.cc:606] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:35.799819 32038 raft_consensus.cc:614] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.801589 32112 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:20:35.803687 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:35.805415 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:35.806150 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:35.806784 32038 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:20:35.806687  7438 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:35.808406  7593 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.870460  7593 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.873814  7593 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.877542 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:35.878177 32113 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 27.
I20260709 12:20:35.879429 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:35.882424  7429 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:35.885310  7429 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.885735  7429 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:35.891917  7593 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:35.892376 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:35.950363  7429 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:35.953635 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:35.954269 32113 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:35.962232 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:35.966780 32113 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 28.
I20260709 12:20:35.973760 31938 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:35.978986  7718 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:35.979545  7718 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 28 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:35.980422  7718 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.009392  7568 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:36.010016  7568 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.013857  7707 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.014424  7429 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:36.014513  7707 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.016881  7707 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:36.018069 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:36.018733 31963 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 30.
I20260709 12:20:36.020038 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:36.018663 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:36.024036 32039 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 30.
I20260709 12:20:36.025552 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:36.026355 32118 raft_consensus.cc:2368] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 27. Current term is 28.
I20260709 12:20:36.027838 32013 leader_election.cc:400] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 27. Current term is 28.
I20260709 12:20:36.028143 32013 leader_election.cc:403] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:36.027628 31867 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 27, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.028759  7716 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:20:36.027854  7622 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:20:36.029640 31867 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 23, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.030794  7622 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.031191  7622 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:36.031672 31867 catalog_manager.cc:5697] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:36.042841  7568 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:36.049612 32113 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:36.035475 31867 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.052337 31867 catalog_manager.cc:5697] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 16 to 17, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 17 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:36.053864  7274 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:36.071831 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:36.071632 31867 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 27, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 27 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:36.074043 31867 catalog_manager.cc:5697] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 22, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.075578 31867 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.077083 31867 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 29, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 29 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.089314 31867 catalog_manager.cc:5697] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 24, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 24 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:36.057713  7716 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 27. Current term is 28.
I20260709 12:20:36.094985  7622 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.103273 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:36.103904 31963 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:20:36.114264 31863 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 234) took 15662 ms (client timeout 15000 ms). Trace:
I20260709 12:20:36.121996  7622 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:36.123414 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:36.098963 31867 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 26, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 26 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.129891 32045 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:36.146889 32045 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 31.
I20260709 12:20:36.148371 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:36.150172  7707 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:20:36.155721  7707 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:36.156757  7707 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:36.127781 31863 rpcz_store.cc:269] 0709 12:20:20.452099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.452420 (+   321us) service_pool.cc:224] Handling call
0709 12:20:36.114179 (+15661759us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:34.980832 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:35.111429 (+130597us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:35.113042 (+  1613us) write_op.cc:435] Acquired schema lock
0709 12:20:35.114890 (+  1848us) tablet.cc:662] Decoding operations
0709 12:20:35.127732 (+ 12842us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:35.127811 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:35.127840 (+    29us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:20:35.128922 (+  1082us) tablet.cc:701] Row locks acquired
0709 12:20:35.128941 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:20:35.129085 (+   144us) write_op.cc:270] Start()
0709 12:20:35.204715 (+ 75630us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b540036c5a8. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f229258b9e1 00007f229258a64c 00007f228fc1a624 00007f228fc05286 00007f228fc04d58 00007f228fc099f1 00007f228fc099a9 00007f228fc09939 00007f228fc09901 00007f228fc08b2d 0000000000388634
0709 12:20:35.204932 (+   217us) write_op.cc:276] Timestamp: P: 1783599635129030 usec, L: 0
0709 12:20:35.204958 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:20:35.206246 (+  1288us) log.cc:844] Serialized 5840 byte log entry
0709 12:20:35.411759 (+205513us) op_driver.cc:464] REPLICATION: finished
0709 12:20:35.544414 (+132655us) spinlock_profiling.cc:243] Waited 9.04 ms on lock 0x7b18004bad20. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2291faaab1 00007f2291fb7281 00007f22890e23f3 00007f22890e47c1 00007f22890e4779 00007f22890e4709 00007f22890e46d1 00007f22890e39cd 0000000000388634 0000000000388479 00007f22890c583a
0709 12:20:35.544579 (+   165us) write_op.cc:301] APPLY: starting
0709 12:20:35.544686 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:35.756259 (+211573us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:35.756302 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:35.855617 (+ 99315us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:35.855912 (+   295us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 12:20:35.855963 (+    51us) write_op.cc:312] APPLY: finished
0709 12:20:35.856335 (+   372us) log.cc:844] Serialized 139 byte log entry
0709 12:20:35.875424 (+ 19089us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:35.875584 (+   160us) write_op.cc:454] Released schema lock
0709 12:20:35.875641 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:35.928999 (+ 53358us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:35.929031 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17547,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":91060,"apply.run_cpu_time_us":23328,"apply.run_wall_time_us":384397,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66767,"num_ops":15,"prepare.queue_time_us":20420,"prepare.run_cpu_time_us":17353,"prepare.run_wall_time_us":289060,"raft.queue_time_us":119730,"raft.run_cpu_time_us":67795,"raft.run_wall_time_us":94969,"replication_time_us":206646,"spinlock_wait_cycles":47195776,"thread_start_us":223487,"threads_started":4,"wal-append.queue_time_us":78473}]]}
I20260709 12:20:36.195195 31964 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:36.198922  7274 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:36.169926 31963 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 31.
I20260709 12:20:36.131950 31867 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 19 to 21, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 21 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.223134 31867 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 25, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 25 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.242029 31867 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.250115  7718 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:20:36.250187 31867 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 23, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.253412 31867 catalog_manager.cc:5697] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 21, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 21 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:36.255100 31867 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 15 to 21, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 21 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.258939 31867 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 24 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.260620 31867 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 24 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.257681  7718 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:36.272007 31867 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 25 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:36.274552 31867 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.274534 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:36.286283  7377 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.286927  7377 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.289070  7377 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:36.291072 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:36.295534 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:36.296424 32045 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:20:36.311823 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:36.313958  7622 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:36.314414  7622 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.314764  7622 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:36.325870 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:36.320305  7622 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.376147  7718 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:36.385783 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:36.389520 31936 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:36.382226  7622 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:36.392388  7718 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:36.401050 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:36.405449 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:36.406428 32045 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:36.452960 32045 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:20:36.457861 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:36.470346  7759 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:36.477641 32042 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:36.493046  7718 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:36.522456 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 65597.5us
I20260709 12:20:36.476853  7759 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:36.546289  7759 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.571872 32113 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:36.573385  7429 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:36.582829  7593 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:36.583405  7593 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:36.586959  7593 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:36.591825 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:36.595057 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:36.596269 32042 raft_consensus.cc:2368] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24.
I20260709 12:20:36.599958 31935 leader_election.cc:400] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24.
I20260709 12:20:36.600281 31935 leader_election.cc:403] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:36.600797  7756 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:20:36.668710 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 52327.6us
I20260709 12:20:36.667963  7756 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) with higher term. Message: Invalid argument: T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f05e4741d35c4cf3a31358c56e65b30c for earlier term 23. Current term is 24.
W20260709 12:20:36.682542 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 116140us
I20260709 12:20:36.724771 31964 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:20:36.726505  7761 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:36.746752 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 60830.2us
I20260709 12:20:36.765264 32042 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:20:36.778034  7377 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:36.918462 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:36.985941 31867 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 249) took 2436 ms. Trace:
I20260709 12:20:36.986405 31867 rpcz_store.cc:276] 0709 12:20:34.549982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:34.550132 (+   150us) service_pool.cc:224] Handling call
0709 12:20:36.985871 (+2435739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:36.507271 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:36.519290 (+ 12019us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:36.519330 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:20:36.519350 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:36.558624 (+ 39274us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:36.558681 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:36.558706 (+    25us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:20:36.560081 (+  1375us) tablet.cc:701] Row locks acquired
0709 12:20:36.560099 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:20:36.560229 (+   130us) write_op.cc:270] Start()
0709 12:20:36.560398 (+   169us) write_op.cc:276] Timestamp: P: 1783599636560186 usec, L: 0
0709 12:20:36.560423 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:20:36.561080 (+   657us) log.cc:844] Serialized 8142 byte log entry
0709 12:20:36.690838 (+129758us) op_driver.cc:464] REPLICATION: finished
0709 12:20:36.718936 (+ 28098us) write_op.cc:301] APPLY: starting
0709 12:20:36.719041 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:36.878227 (+159186us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:36.878343 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:36.901109 (+ 22766us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:36.901434 (+   325us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 12:20:36.901478 (+    44us) write_op.cc:312] APPLY: finished
0709 12:20:36.901871 (+   393us) log.cc:844] Serialized 188 byte log entry
0709 12:20:36.902047 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:36.902240 (+   193us) write_op.cc:454] Released schema lock
0709 12:20:36.902295 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:36.902678 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:36.902707 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":92526,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":27820,"apply.run_cpu_time_us":26088,"apply.run_wall_time_us":183743,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91823,"num_ops":21,"prepare.queue_time_us":106056,"prepare.run_cpu_time_us":5169,"prepare.run_wall_time_us":91869,"replication_time_us":130274,"thread_start_us":84795,"threads_started":2}]]}
I20260709 12:20:37.029341 32039 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:37.030963  7761 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:37.033668 31864 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 17 to 27. New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:37.061555 31964 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:37.064337  7758 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:37.063055 31864 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:37.151603 31867 catalog_manager.cc:5697] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 24 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.153136 31867 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 26, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 26 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.154621 31867 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.155915 31867 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 26 to 27, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.157272 31867 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.159004 31867 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.179010 31867 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 3 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:37.180733 31867 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 25 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.199142 31867 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.200876 31867 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.202265 31867 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 25, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 25 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.241134 31867 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 14 to 28. New cstate: current_term: 28 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
I20260709 12:20:37.242570 31867 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 18 to 23, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 23 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:20:37.243587 31854 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet b83e7300b3fe4d21a97d941dee98fa81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 14
I20260709 12:20:37.244057 31867 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } }
W20260709 12:20:37.217605 31868 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 154) took 36708 ms (client timeout 15000 ms). Trace:
W20260709 12:20:37.246560 31868 rpcz_store.cc:269] 0709 12:20:00.514694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.515364 (+   670us) service_pool.cc:224] Handling call
0709 12:20:37.217402 (+36702038us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28685,"reported_tablets":27}
W20260709 12:20:37.252882 31864 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 248) took 14898 ms (client timeout 15000 ms). Trace:
W20260709 12:20:37.253307 31864 rpcz_store.cc:269] 0709 12:20:22.367262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.369785 (+  2523us) service_pool.cc:224] Handling call
0709 12:20:23.176332 (+806547us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540036c754. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c139090 00007f228c1d7f3a 00007f2291f8de9c 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc 00007f229054c392 00007f228a50a48c 00007f228a5098c6
0709 12:20:37.252809 (+14076477us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:37.072306 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:37.072813 (+   507us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:37.072855 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:20:37.072868 (+    13us) tablet.cc:662] Decoding operations
0709 12:20:37.073257 (+   389us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:37.073307 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:37.073337 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:20:37.073622 (+   285us) tablet.cc:701] Row locks acquired
0709 12:20:37.073635 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:20:37.073790 (+   155us) write_op.cc:270] Start()
0709 12:20:37.073906 (+   116us) write_op.cc:276] Timestamp: P: 1783599637073747 usec, L: 0
0709 12:20:37.073925 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:20:37.074523 (+   598us) log.cc:844] Serialized 907 byte log entry
0709 12:20:37.076948 (+  2425us) op_driver.cc:464] REPLICATION: finished
0709 12:20:37.077359 (+   411us) write_op.cc:301] APPLY: starting
0709 12:20:37.077437 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:37.078790 (+  1353us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:37.078891 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:37.080176 (+  1285us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:37.080336 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:20:37.080383 (+    47us) write_op.cc:312] APPLY: finished
0709 12:20:37.080562 (+   179us) log.cc:844] Serialized 33 byte log entry
0709 12:20:37.080709 (+   147us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:37.080793 (+    84us) write_op.cc:454] Released schema lock
0709 12:20:37.080842 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:37.081143 (+   301us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:37.081168 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":24521,"reported_tablets":51,"spinlock_wait_cycles":464402304,"child_traces":[["op",{"apply.queue_time_us":227,"apply.run_cpu_time_us":3752,"apply.run_wall_time_us":3752,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8352,"num_ops":2,"prepare.queue_time_us":1618,"prepare.run_cpu_time_us":2480,"prepare.run_wall_time_us":2480,"replication_time_us":2907}]]}
I20260709 12:20:37.294060 32042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:37.294744 32042 raft_consensus.cc:606] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:37.295114 32042 raft_consensus.cc:614] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:37.298007 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:37.298616 32039 raft_consensus.cc:606] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:37.302709 31963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:37.303321 31963 raft_consensus.cc:606] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:37.311462 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:37.313374 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:37.325601 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:37.328745  7754 raft_consensus.cc:993] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:37.329316  7754 raft_consensus.cc:1081] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:37.339257 32113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:37.339883 32113 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:37.340237 32113 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:37.365571 32113 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:37.367487 32113 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:37.370275 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:37.370864 32045 raft_consensus.cc:3055] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:37.371135 32045 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:37.371820 32045 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:37.373037 32045 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:37.375526 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:37.376084 31964 raft_consensus.cc:3060] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:37.441036 31964 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:37.505666 32084 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:37.506619  7758 raft_consensus.cc:2804] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:37.507169  7758 raft_consensus.cc:697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:37.508064  7758 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:37.522193 32045 raft_consensus.cc:2468] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:20:37.641768  7593 raft_consensus.cc:993] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:37.642249  7593 raft_consensus.cc:1081] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:37.644117 32045 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:20:37.644804 32045 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:37.645150 32045 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:37.645819 31865 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 155) took 19888 ms (client timeout 15000 ms). Trace:
W20260709 12:20:37.646306 31865 rpcz_store.cc:269] 0709 12:20:17.761877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.765566 (+  3689us) service_pool.cc:224] Handling call
0709 12:20:37.645623 (+19880057us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1553312,"reported_tablets":27}
I20260709 12:20:37.653738 32045 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:37.656922 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:37.657511 31963 raft_consensus.cc:3055] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:37.657802 31963 raft_consensus.cc:740] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:37.658128 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:37.658749 32654 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:37.658772 31963 consensus_queue.cc:260] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:37.660233 31963 raft_consensus.cc:3060] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:37.665470 32654 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:20:37.666776 32011 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:37.667846 32045 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 17 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:37.670524  7754 raft_consensus.cc:2804] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:37.672894  7754 raft_consensus.cc:697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:37.676107 31963 raft_consensus.cc:2468] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 17.
I20260709 12:20:37.673995  7754 consensus_queue.cc:237] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:37.756261 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 69424.5us
W20260709 12:20:37.761050 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 69437.7us
I20260709 12:20:37.767686 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet b83e7300b3fe4d21a97d941dee98fa81 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:60388
I20260709 12:20:37.772485  7787 tablet_replica.cc:333] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:20:37.773741  7787 raft_consensus.cc:2243] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.774900  7787 raft_consensus.cc:2272] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.778926  7787 ts_tablet_manager.cc:1916] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:37.797514  7787 ts_tablet_manager.cc:1929] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.13
I20260709 12:20:37.798110  7787 log.cc:1199] T b83e7300b3fe4d21a97d941dee98fa81 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/b83e7300b3fe4d21a97d941dee98fa81
I20260709 12:20:37.801941 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet b83e7300b3fe4d21a97d941dee98fa81 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:37.807013 31865 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 21 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.810531 31870 catalog_manager.cc:5697] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.856357 31870 catalog_manager.cc:5697] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 24 to 28, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 28 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.858597 31870 catalog_manager.cc:5697] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 23 to 24, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 24 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.875797 31870 catalog_manager.cc:5697] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:37.908131 31866 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 247) took 37475 ms (client timeout 15000 ms). Trace:
W20260709 12:20:37.910244 31866 rpcz_store.cc:269] 0709 12:20:00.432606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.445810 (+ 13204us) service_pool.cc:224] Handling call
0709 12:20:37.907928 (+37462118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17848,"reported_tablets":51}
I20260709 12:20:37.957039 31870 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 156) took 4381 ms. Trace:
I20260709 12:20:37.970356 31869 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:37.975071 31870 rpcz_store.cc:276] 0709 12:20:33.575923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:33.576136 (+   213us) service_pool.cc:224] Handling call
0709 12:20:33.716358 (+140222us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890ab0b2 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:20:37.919778 (+4203420us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:37.919813 (+    35us) write_op.cc:454] Released schema lock
0709 12:20:37.956977 (+ 37164us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:37.892316 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:37.892800 (+   484us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:37.892847 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:20:37.892867 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:37.893424 (+   557us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:37.893476 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:37.893501 (+    25us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:20:37.893879 (+   378us) tablet.cc:701] Row locks acquired
0709 12:20:37.893897 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:20:37.894024 (+   127us) write_op.cc:270] Start()
0709 12:20:37.894134 (+   110us) write_op.cc:276] Timestamp: P: 1783599637893982 usec, L: 0
0709 12:20:37.894154 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:20:37.894867 (+   713us) log.cc:844] Serialized 1658 byte log entry
0709 12:20:37.911648 (+ 16781us) op_driver.cc:464] REPLICATION: finished
0709 12:20:37.911974 (+   326us) write_op.cc:301] APPLY: starting
0709 12:20:37.912051 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:37.914059 (+  2008us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:37.914161 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:37.916320 (+  2159us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:37.916446 (+   126us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:20:37.916473 (+    27us) write_op.cc:312] APPLY: finished
0709 12:20:37.916658 (+   185us) log.cc:844] Serialized 49 byte log entry
0709 12:20:37.916858 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:37.916955 (+    97us) write_op.cc:454] Released schema lock
0709 12:20:37.916996 (+    41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"dns_us":175,"mutex_wait_us":70007,"reported_tablets":27,"spinlock_wait_cycles":21657472,"child_traces":[["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":5208,"apply.run_wall_time_us":5276,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16704,"num_ops":4,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":2772,"prepare.run_wall_time_us":2785,"replication_time_us":17322}]]}
I20260709 12:20:38.002537 32654 consensus_queue.cc:237] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:38.028851 31963 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:38.030942 32118 raft_consensus.cc:1275] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:38.030419  7779 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:20:38.035491  7710 consensus_queue.cc:1048] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:38.084048 32042 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:38.085714  7707 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:38.095788  7710 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 7 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:38.115128 32113 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 7 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:38.118443 31964 raft_consensus.cc:2955] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 7 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:38.154433 31963 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:38.166747  7782 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:38.179978 31964 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:38.181914  7754 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:38.213210 31866 catalog_manager.cc:5697] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 7 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 22 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:38.236585 32656 raft_consensus.cc:1275] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:38.241691 31855 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1612f8e575bd46cf8069fa1c21ca192b with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:38.248091  7799 consensus_queue.cc:1048] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:20:38.287568 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 1612f8e575bd46cf8069fa1c21ca192b with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:37884
I20260709 12:20:38.339066 31869 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55344 (request call id 235) took 2645 ms. Trace:
I20260709 12:20:38.349357 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet b83e7300b3fe4d21a97d941dee98fa81 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:60388
I20260709 12:20:38.362947  7805 tablet_replica.cc:333] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:20:38.364070  7805 raft_consensus.cc:2243] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.364748  7805 raft_consensus.cc:2272] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.366016 31864 catalog_manager.cc:5697] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 16 to 17, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 17 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.339679 31869 rpcz_store.cc:276] 0709 12:20:35.693861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:35.694005 (+   144us) service_pool.cc:224] Handling call
0709 12:20:38.338974 (+2644969us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:37.993037 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:37.993543 (+   506us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:37.993581 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:20:37.993594 (+    13us) tablet.cc:662] Decoding operations
0709 12:20:37.993925 (+   331us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:37.993982 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:37.994007 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:37.994191 (+   184us) tablet.cc:701] Row locks acquired
0709 12:20:37.994206 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:20:37.994387 (+   181us) write_op.cc:270] Start()
0709 12:20:38.003107 (+  8720us) write_op.cc:276] Timestamp: P: 1783599637994304 usec, L: 0
0709 12:20:38.003133 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:20:38.003780 (+   647us) log.cc:844] Serialized 530 byte log entry
0709 12:20:38.005646 (+  1866us) op_driver.cc:464] REPLICATION: finished
0709 12:20:38.006059 (+   413us) write_op.cc:301] APPLY: starting
0709 12:20:38.006138 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:38.006936 (+   798us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:38.007023 (+    87us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:38.007723 (+   700us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:38.007879 (+   156us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:38.007922 (+    43us) write_op.cc:312] APPLY: finished
0709 12:20:38.008085 (+   163us) log.cc:844] Serialized 25 byte log entry
0709 12:20:38.008233 (+   148us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:38.008311 (+    78us) write_op.cc:454] Released schema lock
0709 12:20:38.008348 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:38.008634 (+   286us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:38.008663 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28831,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":235,"apply.run_cpu_time_us":2550,"apply.run_wall_time_us":2550,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4176,"num_ops":1,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":2463,"prepare.run_wall_time_us":11036,"replication_time_us":2397}]]}
I20260709 12:20:38.403972  7805 ts_tablet_manager.cc:1916] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:38.497226 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet b83e7300b3fe4d21a97d941dee98fa81 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:38.600575 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:38.627048  7805 ts_tablet_manager.cc:1929] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 12:20:38.634121  7805 log.cc:1199] T 1612f8e575bd46cf8069fa1c21ca192b P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/1612f8e575bd46cf8069fa1c21ca192b
I20260709 12:20:38.640542 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 1612f8e575bd46cf8069fa1c21ca192b (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:38.773648 31963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:38.774251 31963 raft_consensus.cc:606] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:38.781337 31963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:38.783825 31963 raft_consensus.cc:606] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:38.787348 31963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:38.788086 31963 raft_consensus.cc:606] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:38.788105 32118 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:38.789335 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:38.791119 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:38.791867 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:38.814970 32042 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:38.816627  7809 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:38.819590 31963 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:38.825664  7593 raft_consensus.cc:993] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:38.826148  7593 raft_consensus.cc:1081] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:38.937129  7808 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:38.940073 32045 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:20:38.940687 32045 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:38.941067 32045 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:38.937850 32085 consensus_peers.cc:597] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6679c928e92e4f799b88357707650b60. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:38.943683 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 114533us
I20260709 12:20:38.969794 32045 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:38.990609 32045 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:39.010812 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 74613.5us
W20260709 12:20:39.059034 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 120091us
I20260709 12:20:39.093124 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:39.093775 31963 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:39.094072 31963 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:39.095075 31963 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:39.102577 31963 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:39.123066 31963 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:39.136281 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:39.146847 32118 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:39.153213 32010 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:39.177619 32118 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:39.174468  7827 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:20:39.193764 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 257218us
W20260709 12:20:39.197467 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64071.9us
W20260709 12:20:39.214192 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61073.6us
W20260709 12:20:39.230443 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 80966.1us
I20260709 12:20:39.231279  7827 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:39.258126  7827 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:39.369063  7784 raft_consensus.cc:993] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:39.369794  7784 raft_consensus.cc:993] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:39.370251  7839 raft_consensus.cc:1081] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:39.370405  7784 raft_consensus.cc:1081] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:39.384857 32044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78b16558212549a98cd68f4e744eb410"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:20:39.385118 32042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58404
I20260709 12:20:39.385607 32044 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:39.386469 32042 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:39.386901 32042 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:39.336055  7831 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:39.386013 32044 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:39.394796 31864 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 25 to 26, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 26 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.394480 32042 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:39.417742 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84911us
W20260709 12:20:39.430823 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 98910.6us
I20260709 12:20:39.432014 32042 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:39.432603 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:39.433298 31969 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:39.433597 31969 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:39.438944 32044 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:39.451740 31963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:39.452313 31963 raft_consensus.cc:3055] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:39.452597 31963 raft_consensus.cc:740] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:39.453310 31963 consensus_queue.cc:260] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:39.454433 31963 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:39.462548 31963 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:39.463541 32010 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:39.464277  7818 raft_consensus.cc:2804] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:39.434784 31969 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:39.466078 31969 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:39.464874  7818 raft_consensus.cc:697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:39.466753 32044 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:39.496568 31969 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
W20260709 12:20:39.494261 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 44514.4us
I20260709 12:20:39.506637 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b16558212549a98cd68f4e744eb410" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:39.488566 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:39.507262 32656 raft_consensus.cc:3060] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:39.507516 32115 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:39.480705 32042 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:39.495779 31968 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:39.512054 32010 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:39.516676  7800 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:39.517360  7800 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:39.519078  7800 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:39.520349 31869 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 20 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.468377  7818 consensus_queue.cc:237] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:39.546283 32656 raft_consensus.cc:2468] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:20:39.559682 32115 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:20:39.638731  7853 ts_tablet_manager.cc:933] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:39.768584  7853 tablet_copy_client.cc:323] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:20:39.770944 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 6679c928e92e4f799b88357707650b60 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:20:39.771557 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 6679c928e92e4f799b88357707650b60 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-6679c928e92e4f799b88357707650b60
I20260709 12:20:39.782572 32131 tablet_copy_source_session.cc:215] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:39.788987  7853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6679c928e92e4f799b88357707650b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:39.794283 32042 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:39.816499 31968 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:39.817975  7819 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:39.826550  7853 tablet_copy_client.cc:806] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:39.834651  7853 tablet_copy_client.cc:670] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:39.837740 32112 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:39.839741 31971 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:39.843350  7819 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
W20260709 12:20:39.856798 32011 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f4287254496472398f69d3a10b48d77. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:39.856711  7800 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:39.870692 32115 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:39.873821  7819 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:39.875522  7853 tablet_copy_client.cc:538] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:39.934634  7754 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Committing config change with OpId 25.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:39.943315 31971 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:39.945062 31965 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:39.946833  7800 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:39.950737 32112 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:39.977815 31864 catalog_manager.cc:5697] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 22 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.987169 31864 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 26 to 27, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:40.010840 32656 raft_consensus.cc:1275] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:40.011294 32112 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:40.012878  7837 consensus_queue.cc:1048] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:40.021747  7853 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:40.051014 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:40.054965  7819 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:40.086777 31969 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:40.115653 31870 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.110417  7819 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:20:40.161164 32118 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:20:40.236493 32085 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 775fd46a41044b1ea06bc796ef1b233b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.311996 32044 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:40.323928 31969 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:40.333398 32045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:40.350585 32045 raft_consensus.cc:606] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:40.416972 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:40.435930 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:20:40.437008  7852 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.054s	user 0.003s	sys 0.000s
I20260709 12:20:40.437450 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:40.483162  7809 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:20:40.488204 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 63682.5us
W20260709 12:20:40.493724 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 47589.5us
I20260709 12:20:40.542440  7811 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:40.544082  7819 raft_consensus.cc:993] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:40.544713  7819 raft_consensus.cc:1081] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:40.547176 32112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:40.547853 32112 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:40.548269 32112 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:40.558883  7853 log.cc:927] Time spent T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:20:40.564188  7885 ts_tablet_manager.cc:933] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:40.576032 32112 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:40.579092 32112 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:40.596238 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 75701.9us
I20260709 12:20:40.641747  7885 tablet_copy_client.cc:323] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:20:40.652062 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:40.652870 32042 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:40.653218 32042 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:40.654456 32042 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:40.656024 32042 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:40.661769 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 8f4287254496472398f69d3a10b48d77 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:20:40.662434 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 8f4287254496472398f69d3a10b48d77 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-8f4287254496472398f69d3a10b48d77
I20260709 12:20:40.664567 31968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:40.671799 31968 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:40.674455 32042 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:20:40.676371 32057 tablet_copy_source_session.cc:215] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:40.682682  7885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4287254496472398f69d3a10b48d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.705145  7811 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.727509 31968 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:20:40.745101  7853 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:40.764062 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:40.767107 31964 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.769749 32041 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.766286  7874 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:40.774623  7874 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:40.777233  7874 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:40.801004 32112 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:40.808319 31869 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.808931 31864 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 26 to 27, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 27 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:40.821090  7853 tablet_bootstrap.cc:492] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:40.822243  7853 ts_tablet_manager.cc:1403] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.801s	user 0.041s	sys 0.015s
I20260709 12:20:40.833026  7853 raft_consensus.cc:359] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:40.834115  7853 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:40.835119  7853 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:40.838614  7853 ts_tablet_manager.cc:1434] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.004s
I20260709 12:20:40.840859 32042 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:40.810649  7885 tablet_copy_client.cc:806] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:40.842443  7874 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:40.848304  7889 ts_tablet_manager.cc:933] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:40.857156 32085 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51b3fd343e81436fa5bea82486abcc64. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.862052 31964 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:40.864305  7808 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:40.866356 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-6679c928e92e4f799b88357707650b60 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:20:40.866916 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-6679c928e92e4f799b88357707650b60 on tablet 6679c928e92e4f799b88357707650b60 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:40.873036  7885 tablet_copy_client.cc:670] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:40.908942  7889 tablet_copy_client.cc:323] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:20:40.911125 32656 raft_consensus.cc:1217] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.11->[20.12-20.12]   Dedup: 20.12->[]
I20260709 12:20:40.912554 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 775fd46a41044b1ea06bc796ef1b233b from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:20:40.921532 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 775fd46a41044b1ea06bc796ef1b233b from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-775fd46a41044b1ea06bc796ef1b233b
I20260709 12:20:40.930266  7885 tablet_copy_client.cc:538] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:40.981864  7808 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.984102 32131 tablet_copy_source_session.cc:215] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:40.983942 32042 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:40.988301  7889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775fd46a41044b1ea06bc796ef1b233b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.029978 31964 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:41.044592  7885 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:41.078413  7889 tablet_copy_client.cc:806] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:41.079182  7889 tablet_copy_client.cc:670] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:41.088145  7889 tablet_copy_client.cc:538] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:41.091666 31870 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:41.097054  7889 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:41.252746 31965 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:41.255712  7809 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:41.321055  7885 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.328019  7885 tablet_bootstrap.cc:492] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:41.330833  7885 ts_tablet_manager.cc:1403] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.286s	user 0.051s	sys 0.009s
I20260709 12:20:41.350378  7885 raft_consensus.cc:359] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:41.351459  7885 raft_consensus.cc:740] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:41.371694 32041 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:41.375372  7809 raft_consensus.cc:1064] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:41.352326  7885 consensus_queue.cc:260] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:41.382457  7885 ts_tablet_manager.cc:1434] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.000s
I20260709 12:20:41.384995 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-8f4287254496472398f69d3a10b48d77 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:20:41.385491 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-8f4287254496472398f69d3a10b48d77 on tablet 8f4287254496472398f69d3a10b48d77 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:41.378618  7809 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:41.386526  7808 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:41.513906 32656 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:41.519533  7889 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:20:41.549679 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 105960us
I20260709 12:20:41.567497  7889 tablet_bootstrap.cc:492] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:41.594074  7889 ts_tablet_manager.cc:1403] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.497s	user 0.050s	sys 0.017s
W20260709 12:20:41.606494 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73237us
W20260709 12:20:41.625038 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 113004us
W20260709 12:20:41.598958 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 83439.6us
I20260709 12:20:41.646191  7904 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:41.713575 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 80026.1us
I20260709 12:20:41.743495 32038 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:41.763206  7885 ts_tablet_manager.cc:933] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:20:41.783485 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 151303us
I20260709 12:20:41.799824  7889 raft_consensus.cc:359] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:20:41.832600 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 100312us
I20260709 12:20:41.833036  7829 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:41.823587  7889 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:41.849177  7889 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:41.876958  7889 ts_tablet_manager.cc:1434] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.282s	user 0.008s	sys 0.000s
I20260709 12:20:41.833923  7885 tablet_copy_client.cc:323] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:20:41.922128 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 51b3fd343e81436fa5bea82486abcc64 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:20:41.922995 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 51b3fd343e81436fa5bea82486abcc64 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-51b3fd343e81436fa5bea82486abcc64
I20260709 12:20:41.933044 32130 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-775fd46a41044b1ea06bc796ef1b233b received from {username='slave'} at 127.0.0.1:34592
I20260709 12:20:41.933557 32130 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-775fd46a41044b1ea06bc796ef1b233b on tablet 775fd46a41044b1ea06bc796ef1b233b with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:41.962766 32131 tablet_copy_source_session.cc:215] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:41.968191  7885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b3fd343e81436fa5bea82486abcc64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.982695 31964 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:42.027642 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:42.079466 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 83194.4us
I20260709 12:20:42.071117  7904 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:42.085023 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 84623.4us
I20260709 12:20:42.108367  7911 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.125748 32039 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.147998 31864 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.149367 32112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:42.149986 32112 raft_consensus.cc:606] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:42.154503 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:42.156330 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:42.184505 32653 raft_consensus.cc:1217] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEARNER]: Deduplicated request from leader. Original: 25.14->[25.15-25.15]   Dedup: 25.15->[]
I20260709 12:20:42.186622 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:42.214479 32659 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.239850 31964 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.248751 32656 raft_consensus.cc:1217] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.13->[24.14-24.14]   Dedup: 24.14->[]
I20260709 12:20:42.251328  7885 tablet_copy_client.cc:806] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:42.251977  7885 tablet_copy_client.cc:670] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:42.281982  7885 tablet_copy_client.cc:538] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:42.327335  7885 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:42.397276  7904 raft_consensus.cc:993] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:42.397881  7904 raft_consensus.cc:1081] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:42.420367 32115 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:42.424654  7910 raft_consensus.cc:1064] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:42.427474  7910 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:42.446621 32044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:20:42.447222 32044 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:42.447609 32044 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:42.448150 32659 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:42.451436  7811 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:42.461974 32045 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:42.464999  7809 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.476514 32117 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:20:42.477614 32656 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.478708  7754 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:42.488402 32653 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:20:42.491321  7754 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:42.518483  7930 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:42.561126 31964 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:20:42.570701  7885 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.575765 32039 raft_consensus.cc:2955] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.579155  7754 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 12:20:42.579303  7885 tablet_bootstrap.cc:492] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:42.597577  7885 ts_tablet_manager.cc:1403] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.270s	user 0.046s	sys 0.000s
I20260709 12:20:42.578210 32044 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:42.592008 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6679c928e92e4f799b88357707650b60 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:42.615190 32044 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:42.617682 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:42.618336 32112 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:42.618636 32112 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:42.615336 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:42.619472 31969 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:42.619347 32112 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:42.626288 32112 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:42.623939 31864 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 13 to 14, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 20 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.666875 31969 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:42.694587  7811 raft_consensus.cc:1064] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:42.670188  7885 raft_consensus.cc:359] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:42.698463  7885 raft_consensus.cc:740] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:42.675887  7754 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Committing config change with OpId 25.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.699095  7885 consensus_queue.cc:260] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:42.700404  7811 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 12, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:42.671337 32010 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:42.706494  7920 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:42.707048  7920 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:42.707911  7920 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:42.677866 32653 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.704579 32112 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:42.722605 32656 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:20:42.701543  7885 ts_tablet_manager.cc:1434] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.000s
I20260709 12:20:42.724962  7904 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:20:42.725472 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 54224.8us
I20260709 12:20:42.729799 32039 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:20:42.730176 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-51b3fd343e81436fa5bea82486abcc64 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:20:42.730664 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-51b3fd343e81436fa5bea82486abcc64 on tablet 51b3fd343e81436fa5bea82486abcc64 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:42.742414  7904 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:42.766834 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 6679c928e92e4f799b88357707650b60 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:60388
I20260709 12:20:42.782675 31969 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:20:42.784878  7811 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:42.772321 31964 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.791631  7947 tablet_replica.cc:333] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:20:42.802222  7947 raft_consensus.cc:2243] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:42.820670 31870 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.839955  7947 raft_consensus.cc:2272] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:42.843796 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90384us
I20260709 12:20:42.852836 32112 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.864764 31869 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.895876  7808 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Committing config change with OpId 24.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.897878  7947 ts_tablet_manager.cc:1916] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:42.899719 32044 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.907584 32653 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.946727 31869 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:42.972965 31969 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.002759 32046 consensus_queue.cc:237] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:43.015609 31969 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:43.017243  7909 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:43.017603  7947 ts_tablet_manager.cc:1929] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.13
I20260709 12:20:43.018186  7947 log.cc:1199] T 6679c928e92e4f799b88357707650b60 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/6679c928e92e4f799b88357707650b60
I20260709 12:20:43.020328 32653 raft_consensus.cc:1275] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:43.021821  7909 consensus_queue.cc:1048] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:43.026150 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 6679c928e92e4f799b88357707650b60 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:43.026894  7754 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Committing config change with OpId 25.17: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.034255 32653 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.036831 31964 raft_consensus.cc:2955] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.058734 32112 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:43.060817 31864 catalog_manager.cc:5697] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.072409 32653 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:43.079080 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f4287254496472398f69d3a10b48d77 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:43.090837  7904 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:43.097479 31964 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:43.097654 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f4287254496472398f69d3a10b48d77 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:33516
I20260709 12:20:43.099099  7809 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:43.110886 32653 raft_consensus.cc:1217] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:20:43.109999  7808 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.113646  7952 tablet_replica.cc:333] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:20:43.114775  7952 raft_consensus.cc:2243] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:43.118412 32653 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.127952 31964 raft_consensus.cc:2955] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.152433 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 775fd46a41044b1ea06bc796ef1b233b with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:43.140642  7952 raft_consensus.cc:2272] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:43.164717  7952 ts_tablet_manager.cc:1916] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:43.169425 31869 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.200835 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 775fd46a41044b1ea06bc796ef1b233b with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:37884
I20260709 12:20:43.201469  7952 ts_tablet_manager.cc:1929] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 25.16
I20260709 12:20:43.209322 31965 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:43.210585  7952 log.cc:1199] T 8f4287254496472398f69d3a10b48d77 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/8f4287254496472398f69d3a10b48d77
I20260709 12:20:43.212320  7909 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:43.235324 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 8f4287254496472398f69d3a10b48d77 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:43.276398  7953 tablet_replica.cc:333] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:20:43.277604  7953 raft_consensus.cc:2243] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:43.286994 32112 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:43.290982  7946 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:20:43.287865  7953 raft_consensus.cc:2272] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:43.320564  7953 ts_tablet_manager.cc:1916] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:43.423460 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:43.439848  7953 ts_tablet_manager.cc:1929] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.15
I20260709 12:20:43.446580  7953 log.cc:1199] T 775fd46a41044b1ea06bc796ef1b233b P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/775fd46a41044b1ea06bc796ef1b233b
I20260709 12:20:43.448427 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 775fd46a41044b1ea06bc796ef1b233b (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:43.560226  7930 raft_consensus.cc:1064] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:43.566788  7930 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:43.595046 32046 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:43.597709  7808 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:43.607137 32653 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:43.607692 31965 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:43.608790  7930 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:43.623202  7904 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:43.624352 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:43.625030 32039 raft_consensus.cc:606] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:43.626565 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:43.628350 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:43.631249 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:43.682578 32039 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.685815 32656 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.681432  7809 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.722961 31965 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:43.840026 31864 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:43.877713 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52760us
W20260709 12:20:43.997985 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 210357us
W20260709 12:20:44.037535 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 54213.7us
I20260709 12:20:44.271531  7992 raft_consensus.cc:993] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:44.272156  7992 raft_consensus.cc:1081] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:44.317083 32116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:44.318699 32116 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.319161 32116 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.344529 32116 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:44.346740 32116 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:44.354899 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:44.355517 32044 raft_consensus.cc:3055] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:44.355937 32044 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:44.357224 32116 consensus_queue.cc:237] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:44.356765 32044 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:44.359025 32044 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.355531 31964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:44.360054 31964 raft_consensus.cc:3060] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.363082 32656 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:44.365126  7930 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:44.367537 31969 raft_consensus.cc:1275] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:44.370144  7808 consensus_queue.cc:1048] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:44.392937  7967 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:44.400187 31964 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:44.423471 32084 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:44.424870  7808 raft_consensus.cc:2804] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:44.425537  7808 raft_consensus.cc:697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:44.426549  7808 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:44.427228 32044 raft_consensus.cc:2468] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:44.443176 32656 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:44.463043 31969 raft_consensus.cc:2955] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:44.529865 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51b3fd343e81436fa5bea82486abcc64 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:44.565001 31870 catalog_manager.cc:5697] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 15 to 16, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:44.586148 31864 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 26, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.646541 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 51b3fd343e81436fa5bea82486abcc64 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:37884
I20260709 12:20:44.678604  7998 tablet_replica.cc:333] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:20:44.679944  7998 raft_consensus.cc:2243] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:44.680955  7998 raft_consensus.cc:2272] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:44.685010  7998 ts_tablet_manager.cc:1916] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:44.708137  7998 ts_tablet_manager.cc:1929] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 12:20:44.708786  7998 log.cc:1199] T 51b3fd343e81436fa5bea82486abcc64 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/51b3fd343e81436fa5bea82486abcc64
I20260709 12:20:44.711910 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 51b3fd343e81436fa5bea82486abcc64 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:44.840174 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:44.840884 32046 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:44.847393  7808 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:44.965265 31969 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:44.968770  7808 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:45.078636 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 108310us
I20260709 12:20:45.162439 32118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:45.163076 32118 raft_consensus.cc:606] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:45.170516 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:45.172649 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:45.179138 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:45.597307  8002 raft_consensus.cc:993] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:20:45.597862  8002 raft_consensus.cc:1081] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Signalling peer f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:20:45.605350 31968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:56072
I20260709 12:20:45.645561 31968 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:45.649704 31968 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:45.728344 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50218.9us
W20260709 12:20:45.840777 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 92767.3us
I20260709 12:20:45.933431 31968 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:46.047674 31968 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:46.049316 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:46.049985 32653 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:46.057613 32653 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:46.066555 31936 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:46.067670  7974 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:46.068310  7974 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:46.069353  7974 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:46.094380 32659 raft_consensus.cc:1240] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 24. Current term is 25. Ops: []
W20260709 12:20:46.098507 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 314807us
W20260709 12:20:46.099159 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 157000us
I20260709 12:20:46.099418 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:46.100033 32112 raft_consensus.cc:3055] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:46.100399 32112 raft_consensus.cc:740] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:46.101964  8005 consensus_queue.cc:1059] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:46.152340 32112 consensus_queue.cc:260] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:46.154130 32112 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:46.215971 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 107450us
W20260709 12:20:46.223435 32085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:46.190300 32112 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:46.211153 32046 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:20:46.262207 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 160528us
I20260709 12:20:46.284495 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:46.293332 31864 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 25 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:46.328644 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 73475.5us
W20260709 12:20:46.350832 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 94627.7us
W20260709 12:20:46.402354 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 57540.6us
I20260709 12:20:46.406301 32112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:46.406937 32112 raft_consensus.cc:606] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:46.432819 32118 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:46.447618  8008 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:46.494592 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:46.496847 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:20:46.545461 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51837.5us
I20260709 12:20:46.551108 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:46.558646 32011 consensus_peers.cc:597] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 193a74284231469f9229c961dfaff799. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:46.577986 31969 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:20:46.584411 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 160591us
I20260709 12:20:46.595360  7993 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:46.605470  8030 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.610693 32112 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.613749 32654 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:46.615751  8017 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:46.684062 32118 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:46.689719  8021 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:46.700724 31964 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.712870 32046 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:46.711874 31870 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.771865 32115 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:20:46.777057 32011 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19357e2a1cf1408b9391be0c12c3b0e8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:46.778388 31965 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:46.780293  8006 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:46.782464  8006 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:46.822227  7993 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.836786 31965 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.899145 31870 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.897182 32115 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:46.932070 32115 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:46.978042 32045 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
W20260709 12:20:46.981600 32085 consensus_peers.cc:597] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c50b710409be4b8986d1a0b839db7004. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:46.982417 31965 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:20:46.984200  8018 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:46.985586  8024 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:20:46.990967  7930 raft_consensus.cc:993] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:20:46.991493  8018 raft_consensus.cc:1081] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Signalling peer f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:20:46.995709 31965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:56072
I20260709 12:20:46.996271 31965 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:46.996603 31965 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:47.003975 31965 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:47.007601 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:47.008296 32046 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:47.030015 32046 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 28.
I20260709 12:20:47.031733 31969 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Committing config change with OpId 31.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.032029 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:47.033326 32115 raft_consensus.cc:3055] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:47.033694 32115 raft_consensus.cc:740] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:47.034631 32115 consensus_queue.cc:260] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:47.036334 32115 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:47.043675 32115 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 28.
I20260709 12:20:47.029896  7930 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Committing config change with OpId 31.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.044925 31935 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:47.047044  7974 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:47.047540  7974 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 28 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:47.048395  7974 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:47.046032 32045 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Committing config change with OpId 31.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.032486 31965 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:47.084463  8049 ts_tablet_manager.cc:933] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:47.116271 32046 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:47.117304 31870 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 31 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.149870 31866 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 27 to 28, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 28 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.153587  8049 tablet_copy_client.cc:323] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:20:47.156934 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 193a74284231469f9229c961dfaff799 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:20:47.157490 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 193a74284231469f9229c961dfaff799 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-193a74284231469f9229c961dfaff799
I20260709 12:20:47.197461 32057 tablet_copy_source_session.cc:215] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:47.204087  8049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 193a74284231469f9229c961dfaff799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:47.210469 31969 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:20:47.213596 32011 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 478b63c892a745d09e9508bc9cc5321f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:47.219000 32116 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:47.221952  8030 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:47.224880  8030 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:47.280177  8052 ts_tablet_manager.cc:933] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:47.286769  8030 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.288625 32116 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.313089 31969 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.315763  8049 tablet_copy_client.cc:806] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:47.316399  8049 tablet_copy_client.cc:670] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:47.318732 31870 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:47.321447  8049 tablet_copy_client.cc:538] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:47.346282  8049 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:47.362864  8052 tablet_copy_client.cc:323] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:20:47.371723 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 19357e2a1cf1408b9391be0c12c3b0e8 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:20:47.372318 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 19357e2a1cf1408b9391be0c12c3b0e8 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-19357e2a1cf1408b9391be0c12c3b0e8
I20260709 12:20:47.385715 32057 tablet_copy_source_session.cc:215] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:47.399118  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19357e2a1cf1408b9391be0c12c3b0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:47.585004 32045 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:20:47.595551  8013 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:47.634223  8058 ts_tablet_manager.cc:933] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:47.642843  8052 tablet_copy_client.cc:806] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:47.657454  8052 tablet_copy_client.cc:670] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:47.661885  8052 tablet_copy_client.cc:538] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:47.666953 32116 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:20:47.674108  8013 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:47.703155  8058 tablet_copy_client.cc:323] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:20:47.740015  8049 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=1 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:47.741103  8049 tablet_bootstrap.cc:492] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:47.741875  8049 ts_tablet_manager.cc:1403] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.396s	user 0.046s	sys 0.007s
I20260709 12:20:47.745040  8049 raft_consensus.cc:359] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:47.751477 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet c50b710409be4b8986d1a0b839db7004 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:20:47.752106 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet c50b710409be4b8986d1a0b839db7004 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-c50b710409be4b8986d1a0b839db7004
I20260709 12:20:47.760262  8063 ts_tablet_manager.cc:933] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:20:47.803051 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45135.9us
I20260709 12:20:47.754617  8049 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:47.821885 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:47.862403  8049 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:47.870879  8049 ts_tablet_manager.cc:1434] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.129s	user 0.007s	sys 0.000s
I20260709 12:20:47.873072  8052 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
W20260709 12:20:47.960330 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 139053us
I20260709 12:20:47.970795 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-193a74284231469f9229c961dfaff799 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:20:47.971292 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-193a74284231469f9229c961dfaff799 on tablet 193a74284231469f9229c961dfaff799 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:47.994758  8063 tablet_copy_client.cc:323] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:20:48.000463 32131 tablet_copy_source_session.cc:215] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:20:48.039899 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 53316.2us
I20260709 12:20:48.067265 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:48.067921 31969 raft_consensus.cc:606] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:48.071357 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:48.071923 31969 raft_consensus.cc:606] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:48.073987 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:48.087683 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:48.088559 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:48.105955 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68166.1us
I20260709 12:20:48.107352  7974 raft_consensus.cc:993] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:48.107884  7974 raft_consensus.cc:1081] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:48.115044 32116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47766
I20260709 12:20:48.115610 32116 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.115978 32116 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:48.171335 32116 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:48.192356  8058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50b710409be4b8986d1a0b839db7004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:48.206451 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:48.207218 31969 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:48.207515 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:48.210301 32044 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:48.213888  8076 raft_consensus.cc:993] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:48.214438  8076 raft_consensus.cc:1081] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:48.216073 32112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47766
I20260709 12:20:48.216576 32112 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:48.216899 32112 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:48.207566 32116 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:48.231297 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 478b63c892a745d09e9508bc9cc5321f from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:20:48.231889 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 478b63c892a745d09e9508bc9cc5321f from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-478b63c892a745d09e9508bc9cc5321f
I20260709 12:20:48.231293 31969 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:48.235320 31969 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:48.237525 31969 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:48.247066 32654 raft_consensus.cc:1217] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.15->[27.16-27.16]   Dedup: 27.16->[]
I20260709 12:20:48.274071  8058 tablet_copy_client.cc:806] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:48.274760  8058 tablet_copy_client.cc:670] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:48.276615 32057 tablet_copy_source_session.cc:215] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:48.276376 32112 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:48.251434 32044 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:20:48.279814  8063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478b63c892a745d09e9508bc9cc5321f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:48.283381 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:48.284044 31965 raft_consensus.cc:3055] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:48.284339 31965 raft_consensus.cc:740] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:48.285070 31965 consensus_queue.cc:260] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:48.286710 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:48.288740  8081 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:48.289245  8081 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:48.290165  8081 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:48.286212 31965 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:48.301118  8058 tablet_copy_client.cc:538] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:48.306744 32112 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:48.307773 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:48.308395 32045 raft_consensus.cc:3060] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:48.311497  8058 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:48.316957 32045 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:48.318397 32084 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:48.318586 31969 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:20:48.319358  8024 raft_consensus.cc:2804] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:48.319847  8024 raft_consensus.cc:697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:48.320711  8024 consensus_queue.cc:237] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:48.337078 31965 raft_consensus.cc:2468] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:48.363130 31866 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 26 to 27, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 27 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.379868  8063 tablet_copy_client.cc:806] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:48.380530  8063 tablet_copy_client.cc:670] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:48.406355  8063 tablet_copy_client.cc:538] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:48.461205  8063 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:48.502416  8052 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:48.503556  8052 tablet_bootstrap.cc:492] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:48.504437  8052 ts_tablet_manager.cc:1403] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.632s	user 0.044s	sys 0.022s
I20260709 12:20:48.537640  8052 raft_consensus.cc:359] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:48.540112  8052 raft_consensus.cc:740] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:48.540830  8052 consensus_queue.cc:260] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:48.606601  8058 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=3 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:48.607671  8058 tablet_bootstrap.cc:492] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:48.608438  8058 ts_tablet_manager.cc:1403] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.297s	user 0.050s	sys 0.018s
I20260709 12:20:48.633075  8058 raft_consensus.cc:359] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:48.633983  8058 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:48.643218  8052 ts_tablet_manager.cc:1434] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.138s	user 0.005s	sys 0.003s
I20260709 12:20:48.645249 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-19357e2a1cf1408b9391be0c12c3b0e8 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:20:48.645686 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-19357e2a1cf1408b9391be0c12c3b0e8 on tablet 19357e2a1cf1408b9391be0c12c3b0e8 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:48.662974 32654 raft_consensus.cc:1217] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
I20260709 12:20:48.671016  8058 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 31.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:48.672186  8012 raft_consensus.cc:1064] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:48.674270  8058 ts_tablet_manager.cc:1434] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.062s	user 0.002s	sys 0.005s
I20260709 12:20:48.680151 32131 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-c50b710409be4b8986d1a0b839db7004 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:20:48.680666 32131 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-c50b710409be4b8986d1a0b839db7004 on tablet c50b710409be4b8986d1a0b839db7004 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:48.680470  8012 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 14, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:48.687439 31866 catalog_manager.cc:5697] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 26, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.742834 32116 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:48.745430  8084 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:48.771694 32044 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:20:48.773622  8024 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:48.823776  8063 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:48.831146  8063 tablet_bootstrap.cc:492] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:48.838970  8063 ts_tablet_manager.cc:1403] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.378s	user 0.039s	sys 0.026s
I20260709 12:20:48.852897  8063 raft_consensus.cc:359] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:48.882926 32659 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:48.884038 31969 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:48.885591 32046 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:48.888974  7993 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:48.892251  7993 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:48.859684  8063 raft_consensus.cc:740] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:48.909336 31969 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:48.910579  8063 consensus_queue.cc:260] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:48.904985  8024 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:48.923430  8063 ts_tablet_manager.cc:1434] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.001s
I20260709 12:20:48.934391 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-478b63c892a745d09e9508bc9cc5321f received from {username='slave'} at 127.0.0.1:54754
I20260709 12:20:48.934830 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-478b63c892a745d09e9508bc9cc5321f on tablet 478b63c892a745d09e9508bc9cc5321f with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:48.957768  8018 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:20:48.975167 31969 raft_consensus.cc:1275] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:20:48.978881  7930 consensus_queue.cc:1048] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:20:49.018167 31969 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:49.034425  8100 log.cc:927] Time spent T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:20:49.054720 32654 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.070696  8084 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.100997 31866 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.109594 32116 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.111788 32654 raft_consensus.cc:1217] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 LEARNER]: Deduplicated request from leader. Original: 31.15->[31.16-31.16]   Dedup: 31.16->[]
I20260709 12:20:49.165326 32045 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.209084 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:49.236399  7993 raft_consensus.cc:1064] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:49.239728 32112 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:49.241515  8084 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:49.245467  7993 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:20:49.296763 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 87919.5us
I20260709 12:20:49.327092 32654 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:49.351143  8009 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:49.339339  8082 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.341230 32116 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.389420 32654 raft_consensus.cc:2955] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.409899 32659 raft_consensus.cc:1217] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:20:49.428751 31869 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.460400 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 193a74284231469f9229c961dfaff799 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:49.462842 32659 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:49.464351  8009 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:49.473452 32115 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:49.475260  8103 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:49.510440  8018 raft_consensus.cc:1064] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:49.513346  8018 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.16, Last appended by leader: 14, Current term: 31, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.530347 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 193a74284231469f9229c961dfaff799 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:60388
I20260709 12:20:49.536290 31969 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:49.544572  8084 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:49.603466  8125 tablet_replica.cc:333] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:20:49.604884  8125 raft_consensus.cc:2243] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:49.611433 32116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:49.612110 32116 raft_consensus.cc:606] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:49.614723  8125 raft_consensus.cc:2272] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:49.629635 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:49.633416 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:49.655565 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:49.653127  8082 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.667799  8125 ts_tablet_manager.cc:1916] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:49.656083 32112 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.684954  8108 raft_consensus.cc:993] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:20:49.685463  8108 raft_consensus.cc:1081] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Signalling peer f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:20:49.688511 31964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:56072
I20260709 12:20:49.689096 31964 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:49.689456 31964 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:49.696096 31964 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.698177 31964 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:49.702893 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:49.703475 32116 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:49.703779 32116 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:49.704593 32116 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.705932 32116 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:49.710346 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:49.710980 32654 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:49.717056 32046 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:20:49.719118  8108 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:49.723143 32659 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:20:49.724958  8057 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:49.737517 31964 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:20:49.739120  8125 ts_tablet_manager.cc:1929] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.17
I20260709 12:20:49.739778  8125 log.cc:1199] T 193a74284231469f9229c961dfaff799 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/193a74284231469f9229c961dfaff799
I20260709 12:20:49.741513  8108 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:49.742403 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 193a74284231469f9229c961dfaff799 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:49.752481 32653 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.757604 31866 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:49.775843 31964 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.798488  8081 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Committing config change with OpId 31.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.807528 32659 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Committing config change with OpId 31.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.844249 32654 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:49.848016 32116 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 25.
I20260709 12:20:49.859493 32045 consensus_queue.cc:237] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.860433 31938 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:49.861675 31968 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Committing config change with OpId 31.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.879719 32654 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:49.872128  8120 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:49.902976  8120 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:49.904013  8120 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.908595  8009 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:49.919296  8082 raft_consensus.cc:1064] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:49.929836 32045 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Committing config change with OpId 31.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.943449  8084 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.951273 32659 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.955766 31968 raft_consensus.cc:1275] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:49.957391 31870 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 31 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:49.958259  8074 consensus_queue.cc:1048] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:49.922043  8082 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:49.966359 31968 raft_consensus.cc:2955] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.018716 31869 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 25 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.036046 32116 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:50.046627 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 19357e2a1cf1408b9391be0c12c3b0e8 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:50.051182 32654 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:20:50.053526 32045 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:20:50.053570  8108 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:20:50.063144  8108 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:50.066826  7930 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Committing config change with OpId 31.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.070261 32659 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Committing config change with OpId 31.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.080238 31869 catalog_manager.cc:5697] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 16 to 17, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 23 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.086158 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c50b710409be4b8986d1a0b839db7004 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:20:50.088176 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 56333.3us
I20260709 12:20:50.113726 32045 raft_consensus.cc:2955] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Committing config change with OpId 31.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.155504 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 19357e2a1cf1408b9391be0c12c3b0e8 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:33516
I20260709 12:20:50.157943 31866 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 31 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.204027  8139 tablet_replica.cc:333] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:20:50.205847  8139 raft_consensus.cc:2243] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:50.208092  8139 raft_consensus.cc:2272] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:50.228010  8139 ts_tablet_manager.cc:1916] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:50.247397 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet c50b710409be4b8986d1a0b839db7004 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:60388
I20260709 12:20:50.242172 31965 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:50.270818 32112 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:50.275369  8134 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:50.286652  8139 ts_tablet_manager.cc:1929] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.16
I20260709 12:20:50.292336  8139 log.cc:1199] T 19357e2a1cf1408b9391be0c12c3b0e8 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/19357e2a1cf1408b9391be0c12c3b0e8
I20260709 12:20:50.294162 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 19357e2a1cf1408b9391be0c12c3b0e8 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:50.295398  8082 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:50.310920 32654 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:50.314837 32119 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:50.321291  8082 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:50.328135  8119 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:50.347055  8141 tablet_replica.cc:333] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:20:50.361543  8141 raft_consensus.cc:2243] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:50.362499  8141 raft_consensus.cc:2272] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:50.424005  8141 ts_tablet_manager.cc:1916] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:50.435662 32654 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:50.439198  8119 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:50.438469  7993 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.443806 32654 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.483103 32119 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.483817 31969 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.542461  8141 ts_tablet_manager.cc:1929] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 31.17
I20260709 12:20:50.543078  8141 log.cc:1199] T c50b710409be4b8986d1a0b839db7004 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/c50b710409be4b8986d1a0b839db7004
I20260709 12:20:50.583753 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet c50b710409be4b8986d1a0b839db7004 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:50.589208 31866 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:50.875094 32044 consensus_queue.cc:237] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:50.903880 32659 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:50.911126  8134 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:50.915320 31965 raft_consensus.cc:1275] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:50.924525  7993 consensus_queue.cc:1048] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:50.939285  8074 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.943213 31969 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:50.994874 32659 raft_consensus.cc:2955] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:51.061040 31870 catalog_manager.cc:5697] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:51.065032 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 478b63c892a745d09e9508bc9cc5321f with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:20:51.066007 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 104615us
I20260709 12:20:51.077826 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 478b63c892a745d09e9508bc9cc5321f with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:33516
I20260709 12:20:51.082617 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.157985  8157 tablet_replica.cc:333] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:20:51.160090  8157 raft_consensus.cc:2243] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:51.171123  8157 raft_consensus.cc:2272] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:51.213527  8157 ts_tablet_manager.cc:1916] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:51.272856 31965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:20:51.273648 31965 raft_consensus.cc:606] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:51.294513 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:51.296466 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:51.299374 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.320982  8157 ts_tablet_manager.cc:1929] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 12:20:51.321771  8157 log.cc:1199] T 478b63c892a745d09e9508bc9cc5321f P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/478b63c892a745d09e9508bc9cc5321f
I20260709 12:20:51.324972 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 478b63c892a745d09e9508bc9cc5321f (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:51.485358  8152 raft_consensus.cc:993] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:51.487250  8152 raft_consensus.cc:1081] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:51.489538 32119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47766
I20260709 12:20:51.490128 32119 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.490463 32119 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:51.514832 32119 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:51.528548 32119 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:51.551677 31965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:51.552402 31965 raft_consensus.cc:3055] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:51.552722 31965 raft_consensus.cc:740] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:20:51.557966 31965 consensus_queue.cc:260] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:51.564870 31965 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:51.572306 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:51.572912 32656 raft_consensus.cc:3060] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:51.603108 32656 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:51.604642 31965 raft_consensus.cc:2468] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:51.604921 32085 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:51.606775  8106 raft_consensus.cc:2804] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:51.607290  8106 raft_consensus.cc:697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:51.608424  8106 consensus_queue.cc:237] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:51.634615 31864 catalog_manager.cc:5697] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 26, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.023536 32656 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:20:52.026477  8161 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:20:52.100492 31965 raft_consensus.cc:1275] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:20:52.110570  8106 consensus_queue.cc:1048] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:20:52.415381 32119 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:52.481966 32046 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:52.483741  8106 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:20:52.486125 32085 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81129181ae3246c1bfb02954120c9b91. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.493953 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.498198 31965 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:52.500070  8106 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:52.512995  8163 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.514636 32044 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.538736 31965 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.558468 31864 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.569335 32119 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:52.628719 32044 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:52.631579 31965 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:20:52.633401 32085 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 786db0eb7e9b431ba90c5ad880c46c9e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.634186  8106 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:52.635706  8163 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:52.708108  8018 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Committing config change with OpId 23.12: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.709932 32044 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Committing config change with OpId 23.12: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.711277 32119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:52.711864 32119 raft_consensus.cc:606] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:52.730566 32046 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:52.730656 31968 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Committing config change with OpId 23.12: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.743983 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:52.748580 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:52.751439 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.752094 32119 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:52.753692  8134 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:52.745064 31870 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 12, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:52.764580 32011 consensus_peers.cc:597] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08b3d325c1944e45a59e666e91d0dbb6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.772879  8082 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.777890 31965 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:52.779140 32118 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.786582  8134 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:52.831697 31864 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.869015 31969 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.928066 31969 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:52.942570 32118 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:20:52.943492 31936 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58ee45b18b714479af8291d9056afb9e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.944864  8083 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:52.949697 32046 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:52.951683  8083 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:52.972154  8083 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Committing config change with OpId 27.17: config changed from index -1 to 17, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.973181 32119 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index -1 to 17, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:52.996109 32046 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index -1 to 17, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:53.045200 31864 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 17, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:53.099241  8188 ts_tablet_manager.cc:933] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:53.127949  8190 ts_tablet_manager.cc:933] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:53.159523  8188 tablet_copy_client.cc:323] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:20:53.200969  8190 tablet_copy_client.cc:323] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:20:53.215744 32131 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 81129181ae3246c1bfb02954120c9b91 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:20:53.216454 32131 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 81129181ae3246c1bfb02954120c9b91 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-81129181ae3246c1bfb02954120c9b91
I20260709 12:20:53.220996 32130 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 786db0eb7e9b431ba90c5ad880c46c9e from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:20:53.221616 32130 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 786db0eb7e9b431ba90c5ad880c46c9e from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-786db0eb7e9b431ba90c5ad880c46c9e
W20260709 12:20:53.235110 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45222us
I20260709 12:20:53.240682  8163 raft_consensus.cc:993] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:53.241204  8163 raft_consensus.cc:1081] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:53.244534 32044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:20:53.245257 32044 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:53.245757 32044 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:53.249460 32131 tablet_copy_source_session.cc:215] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:53.259528 32130 tablet_copy_source_session.cc:215] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:53.262100 32044 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 32 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:53.266222  8188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81129181ae3246c1bfb02954120c9b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:53.288674 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:53.289574 32118 raft_consensus.cc:3055] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:20:53.289949 32118 raft_consensus.cc:740] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:53.292487 32044 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 32 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:53.299463 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:53.300161 32656 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:53.307511 32118 consensus_queue.cc:260] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:53.298103  8190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 786db0eb7e9b431ba90c5ad880c46c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:53.328147  8198 ts_tablet_manager.cc:933] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:53.334355 32118 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:53.352412 32656 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 32.
I20260709 12:20:53.397830 32118 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 32.
I20260709 12:20:53.405458 32013 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:53.406857  8082 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:20:53.407377  8082 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 32 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:53.408367  8082 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:53.442574  8198 tablet_copy_client.cc:323] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:20:53.493332  8205 ts_tablet_manager.cc:933] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:20:53.482497  8190 tablet_copy_client.cc:806] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:53.496419  8190 tablet_copy_client.cc:670] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:53.504236  8190 tablet_copy_client.cc:538] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:20:53.524820 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 70557.7us
I20260709 12:20:53.540434 31869 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 31 to 32, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 32 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:53.542277  8205 tablet_copy_client.cc:323] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:20:53.543963  8188 tablet_copy_client.cc:806] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:53.544740  8188 tablet_copy_client.cc:670] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:53.544742 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 58ee45b18b714479af8291d9056afb9e from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:20:53.546643 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 58ee45b18b714479af8291d9056afb9e from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-58ee45b18b714479af8291d9056afb9e
I20260709 12:20:53.558611  8190 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:53.559984  8188 tablet_copy_client.cc:538] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:53.602826 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 08b3d325c1944e45a59e666e91d0dbb6 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:20:53.603528 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 08b3d325c1944e45a59e666e91d0dbb6 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-08b3d325c1944e45a59e666e91d0dbb6
I20260709 12:20:53.604571 31981 tablet_copy_source_session.cc:215] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:53.629791  8205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ee45b18b714479af8291d9056afb9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:53.630406 32057 tablet_copy_source_session.cc:215] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:53.661152  8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b3d325c1944e45a59e666e91d0dbb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:53.658874  8188 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:53.707190  8205 tablet_copy_client.cc:806] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:53.707939  8205 tablet_copy_client.cc:670] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:53.738456  8205 tablet_copy_client.cc:538] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:53.785629  8205 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:53.786010  8198 tablet_copy_client.cc:806] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:53.791311  8198 tablet_copy_client.cc:670] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:53.808532  8198 tablet_copy_client.cc:538] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:53.816759  8190 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=3 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:53.819679  8190 tablet_bootstrap.cc:492] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:53.831137  8190 ts_tablet_manager.cc:1403] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.281s	user 0.045s	sys 0.004s
I20260709 12:20:53.839509  8198 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:20:53.842038  8190 raft_consensus.cc:359] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:53.846577  8190 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:53.847479  8190 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:53.863109 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.890913  8190 ts_tablet_manager.cc:1434] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.004s
I20260709 12:20:53.911814 32130 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-786db0eb7e9b431ba90c5ad880c46c9e received from {username='slave'} at 127.0.0.1:34592
I20260709 12:20:53.912334 32130 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-786db0eb7e9b431ba90c5ad880c46c9e on tablet 786db0eb7e9b431ba90c5ad880c46c9e with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:53.921471 32112 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:20:53.923030  8180 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:54.010138 32659 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:20:54.042810 32039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:54.044322 32039 raft_consensus.cc:606] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:54.029023  8206 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 12:20:54.052126  8188 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.055053 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:54.055920 32659 raft_consensus.cc:1217] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.11->[23.12-23.12]   Dedup: 23.12->[]
I20260709 12:20:54.069068 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:54.070155 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:54.078974  8188 tablet_bootstrap.cc:492] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:54.079880  8188 ts_tablet_manager.cc:1403] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.454s	user 0.040s	sys 0.016s
W20260709 12:20:54.172088 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 65632.5us
I20260709 12:20:54.178584  8188 raft_consensus.cc:359] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:54.179630  8188 raft_consensus.cc:740] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:54.180406  8188 consensus_queue.cc:260] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:54.259150  8188 ts_tablet_manager.cc:1434] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.179s	user 0.009s	sys 0.000s
I20260709 12:20:54.261477 32130 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-81129181ae3246c1bfb02954120c9b91 received from {username='slave'} at 127.0.0.1:34592
I20260709 12:20:54.261950 32130 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-81129181ae3246c1bfb02954120c9b91 on tablet 81129181ae3246c1bfb02954120c9b91 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:54.308437  8206 raft_consensus.cc:993] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:54.308954  8206 raft_consensus.cc:1081] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:54.310778 32115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "193a74284231469f9229c961dfaff799"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:54.311360 32115 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:54.311698 32115 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:54.319577 32115 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:54.324038 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:54.324290 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:54.328523 32045 raft_consensus.cc:3055] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:54.328625 32659 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:54.328955 32045 raft_consensus.cc:740] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:54.329892 32045 consensus_queue.cc:260] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:54.331125 32045 raft_consensus.cc:3060] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:20:54.331485 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 201883us
I20260709 12:20:54.334429 32115 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:54.342677 32659 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:20:54.344002 32085 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:54.345681  8196 raft_consensus.cc:2804] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:54.346184  8196 raft_consensus.cc:697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:54.347114  8196 consensus_queue.cc:237] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:54.379590 32045 raft_consensus.cc:2468] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:20:54.421007  8205 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.422010  8205 tablet_bootstrap.cc:492] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:54.422763  8205 ts_tablet_manager.cc:1403] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.637s	user 0.055s	sys 0.019s
I20260709 12:20:54.425992  8205 raft_consensus.cc:359] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:54.432778  8205 raft_consensus.cc:740] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:54.433554  8205 consensus_queue.cc:260] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:54.436496  8205 ts_tablet_manager.cc:1434] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 12:20:54.449527 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-58ee45b18b714479af8291d9056afb9e received from {username='slave'} at 127.0.0.1:58058
I20260709 12:20:54.450023 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-58ee45b18b714479af8291d9056afb9e on tablet 58ee45b18b714479af8291d9056afb9e with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:54.446137 31869 catalog_manager.cc:5697] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 27 to 28, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 28 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:54.516264 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 52508.6us
I20260709 12:20:54.516800 32659 raft_consensus.cc:1217] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.12->[22.13-22.13]   Dedup: 22.13->[]
I20260709 12:20:54.540493  8198 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.549034 32659 raft_consensus.cc:1217] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.16->[27.17-27.17]   Dedup: 27.17->[]
I20260709 12:20:54.559327  8198 tablet_bootstrap.cc:492] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:20:54.563458  8198 ts_tablet_manager.cc:1403] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.724s	user 0.054s	sys 0.020s
I20260709 12:20:54.578827  8198 raft_consensus.cc:359] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:54.580014  8198 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:20:54.580821  8198 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:54.583623  8198 ts_tablet_manager.cc:1434] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:20:54.585497 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-08b3d325c1944e45a59e666e91d0dbb6 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:20:54.585922 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-08b3d325c1944e45a59e666e91d0dbb6 on tablet 08b3d325c1944e45a59e666e91d0dbb6 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:54.616150  8196 raft_consensus.cc:1064] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
W20260709 12:20:54.620095 32084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:54.630966  8196 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 10, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:54.670411 32042 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:54.671849 32659 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:54.672340  8018 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:54.675577  8018 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:54.691681 31968 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:54.694165  8196 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:20:54.719376 32654 raft_consensus.cc:1217] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.15->[27.16-27.16]   Dedup: 27.16->[]
I20260709 12:20:54.746894  8195 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Committing config change with OpId 23.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:54.777518 31965 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:54.803999  8233 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:20:54.820641 31864 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:54.858765 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 76785.1us
I20260709 12:20:54.871390 32659 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:54.873924 32659 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:54.887022  8195 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:54.892961 32042 raft_consensus.cc:1275] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:54.901294 32042 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index 12 to 13, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:54.915654  8161 consensus_queue.cc:1048] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:55.009124 32119 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:55.018487 32659 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:55.025929 31965 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:55.032052  8197 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:55.036115  8161 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:55.077067  8237 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index 13 to 14, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:55.083184 32654 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:55.118073  8195 raft_consensus.cc:1064] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:55.120755  8195 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:55.156002 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 786db0eb7e9b431ba90c5ad880c46c9e with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:55.156338  8219 raft_consensus.cc:1064] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:55.160113  8219 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 15, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:55.161195 31965 raft_consensus.cc:2955] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:55.168785 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:55.191771 31864 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 13 to 14, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:55.214469 32659 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:55.217710  8161 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:55.230978 32042 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:55.236251 31969 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:55.254738  8018 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:55.261590 32654 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:20:55.271088 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 109888us
W20260709 12:20:55.304742 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 60023.7us
W20260709 12:20:55.294725 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 131919us
I20260709 12:20:55.265182  8197 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:20:55.328724  8200 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
W20260709 12:20:55.501067 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 192427us
I20260709 12:20:55.554876 32119 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:55.589357 32119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:55.589962 32119 raft_consensus.cc:606] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:20:55.602977 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 49325.3us
I20260709 12:20:55.638720 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:55.640249 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:55.640900 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:55.703387  8091 log.cc:927] Time spent T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:20:55.789153 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 160281us
W20260709 12:20:55.801996 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 166025us
I20260709 12:20:55.772007  8227 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.059s
W20260709 12:20:55.871650 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 81581.8us
W20260709 12:20:55.891542 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 407252us
I20260709 12:20:56.098507 31859 maintenance_manager.cc:419] P a0887d9cdc8044caa86af7acdb454104: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033396
W20260709 12:20:56.108451 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50080.4us
I20260709 12:20:56.126931 32115 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:56.153851 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56727.2us
I20260709 12:20:56.156705 32042 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:20:56.243789 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 181685us
W20260709 12:20:56.251709 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56245.6us
I20260709 12:20:56.213733 31864 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41188 (request call id 219) took 1075 ms. Trace:
I20260709 12:20:56.402913 31870 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55370 (request call id 335) took 1179 ms. Trace:
W20260709 12:20:56.408720 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 155374us
W20260709 12:20:56.410549 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 251889us
I20260709 12:20:56.403378 31870 rpcz_store.cc:276] 0709 12:20:55.223100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:55.223264 (+   164us) service_pool.cc:224] Handling call
0709 12:20:56.402853 (+1179589us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1168,"reported_tablets":2}
I20260709 12:20:56.417810  8203 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:56.418478  8203 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:56.160423  8281 raft_consensus.cc:1064] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:20:56.167073  8197 raft_consensus.cc:993] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:56.443377  8018 raft_consensus.cc:1081] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:56.465020 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 786db0eb7e9b431ba90c5ad880c46c9e with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:37884
W20260709 12:20:56.527287 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 68314.2us
W20260709 12:20:56.529633 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 117440us
I20260709 12:20:56.544350  8203 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:56.547226  8266 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:20:56.401926 31864 rpcz_store.cc:276] 0709 12:20:55.137893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:55.139089 (+  1196us) service_pool.cc:224] Handling call
0709 12:20:55.176285 (+ 37196us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b5400002490. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f2288f6adb1 00007f22890aab71 00007f22890ab0b2 00007f2292024759 00007f2291fbca76 00007f2291f8dd58 00007f229208c04e 00007f229054d2a4 00007f229054d232 00007f229054d161 00007f229054d0dc
0709 12:20:56.213637 (+1037352us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:55.239288 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:55.343477 (+104189us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:55.345466 (+  1989us) write_op.cc:435] Acquired schema lock
0709 12:20:55.345485 (+    19us) tablet.cc:662] Decoding operations
0709 12:20:55.407241 (+ 61756us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:55.464547 (+ 57306us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:55.470528 (+  5981us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:55.484731 (+ 14203us) tablet.cc:701] Row locks acquired
0709 12:20:55.484749 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:20:55.510483 (+ 25734us) write_op.cc:270] Start()
0709 12:20:55.521099 (+ 10616us) write_op.cc:276] Timestamp: P: 1783599655501567 usec, L: 0
0709 12:20:55.521139 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:20:55.617381 (+ 96242us) log.cc:844] Serialized 530 byte log entry
0709 12:20:56.098400 (+481019us) op_driver.cc:464] REPLICATION: finished
0709 12:20:56.160601 (+ 62201us) write_op.cc:301] APPLY: starting
0709 12:20:56.160685 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:56.209755 (+ 49070us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:56.209891 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:56.210960 (+  1069us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:56.211155 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=4,mrs_lookups=0
0709 12:20:56.211203 (+    48us) write_op.cc:312] APPLY: finished
0709 12:20:56.211400 (+   197us) log.cc:844] Serialized 25 byte log entry
0709 12:20:56.211582 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:56.211671 (+    89us) write_op.cc:454] Released schema lock
0709 12:20:56.211719 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:56.212145 (+   426us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:56.212181 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13374,"reported_tablets":1,"spinlock_wait_cycles":82863488,"child_traces":[["op",{"apply.queue_time_us":61899,"apply.run_cpu_time_us":3780,"apply.run_wall_time_us":51546,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8303,"num_ops":1,"prepare.queue_time_us":5857,"prepare.run_cpu_time_us":4558,"prepare.run_wall_time_us":394557,"replication_time_us":527196,"thread_start_us":46501,"threads_started":1}]]}
I20260709 12:20:56.630642 32651 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:56.573993  8287 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:20:56.643834  8287 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:56.548588  8266 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:56.651791  8281 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 14, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:56.249255  8277 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 17 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:56.658630  8287 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:56.672263  8266 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:20:56.701267 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 144462us
W20260709 12:20:56.701730 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 256959us
W20260709 12:20:56.703688 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 240465us
I20260709 12:20:56.720389  8306 tablet_replica.cc:333] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:20:56.700636 31858 maintenance_manager.cc:643] P a0887d9cdc8044caa86af7acdb454104: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.601s	user 0.055s	sys 0.027s Metrics: {"bytes_written":8569,"delete_count":0,"lbm_write_time_us":122977,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":6,"reinsert_count":0,"update_count":318}
I20260709 12:20:56.744997 31859 maintenance_manager.cc:419] P a0887d9cdc8044caa86af7acdb454104: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.422431
I20260709 12:20:56.750283 32044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:20:56.750988 32044 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:56.751336 32044 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:56.756227 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:56.759940 32044 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:56.769115 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:20:56.776403 32044 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:56.777058  8311 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 12:20:56.812080 32044 raft_consensus.cc:1240] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:56.839428 31964 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:56.844386 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:56.845067 32117 raft_consensus.cc:3055] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:56.845350 32117 raft_consensus.cc:740] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:56.846089 32117 consensus_queue.cc:260] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:56.849365 32117 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:56.867846 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
W20260709 12:20:56.879953 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 97262.9us
W20260709 12:20:56.896420 32012 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:56.902940 31972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:56.910485 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c" is_pre_election: true
I20260709 12:20:56.923935 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fd2ead38d34aac9ef68364e1f4a2f8" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:56.924587 31969 raft_consensus.cc:3060] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:56.969277 32010 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:56.976269 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:56.979338 31972 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:56.983093 32117 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:56.985220 31969 raft_consensus.cc:2468] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:20:56.990468 32013 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:56.992650  8268 raft_consensus.cc:2804] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:56.993203  8268 raft_consensus.cc:697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:56.994067  8268 consensus_queue.cc:237] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:20:56.989902 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 92611.9us
I20260709 12:20:57.006050 32117 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:57.000880 32042 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.007827 32010 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:57.012148  8266 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:57.014460  8312 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:57.014403 32656 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.019695 32010 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:57.020768 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:57.024399 32653 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:57.024617  8289 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:56.945788  8293 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.061589 31869 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 17 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:20:57.081563 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 45580.7us
W20260709 12:20:57.082267 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 45874.9us
I20260709 12:20:57.086858  8018 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:57.087657  8018 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:57.092938 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:20:57.094877 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 51867.7us
I20260709 12:20:57.121862  8018 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:57.155058  8246 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:20:57.176182 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:20:57.179574 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 50182.4us
I20260709 12:20:57.201592  8321 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:20:57.202248  8321 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:57.204499  8321 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:57.167881  8269 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:20:57.215729 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:20:57.216914 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:57.239406 32117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:57.240016 32117 raft_consensus.cc:606] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:20:57.246753 32046 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.251294 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:20:57.248208 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:57.267153 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:57.284677  8251 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:57.286698  8222 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:57.290556 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:20:57.333860  8322 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:57.336923  8237 raft_consensus.cc:993] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:57.337589  8237 raft_consensus.cc:1081] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:20:57.345578 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:57.347692 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:57.348506 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:57.386108 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 45935.5us
W20260709 12:20:57.393034 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 43733.1us
I20260709 12:20:57.413807 31866 catalog_manager.cc:5697] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 25 to 26, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 26 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:57.446954 31963 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:20:57.451153  8309 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:57.461458  8306 raft_consensus.cc:2243] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.462442  8306 raft_consensus.cc:2272] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.462734 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 68434.1us
W20260709 12:20:57.518810 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 54499.1us
I20260709 12:20:57.520956 32115 raft_consensus.cc:1275] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:20:57.527104 32046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:20:57.527721 32046 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:57.527973 32046 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:57.573937  8306 ts_tablet_manager.cc:1916] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:57.587167 32658 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.589883 32046 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:57.593338  8255 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.614492 31870 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:57.620081 32112 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.627857 32046 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:20:57.634913 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:20:57.635484 32115 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:57.635779 32115 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:57.637593 32115 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:57.638685 32115 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:57.645694 32115 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:57.650885 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:20:57.663203 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:57.663823 31969 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:57.545936  8309 consensus_queue.cc:1048] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.121s
I20260709 12:20:57.668208  8269 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:57.668757  8269 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:57.669601  8269 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:57.691784 32116 consensus_queue.cc:237] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:57.693425 31864 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.697312 32660 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:57.699077  8320 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:57.703420 31963 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.709214 31971 raft_consensus.cc:1275] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:57.710672 31969 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 25.
I20260709 12:20:57.713727  8260 consensus_queue.cc:1048] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:57.717487  8195 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.723505 32660 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.738505 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81129181ae3246c1bfb02954120c9b91 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:57.739225 31970 raft_consensus.cc:2955] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.763440 31864 catalog_manager.cc:5697] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 14 to 15, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 22 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.797909  8306 ts_tablet_manager.cc:1929] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.13
I20260709 12:20:57.801582 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 81129181ae3246c1bfb02954120c9b91 with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:37884
W20260709 12:20:57.814145  8277 consensus_queue.cc:1175] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "38c927f0ee104b57972899f5b27aa99b" responder_term: 27 status { last_received { term: 27 index: 18 } last_committed_idx: 18 last_received_current_leader { term: 27 index: 18 } }
I20260709 12:20:57.823522 31969 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:57.834930 32660 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:20:57.836650  8277 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:57.835011 32038 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:20:57.842020  8219 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:57.849684  8263 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Committing config change with OpId 27.19: config changed from index 18 to 19, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.856415 31870 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 24 to 25, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 25 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:57.815053  8306 log.cc:1199] T 786db0eb7e9b431ba90c5ad880c46c9e P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/786db0eb7e9b431ba90c5ad880c46c9e
I20260709 12:20:57.870398  8306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37884 (request call id 9) took 1467 ms. Trace:
I20260709 12:20:57.870791  8306 rpcz_store.cc:276] 0709 12:20:56.423473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:56.464578 (+ 41105us) service_pool.cc:224] Handling call
0709 12:20:56.638817 (+174239us) ts_tablet_manager.cc:1151] Deleting tablet 786db0eb7e9b431ba90c5ad880c46c9e
0709 12:20:56.638897 (+    80us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:20:56.694269 (+ 55372us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b54001937d4. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1d7ff9 00007f22911e5482 00007f22912162d0 00007f22911fc83d 00007f22911fc7e9 00007f22911fc779 00007f22911fc741 00007f22911fb96d 0000000000388634 0000000000388479
0709 12:20:57.652089 (+957820us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:57.797772 (+145683us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:57.870250 (+ 72478us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":739153,"spinlock_wait_cycles":13634432,"tablet-delete.queue_time_us":172955,"thread_start_us":97168,"threads_started":1}
I20260709 12:20:57.854053 32038 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Committing config change with OpId 27.19: config changed from index 18 to 19, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.873085 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 786db0eb7e9b431ba90c5ad880c46c9e (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:57.920791  8306 tablet_replica.cc:333] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:20:57.962500 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 58ee45b18b714479af8291d9056afb9e with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:57.964250 32660 raft_consensus.cc:2955] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Committing config change with OpId 27.19: config changed from index 18 to 19, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.969739 31864 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 18 to 19, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 27 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:57.979818  8306 raft_consensus.cc:2243] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.990978  8306 raft_consensus.cc:2272] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.003175 32039 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:58.007480 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 58ee45b18b714479af8291d9056afb9e with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 19) from {username='slave'} at 127.0.0.1:33516
I20260709 12:20:58.017565 32660 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:58.022782 32116 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:58.026593  8309 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:58.028376  8306 ts_tablet_manager.cc:1916] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:58.031860  8269 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:58.046126  8341 tablet_replica.cc:333] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:20:58.047721  8309 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:58.156972  8341 raft_consensus.cc:2243] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.157961  8341 raft_consensus.cc:2272] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.165962 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 115343us
I20260709 12:20:58.216952 31969 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:58.196501 32116 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:58.220180  8222 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:58.231338  8341 ts_tablet_manager.cc:1916] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:58.273542 31870 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:58.290027 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08b3d325c1944e45a59e666e91d0dbb6 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:20:58.290373 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 119042us
I20260709 12:20:58.368019  8306 ts_tablet_manager.cc:1929] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.14
W20260709 12:20:58.403851 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 120685us
I20260709 12:20:58.407806  8306 log.cc:1199] T 81129181ae3246c1bfb02954120c9b91 P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/81129181ae3246c1bfb02954120c9b91
W20260709 12:20:58.418629 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 71493.5us
I20260709 12:20:58.505867 32116 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:20:58.524013 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 178981us
I20260709 12:20:58.539088 32658 raft_consensus.cc:2955] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:20:58.555792 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 81129181ae3246c1bfb02954120c9b91 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:58.567394 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 08b3d325c1944e45a59e666e91d0dbb6 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:60388
I20260709 12:20:58.591048 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:58.615664  8341 ts_tablet_manager.cc:1929] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.18
I20260709 12:20:58.616195  8341 log.cc:1199] T 58ee45b18b714479af8291d9056afb9e P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/58ee45b18b714479af8291d9056afb9e
I20260709 12:20:58.618005 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 58ee45b18b714479af8291d9056afb9e (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:58.623030  8230 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:58.555416 31858 maintenance_manager.cc:643] P a0887d9cdc8044caa86af7acdb454104: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 1.807s	user 0.137s	sys 0.039s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":29542,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":182012,"lbm_read_time_us":2601,"lbm_reads_lt_1ms":26,"lbm_write_time_us":1776,"lbm_writes_lt_1ms":12,"peak_mem_usage":582715,"reinsert_count":0,"thread_start_us":188712,"threads_started":2,"update_count":338}
I20260709 12:20:58.668839 31859 maintenance_manager.cc:419] P a0887d9cdc8044caa86af7acdb454104: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 12:20:58.671759 31858 maintenance_manager.cc:643] P a0887d9cdc8044caa86af7acdb454104: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 12:20:58.694945 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 77473.5us
I20260709 12:20:58.717097  8365 tablet_replica.cc:333] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:20:58.743997 32046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:58.744705 32046 raft_consensus.cc:606] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:58.747888 32046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:20:58.743292  8365 raft_consensus.cc:2243] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.748482 32046 raft_consensus.cc:606] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:20:58.749444  8365 raft_consensus.cc:2272] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.750006 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:58.775477 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:58.753720  8289 raft_consensus.cc:993] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:58.776376 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:58.779835  8289 raft_consensus.cc:1081] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:58.781852 32112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:58.782464 32112 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:58.782806 32112 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:58.807480  8365 ts_tablet_manager.cc:1916] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:58.832162 32112 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:58.837060 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:20:58.837771 31971 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:58.843952 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:58.844686 32046 raft_consensus.cc:3055] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:58.845007 32046 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:58.846745 32112 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:20:58.845778 32046 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:58.900197 32046 raft_consensus.cc:3060] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:58.908646 31971 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:58.913874  8365 ts_tablet_manager.cc:1929] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.17
I20260709 12:20:58.932333  8365 log.cc:1199] T 08b3d325c1944e45a59e666e91d0dbb6 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/08b3d325c1944e45a59e666e91d0dbb6
I20260709 12:20:58.933553 32084 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:20:58.934136 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 08b3d325c1944e45a59e666e91d0dbb6 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:20:58.934470  8195 raft_consensus.cc:2804] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:58.935087  8195 raft_consensus.cc:697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:58.936033  8195 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:20:58.949568 31864 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 25 to 26, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:58.955868 32046 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:20:59.260735  8366 raft_consensus.cc:993] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:59.261287  8366 raft_consensus.cc:1081] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:20:59.264392 32112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47764
I20260709 12:20:59.264950 32112 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:59.265280 32112 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:59.274996 32112 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:59.280210 32112 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:20:59.281728 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:20:59.282426 32046 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:59.282740 32046 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:20:59.283521 32046 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:59.286325 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:20:59.286991 32658 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:59.284854 32046 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:59.305584 32658 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:20:59.306788 32085 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:20:59.307639  8278 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:59.308135  8278 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:20:59.308979  8278 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:20:59.311292 32046 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:20:59.374089 31864 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 27 to 28, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 28 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:59.407829 31963 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:59.412789  8278 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:59.455410 32046 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:59.457321  8195 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:59.682925 32038 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:59.707418 32046 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:59.709491  8195 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:20:59.749208 32011 consensus_peers.cc:597] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4633b71e43824d84895ff136e5765fb1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:59.750208 32115 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:59.750607 31963 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:59.751981  8366 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:59.752957  8289 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:59.797289 32658 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:20:59.799522  8320 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:59.835178 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:59.881636  8289 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:59.883581 31963 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:59.895330 32117 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:59.945127 31864 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 13, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:20:59.960142 32117 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:20:59.966248 32112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:20:59.966954 32112 raft_consensus.cc:606] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Received request to transfer leadership to TS f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:20:59.985320 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:59.988441 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:59.989856 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:00.112229 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 102214us
W20260709 12:21:00.171834 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 61250.7us
W20260709 12:21:00.173389 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 58986.8us
I20260709 12:21:00.181329 32038 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:21:00.200871  8386 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:00.227684  8386 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:00.265065 32038 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:00.290335 31971 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:21:00.301035 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 207282us
W20260709 12:21:00.378757 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 78199.6us
W20260709 12:21:00.415815 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 307866us
I20260709 12:21:00.301491  8278 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.116s
I20260709 12:21:00.485683 31866 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:00.532397 31970 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:00.534138  8411 ts_tablet_manager.cc:933] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:00.565717  8397 raft_consensus.cc:993] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:21:00.566744  8397 raft_consensus.cc:1081] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Signalling peer f05e4741d35c4cf3a31358c56e65b30c to start an election
I20260709 12:21:00.571324 31971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
 from {username='slave'} at 127.0.0.1:56072
I20260709 12:21:00.571882 31971 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.572243 31971 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:21:00.600975 32085 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12a3b65ef9024a64b336b00ad51e294b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.600368 31971 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:00.645998 31964 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:21:00.658463 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 56694us
I20260709 12:21:00.671355 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:00.672058 32112 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:21:00.672358 32112 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:00.673048 32112 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:00.673976 32112 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:21:00.677600 31970 raft_consensus.cc:1240] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 23. Current term is 24. Ops: []
I20260709 12:21:00.690070 32112 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:21:00.692147 31971 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:00.694286 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:21:00.696768 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:00.697373 32658 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:21:00.698036  8398 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:21:00.706925  8398 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Leader. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:21:00.707798  8398 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:00.710947 31936 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a06d9856d134874b3144fd2abdbfc71. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.710250 32658 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e4741d35c4cf3a31358c56e65b30c in term 24.
I20260709 12:21:00.710520 32112 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:21:00.715108  8358 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:21:00.697651  8411 tablet_copy_client.cc:323] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:21:00.735517 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 4633b71e43824d84895ff136e5765fb1 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:21:00.736086 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 4633b71e43824d84895ff136e5765fb1 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-4633b71e43824d84895ff136e5765fb1
I20260709 12:21:00.746481 32039 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:21:00.765892 32057 tablet_copy_source_session.cc:215] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:00.777024  8411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633b71e43824d84895ff136e5765fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:00.768498  8385 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:21:00.794749  8411 tablet_copy_client.cc:806] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:00.795408  8411 tablet_copy_client.cc:670] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:00.804337  8411 tablet_copy_client.cc:538] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:00.807556  8423 ts_tablet_manager.cc:933] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:00.811529  8423 tablet_copy_client.cc:323] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:21:00.817301  8411 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
W20260709 12:21:00.840242 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 57765.4us
I20260709 12:21:00.853644 31870 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: term changed from 23 to 24, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1). New cstate: current_term: 24 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.895150 32130 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet 12a3b65ef9024a64b336b00ad51e294b from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:21:00.896409 32130 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet 12a3b65ef9024a64b336b00ad51e294b from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-12a3b65ef9024a64b336b00ad51e294b
I20260709 12:21:00.924677 32117 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:00.972084  8414 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:21:01.038122 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 64053.4us
I20260709 12:21:01.010008 32130 tablet_copy_source_session.cc:215] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:01.074262 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 49373.4us
I20260709 12:21:01.107580  8423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a3b65ef9024a64b336b00ad51e294b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.112365 32046 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:01.121219 31866 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.154567 32042 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:01.170305 32651 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:21:01.173548  8359 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:01.232556  8427 ts_tablet_manager.cc:933] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
I20260709 12:21:01.232709 32116 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:21:01.234140 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 56538.8us
I20260709 12:21:01.236161 32116 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:21:01.238855  8412 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:01.272029  8268 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:21:01.273221 32011 consensus_peers.cc:597] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3949314523941858ae6d6a4fd770fb8. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:01.287933  8428 log.cc:927] Time spent T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 12:21:01.300192 31970 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:21:01.301968  8423 tablet_copy_client.cc:806] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:01.310676  8401 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:21:01.309715  8423 tablet_copy_client.cc:670] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:01.359951  8423 tablet_copy_client.cc:538] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:01.398453 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:01.418556  8427 tablet_copy_client.cc:323] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
I20260709 12:21:01.417070  8268 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:01.427574 31981 tablet_copy_service.cc:140] P f05e4741d35c4cf3a31358c56e65b30c: Received BeginTabletCopySession request for tablet 1a06d9856d134874b3144fd2abdbfc71 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:58058)
I20260709 12:21:01.428223 31981 tablet_copy_service.cc:161] P f05e4741d35c4cf3a31358c56e65b30c: Beginning new tablet copy session on tablet 1a06d9856d134874b3144fd2abdbfc71 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:58058: session id = ec7f26ea46944ea498ad253b1b1819b0-1a06d9856d134874b3144fd2abdbfc71
I20260709 12:21:01.430138 32117 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:01.433676 31970 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:01.452457 31981 tablet_copy_source_session.cc:215] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:01.466662  8411 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:01.472810  8423 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:21:01.474963  8427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a06d9856d134874b3144fd2abdbfc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:01.488683 31866 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 14, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 21 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:01.586843  8411 tablet_bootstrap.cc:492] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:21:01.588011  8411 ts_tablet_manager.cc:1403] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.771s	user 0.057s	sys 0.048s
I20260709 12:21:01.666543  8427 tablet_copy_client.cc:806] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:01.667299  8427 tablet_copy_client.cc:670] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:01.669111  8411 raft_consensus.cc:359] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:01.670083  8411 raft_consensus.cc:740] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:21:01.712914  8427 tablet_copy_client.cc:538] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:01.744155  8411 consensus_queue.cc:260] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:01.746881  8411 ts_tablet_manager.cc:1434] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.158s	user 0.008s	sys 0.000s
I20260709 12:21:01.757735 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-4633b71e43824d84895ff136e5765fb1 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:01.759553 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-4633b71e43824d84895ff136e5765fb1 on tablet 4633b71e43824d84895ff136e5765fb1 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:01.775650  8427 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:21:01.786698  8411 ts_tablet_manager.cc:933] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:02.016124 31969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e"
dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
mode: GRACEFUL
new_leader_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:46882
I20260709 12:21:02.016929 31969 raft_consensus.cc:606] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Received request to transfer leadership to TS 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:02.022351 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:21:02.026531  8411 tablet_copy_client.cc:323] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
I20260709 12:21:02.028535 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet d3949314523941858ae6d6a4fd770fb8 from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:21:02.029016 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet d3949314523941858ae6d6a4fd770fb8 from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-d3949314523941858ae6d6a4fd770fb8
I20260709 12:21:02.025961 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:02.049678  8358 raft_consensus.cc:993] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: : Instructing follower 38c927f0ee104b57972899f5b27aa99b to start an election
I20260709 12:21:02.050961 32057 tablet_copy_source_session.cc:215] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:02.056931  8414 raft_consensus.cc:1081] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Signalling peer 38c927f0ee104b57972899f5b27aa99b to start an election
W20260709 12:21:02.072343 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 91275us
I20260709 12:21:02.072868  8411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3949314523941858ae6d6a4fd770fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:02.083027 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:02.095156 32112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
 from {username='slave'} at 127.0.0.1:47766
I20260709 12:21:02.095860 32112 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:02.096274 32112 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:02.117012 32112 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:02.129699  8411 tablet_copy_client.cc:806] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:02.130488  8411 tablet_copy_client.cc:670] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:02.153098 32651 raft_consensus.cc:1217] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.12->[22.13-22.13]   Dedup: 22.13->[]
I20260709 12:21:02.155506 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:02.156152 32651 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:02.168181 31971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:21:02.168941 31971 raft_consensus.cc:3055] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:21:02.169315 31971 raft_consensus.cc:740] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f05e4741d35c4cf3a31358c56e65b30c, State: Running, Role: LEADER
I20260709 12:21:02.177642 31971 consensus_queue.cc:260] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:02.179104 31971 raft_consensus.cc:3060] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:02.195442  8423 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=3 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:02.153218 32112 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:02.196662  8423 tablet_bootstrap.cc:492] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:21:02.197428  8423 ts_tablet_manager.cc:1403] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.725s	user 0.060s	sys 0.038s
I20260709 12:21:02.202401  8411 tablet_copy_client.cc:538] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:02.210296 32651 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:21:02.221028  8423 raft_consensus.cc:359] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:02.228209  8411 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
I20260709 12:21:02.235224  8423 raft_consensus.cc:740] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:21:02.235584 32085 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:02.246874  8423 consensus_queue.cc:260] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:02.247434  8377 raft_consensus.cc:2804] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:21:02.248086  8377 raft_consensus.cc:697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:02.249022  8377 consensus_queue.cc:237] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:02.249926  8423 ts_tablet_manager.cc:1434] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.002s
I20260709 12:21:02.234279 31971 raft_consensus.cc:2468] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 25.
I20260709 12:21:02.253758 32130 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-12a3b65ef9024a64b336b00ad51e294b received from {username='slave'} at 127.0.0.1:34592
I20260709 12:21:02.254233 32130 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-12a3b65ef9024a64b336b00ad51e294b on tablet 12a3b65ef9024a64b336b00ad51e294b with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:02.272500 31870 catalog_manager.cc:5697] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 24 to 25, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 25 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:02.351291  8427 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:02.352419  8427 tablet_bootstrap.cc:492] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:21:02.353251  8427 ts_tablet_manager.cc:1403] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.578s	user 0.049s	sys 0.011s
I20260709 12:21:02.376359  8427 raft_consensus.cc:359] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:02.377442  8427 raft_consensus.cc:740] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:21:02.378237  8427 consensus_queue.cc:260] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:02.399881  8427 ts_tablet_manager.cc:1434] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.003s
I20260709 12:21:02.405421  8411 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:02.418790 32651 raft_consensus.cc:1217] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Deduplicated request from leader. Original: 26.14->[26.15-26.15]   Dedup: 26.15->[]
I20260709 12:21:02.422405 31981 tablet_copy_service.cc:342] P f05e4741d35c4cf3a31358c56e65b30c: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-1a06d9856d134874b3144fd2abdbfc71 received from {username='slave'} at 127.0.0.1:58058
I20260709 12:21:02.422917 31981 tablet_copy_service.cc:434] P f05e4741d35c4cf3a31358c56e65b30c: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-1a06d9856d134874b3144fd2abdbfc71 on tablet 1a06d9856d134874b3144fd2abdbfc71 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:02.432925  8411 tablet_bootstrap.cc:492] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:21:02.433735  8411 ts_tablet_manager.cc:1403] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.206s	user 0.052s	sys 0.002s
I20260709 12:21:02.446841  8411 raft_consensus.cc:359] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:02.450922  8411 raft_consensus.cc:740] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:21:02.451586  8411 consensus_queue.cc:260] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:02.453961  8411 ts_tablet_manager.cc:1434] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 12:21:02.480587 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-d3949314523941858ae6d6a4fd770fb8 received from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:02.481026 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-d3949314523941858ae6d6a4fd770fb8 on tablet d3949314523941858ae6d6a4fd770fb8 with peer ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:02.560662  8401 raft_consensus.cc:1064] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:21:02.567456  8401 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:02.607987 32651 raft_consensus.cc:1217] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:21:02.621537 32112 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:21:02.624516  8440 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:21:02.640789 32654 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:21:02.650897 32651 raft_consensus.cc:1217] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.13->[21.14-21.14]   Dedup: 21.14->[]
I20260709 12:21:02.654475  8450 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:02.658057 32660 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:21:02.659675  8268 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:21:02.674705 31971 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:21:02.676841  8289 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:21:02.775611 31969 raft_consensus.cc:1275] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:21:02.784062  8268 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:02.793097  8450 consensus_queue.cc:1048] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:02.805601 32660 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:21:02.865681 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 57353.2us
W20260709 12:21:02.873023 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 63529.7us
W20260709 12:21:02.966780 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 154753us
I20260709 12:21:02.969166 31866 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:21:03.109817 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 148574us
I20260709 12:21:03.169133 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:03.168030 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 108363us
W20260709 12:21:03.258455 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 244207us
I20260709 12:21:03.290797  8440 raft_consensus.cc:1064] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:21:03.293509  8440 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:03.348122 32113 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.390926 32112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
new_leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:37262
I20260709 12:21:03.391556 32112 raft_consensus.cc:606] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Received request to transfer leadership to TS 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:03.395006 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:21:03.396732 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:03.326658 31970 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.467337  8467 raft_consensus.cc:1064] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:21:03.472362  8467 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:03.482532 31971 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:21:03.482872 32119 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:21:03.482863 32658 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:21:03.487488  8407 raft_consensus.cc:1064] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:21:03.484400  8440 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:21:03.497556  8462 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:03.501077  8407 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 13, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:03.504477  8289 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:21:03.496196 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:03.613464 31970 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:21:03.615465 32046 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:21:03.617400 32658 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:21:03.654628  8473 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:21:03.654785 32038 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:03.658385  8485 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:03.661325  8450 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:03.680414 32046 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.692224 32660 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEARNER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.693363 32117 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.696770  8414 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:21:03.706508  8481 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:03.708082  8412 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:21:03.866917 32117 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.869779  8460 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:21:03.871655  8499 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: Append to log took a long time: real 0.107s	user 0.003s	sys 0.000s
W20260709 12:21:03.873781  8498 log.cc:927] Time spent T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:21:03.877938  8505 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.880944  8482 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.882879 32046 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.899808 32658 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.916060  8488 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.940464  8489 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.950659 32117 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.955260  8480 raft_consensus.cc:993] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:21:03.955788  8480 raft_consensus.cc:1081] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:21:03.973802 32660 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.982920 31969 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:03.979290 31870 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c reported cstate change: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:03.992411 31971 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.000931 32038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:21:04.001708 32038 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:04.016752 32038 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:04.078588 32660 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.078100 32038 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:04.100446 32038 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:04.103170 31969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: true dest_uuid: "f05e4741d35c4cf3a31358c56e65b30c"
I20260709 12:21:04.106231 31969 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:04.104058 32660 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.125613  8509 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:04.125815 32042 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:21:04.132941 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 178442us
I20260709 12:21:04.139158 32115 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.141191 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:04.141938 32117 raft_consensus.cc:3055] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:21:04.145229 32117 raft_consensus.cc:740] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:04.149437 32117 consensus_queue.cc:260] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:04.151789 32117 raft_consensus.cc:3060] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:04.170979 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4633b71e43824d84895ff136e5765fb1 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.180337 32112 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.191417 31866 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 14 to 15, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.199316 31969 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:04.205396 32010 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c; no voters: 
I20260709 12:21:04.206540  8508 raft_consensus.cc:2804] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:21:04.208031  8508 raft_consensus.cc:697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:04.209029  8508 consensus_queue.cc:237] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:04.215405 32117 raft_consensus.cc:2468] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:04.216140 32658 raft_consensus.cc:2955] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.379138 31870 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:04.387504 31870 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 14 to 15, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.424384 31866 catalog_manager.cc:5697] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 27 to 28, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 28 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.431790 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet 4633b71e43824d84895ff136e5765fb1 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:60388
I20260709 12:21:04.437350 32117 consensus_queue.cc:237] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:04.447220 32658 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:21:04.449098 31972 raft_consensus.cc:1275] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:21:04.448997  8473 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:04.450795  8482 consensus_queue.cc:1048] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:04.455947  8413 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Committing config change with OpId 26.17: config changed from index 16 to 17, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.457595 32654 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Committing config change with OpId 26.17: config changed from index 16 to 17, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.475443 31972 raft_consensus.cc:2955] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Committing config change with OpId 26.17: config changed from index 16 to 17, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.482368 32038 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:04.488296 31969 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:04.496265 32654 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:04.499017 32039 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:04.498458 31864 catalog_manager.cc:5697] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 16 to 17, VOTER 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) evicted. New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.500806  8463 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:04.502779  8414 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:04.514911  8414 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.515496 32654 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.521327 32658 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:21:04.524763 32032 tablet_service.cc:1558] Processing DeleteTablet for tablet 12a3b65ef9024a64b336b00ad51e294b with delete_type TABLET_DATA_TOMBSTONED (TS 3836f9b5cd36488da56d4540dc9d80ec not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:37884
I20260709 12:21:04.526881  8508 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:04.526118 32039 raft_consensus.cc:2955] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.543227 32117 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:21:04.546265  8508 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:04.561903 31857 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12a3b65ef9024a64b336b00ad51e294b with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.575759  8511 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index 15 to 16, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.576668 31864 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: config changed from index 15 to 16, VOTER 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) evicted. New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.578589  8518 tablet_replica.cc:333] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:04.579805 32654 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 15 to 16, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.583485 32112 raft_consensus.cc:2955] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 15 to 16, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.579815  8518 raft_consensus.cc:2243] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.587713  8518 raft_consensus.cc:2272] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.608829 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1a06d9856d134874b3144fd2abdbfc71 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.635093 31972 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:21:04.645792 31854 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d3949314523941858ae6d6a4fd770fb8 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.647073  8517 tablet_replica.cc:333] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:04.648525  8517 raft_consensus.cc:2243] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.649632  8517 raft_consensus.cc:2272] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.660224 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:04.667241  8518 ts_tablet_manager.cc:1916] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.668931  8517 ts_tablet_manager.cc:1916] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.645440  8462 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 12:21:04.682669 31870 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index 15 to 16, VOTER f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) evicted. New cstate: current_term: 21 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:04.685575 32104 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a06d9856d134874b3144fd2abdbfc71 with delete_type TABLET_DATA_TOMBSTONED (TS 38c927f0ee104b57972899f5b27aa99b not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:33516
I20260709 12:21:04.726405 32112 raft_consensus.cc:1275] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:21:04.732316  8474 consensus_queue.cc:1048] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:04.753724  8523 tablet_replica.cc:333] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:04.755009  8523 raft_consensus.cc:2243] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.755964  8523 raft_consensus.cc:2272] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.774441  8518 ts_tablet_manager.cc:1929] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 26.16
I20260709 12:21:04.775183  8518 log.cc:1199] T 12a3b65ef9024a64b336b00ad51e294b P 3836f9b5cd36488da56d4540dc9d80ec: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/wals/12a3b65ef9024a64b336b00ad51e294b
I20260709 12:21:04.796316 31854 catalog_manager.cc:5028] TS 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): tablet 12a3b65ef9024a64b336b00ad51e294b (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:21:04.825335  8523 ts_tablet_manager.cc:1916] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.831318 32038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
mode: GRACEFUL
new_leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:35908
I20260709 12:21:04.832023 32038 raft_consensus.cc:606] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Received request to transfer leadership to TS ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:21:04.832926 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 201904us
I20260709 12:21:04.843104 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:21:04.845271 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:04.846280 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:04.849665 31961 tablet_service.cc:1558] Processing DeleteTablet for tablet d3949314523941858ae6d6a4fd770fb8 with delete_type TABLET_DATA_TOMBSTONED (TS f05e4741d35c4cf3a31358c56e65b30c not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:60388
I20260709 12:21:04.881505  8517 ts_tablet_manager.cc:1929] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.14
I20260709 12:21:04.935616  8517 log.cc:1199] T 4633b71e43824d84895ff136e5765fb1 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/4633b71e43824d84895ff136e5765fb1
I20260709 12:21:04.980389  8517 tablet_replica.cc:333] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:04.982828 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet 4633b71e43824d84895ff136e5765fb1 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
W20260709 12:21:04.989040 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 72130.4us
I20260709 12:21:05.064797  8517 raft_consensus.cc:2243] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:05.065891  8517 raft_consensus.cc:2272] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:05.073638  8462 raft_consensus.cc:993] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: : Instructing follower ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:21:05.074200  8462 raft_consensus.cc:1081] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Signalling peer ec7f26ea46944ea498ad253b1b1819b0 to start an election
I20260709 12:21:05.075846  8517 ts_tablet_manager.cc:1916] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:05.091995 32654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81"
dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
 from {username='slave'} at 127.0.0.1:47558
I20260709 12:21:05.092687 32654 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:05.093091 32654 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:05.106774 32654 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:05.114065 32654 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:05.131816  8523 ts_tablet_manager.cc:1929] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 12:21:05.132387  8523 log.cc:1199] T 1a06d9856d134874b3144fd2abdbfc71 P 38c927f0ee104b57972899f5b27aa99b: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/wals/1a06d9856d134874b3144fd2abdbfc71
I20260709 12:21:05.153002 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:05.153609 32039 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:21:05.153909 32039 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:05.154630 32039 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:05.156261 32039 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:05.166028 32039 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:21:05.180135 32623 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:05.181015  8492 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:05.181605  8492 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:05.182663  8492 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:05.201112 31857 catalog_manager.cc:5028] TS 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): tablet 1a06d9856d134874b3144fd2abdbfc71 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:21:05.205444 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:05.206153 32112 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:05.216550 32112 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:21:05.223587 31870 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 22 to 23, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 23 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:05.227536  8517 ts_tablet_manager.cc:1929] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.15
I20260709 12:21:05.228101  8517 log.cc:1199] T d3949314523941858ae6d6a4fd770fb8 P f05e4741d35c4cf3a31358c56e65b30c: Deleting WAL directory at /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/wals/d3949314523941858ae6d6a4fd770fb8
I20260709 12:21:05.229796 31854 catalog_manager.cc:5028] TS f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): tablet d3949314523941858ae6d6a4fd770fb8 (table test-workload [id=39fe72954527408bbd18e5daabbcf01a]) successfully deleted
I20260709 12:21:05.679845 32042 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:05.683921  8507 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:05.741691 32112 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:05.748504  8526 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:05.988126 31969 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:06.002858 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.041854 32112 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:21:06.043833 31936 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d6c0fabd5a9486f95f088218e4ba65d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:06.045615 32039 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Refusing update from remote peer f05e4741d35c4cf3a31358c56e65b30c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:06.048138  8519 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:06.050367  8519 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:06.101102  8519 raft_consensus.cc:2955] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:06.104393 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
I20260709 12:21:06.111407 32039 raft_consensus.cc:2955] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:06.113469 32112 raft_consensus.cc:2955] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:21:06.132155 32010 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
I20260709 12:21:06.136874 31870 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 22 leader_uuid: "f05e4741d35c4cf3a31358c56e65b30c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:21:06.196357 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64218.8us
W20260709 12:21:06.207288 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 61929.3us
W20260709 12:21:06.222767 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.224725 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.249153 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.307705 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.308384 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.371518 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.386031 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.439373 31935 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 47955.6us
W20260709 12:21:06.443032 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.506278 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 92086.5us
W20260709 12:21:06.507442 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 48634us
W20260709 12:21:06.549715 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.550240 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 64355.1us
W20260709 12:21:06.558990 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.607336 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 48412.4us
W20260709 12:21:06.608060 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.643172 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 138191us
W20260709 12:21:06.643263 31938 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 154362us
W20260709 12:21:06.642532 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.693383 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.699647 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.731805 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 313538us
W20260709 12:21:06.745219 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.747442 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.748313 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.759545 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.774592 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.775418 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.783141 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.786345 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:06.791208 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:06.833890  8580 ts_tablet_manager.cc:933] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
W20260709 12:21:06.888582 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 98471.6us
I20260709 12:21:06.972047  8580 tablet_copy_client.cc:323] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.1:40903
W20260709 12:21:07.004438 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 110793us
I20260709 12:21:07.062536  8519 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.063133  8519 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.065191  8519 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.067129 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.070441 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef2f2222a1ac4f6f923fdee4de1f48dc" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.072224 31938 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:07.075683  8552 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:07.086884 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:07.092350 30820 tablet_replica.cc:333] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
W20260709 12:21:07.091104 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:07.094053 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:07.095219 30820 raft_consensus.cc:2243] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.095846 30820 raft_consensus.cc:2272] T 78b16558212549a98cd68f4e744eb410 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.098044 30820 tablet_replica.cc:333] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.100260 30820 raft_consensus.cc:2243] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.101562 30820 raft_consensus.cc:2272] T d1ae9fcf8eb748c6b0cb9703adc9f338 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:07.094640 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:07.108467 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:07.116822 30820 tablet_replica.cc:333] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.127122 30820 raft_consensus.cc:2243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.127938 30820 raft_consensus.cc:2272] T 74fd2ead38d34aac9ef68364e1f4a2f8 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.130918 30820 tablet_replica.cc:333] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.136350 30820 raft_consensus.cc:2243] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:07.137888 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:07.143997  8481 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.154899  8481 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.136571  8552 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.158619  8552 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.160713  8552 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.161265  8481 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.162549 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.164671 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.164697 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.166625 31936 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.167162  8578 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.167719  8578 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.169214  8481 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:07.169978  8578 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.171759 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.173933 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.175484 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:21:07.177853 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:07.149966  8519 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.178602  8519 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.180732  8519 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.147526 30820 raft_consensus.cc:2272] T 1612f8e575bd46cf8069fa1c21ca192b P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.185721 30820 tablet_replica.cc:333] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.177774 31935 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:07.195075  8519 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:07.196372 31935 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.197419  8519 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:07.190788 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.211071  8552 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.211727  8481 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.211620  8552 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.212365  8481 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.214071  8552 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.215566  8481 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.215517 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.216953 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.218895 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.220669 31936 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.221360  8578 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:07.222187 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19357e2a1cf1408b9391be0c12c3b0e8" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.227916 30820 raft_consensus.cc:2243] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.228562 30820 raft_consensus.cc:2272] T 16a6933cca8f4bce92ae16ff487ca7c1 P f05e4741d35c4cf3a31358c56e65b30c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.231923 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff18ae9165241a3937358efc94d1309" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.234238  8481 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.234867  8481 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.237915  8481 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.238178 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.240083 31935 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.241010  8552 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:07.249768 31935 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:07.260933 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4287254496472398f69d3a10b48d77" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.261312 30820 tablet_replica.cc:333] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.262769  8552 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:07.264263 31935 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.266546 30820 raft_consensus.cc:2243] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.274536  8552 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:07.275681  8552 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.276214  8552 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.279480  8552 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.280068 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.281510 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478b63c892a745d09e9508bc9cc5321f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.286190 30820 raft_consensus.cc:2272] T 2ff18ae9165241a3937358efc94d1309 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.297585  8552 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.298144  8552 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.300215  8552 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.301276 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.305379 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.308367 31935 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.313567 31935 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.315047  8578 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:07.324745 30820 tablet_replica.cc:333] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.329679 30820 raft_consensus.cc:2243] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.331454  8578 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:07.339150 30820 raft_consensus.cc:2272] T 9d2cc7077f01430da5a31385d674e266 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.343405 30820 tablet_replica.cc:333] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.344307 30820 raft_consensus.cc:2243] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.346436 30820 raft_consensus.cc:2272] T f3f78e3424864d319ee8da87f4848fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.350435 30820 tablet_replica.cc:333] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.350813  8552 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.351305 30820 raft_consensus.cc:2243] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.352860 30820 raft_consensus.cc:2272] T 8ca59f0a11104ab882fac419c003e8b7 P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.354548  8552 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.357481  8552 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.357905 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.359683  8552 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.360206  8552 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.363778 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.364727  8578 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.369837  8578 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.372215  8578 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.373678 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.365902 30820 tablet_replica.cc:333] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.367717  8481 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.376590  8481 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.377039 30820 raft_consensus.cc:2243] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.378769  8481 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.379693 30820 raft_consensus.cc:2272] T 1a06d9856d134874b3144fd2abdbfc71 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.384158 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.384610 30820 tablet_replica.cc:333] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.386137  8481 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.386766  8481 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.389436  8481 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.385633 30820 raft_consensus.cc:2243] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.390906 30820 raft_consensus.cc:2272] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.375816 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.367978 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.379702 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.381143 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.381395  8578 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.390826 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.397020  8578 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.397876 31935 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
W20260709 12:21:07.398835 31935 raft_consensus.cc:2686] T b88d8f995fcc4faea19a190f8dd669ab P f05e4741d35c4cf3a31358c56e65b30c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:21:07.399348  8578 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.399871  8578 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.400050 31935 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:07.400343  8578 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.400800  8519 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:07.402627 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.404059 30820 tablet_replica.cc:333] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.404361 31936 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.404958 30820 raft_consensus.cc:2243] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.410069  8578 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.411101  8578 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:07.413970 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.415067  8578 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.415601  8578 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.419332 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.421380 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.421967 31938 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.422755  8481 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:07.424144 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.426187 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.368122  8552 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.429188 31935 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.430773  8481 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:07.430888 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e0cf115549454c894335e70c43843a" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.432300 31938 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.432364 31936 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.433552 31938 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:07.434144  8519 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:07.435665  8562 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:07.436036  8578 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.436429  8562 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:07.439610 30820 raft_consensus.cc:2272] T 58ee45b18b714479af8291d9056afb9e P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.444662 30820 tablet_replica.cc:333] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.445456 30820 raft_consensus.cc:2243] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.446065 30820 raft_consensus.cc:2272] T 3695041c497f47468894b0baf52fcfd0 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.449154 30820 tablet_replica.cc:333] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.450018 30820 raft_consensus.cc:2243] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.450596 30820 raft_consensus.cc:2272] T ad86bf28242f4576af3db683944d53a3 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.453575 30820 tablet_replica.cc:333] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.454994 30820 raft_consensus.cc:2243] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.459012 30820 raft_consensus.cc:2272] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P f05e4741d35c4cf3a31358c56e65b30c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.463320 30820 tablet_replica.cc:333] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.464164 30820 raft_consensus.cc:2243] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.464655 30820 raft_consensus.cc:2272] T 786db0eb7e9b431ba90c5ad880c46c9e P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.470211  8552 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.481971  8481 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.482822  8481 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.484913  8481 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.486021 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.486722 30820 tablet_replica.cc:333] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.493163  8552 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.494529 30820 raft_consensus.cc:2243] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.495824 30820 raft_consensus.cc:2272] T e0bd59e23a6d4e1f9674d7662259db2f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.501377 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.505735 30820 tablet_replica.cc:333] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.506855 30820 raft_consensus.cc:2243] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.507534 30820 raft_consensus.cc:2272] T 71e0cf115549454c894335e70c43843a P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.511116 30820 tablet_replica.cc:333] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.511991 30820 raft_consensus.cc:2243] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.512854 30820 raft_consensus.cc:2272] T 02ef57fadd044f68ba212dc2fe410f83 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.517773  8552 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.519264 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.520630 30820 tablet_replica.cc:333] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.520843 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.521654 30820 raft_consensus.cc:2243] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.522405 30820 raft_consensus.cc:2272] T 19357e2a1cf1408b9391be0c12c3b0e8 P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.524231 31938 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.528272  8519 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:07.537516 31938 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.538954  8562 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:07.542325  8552 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:07.542980  8552 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.545143  8552 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.546715 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.557358 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.561173 31938 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.569149  8552 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:07.581638  8481 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:07.584192  8481 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:07.588419 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.589322 30820 tablet_replica.cc:333] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.590426 30820 raft_consensus.cc:2243] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.593662 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:07.595976  8578 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:07.596557  8578 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:07.593169  8481 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:07.603271 31938 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:07.611250  8562 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:07.613528 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:07.614369 30820 raft_consensus.cc:2272] T e828882e6b1940ffb33d4c2f76b47fe9 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.616134  8578 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:07.617746 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "f05e4741d35c4cf3a31358c56e65b30c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:07.625766 31935 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f05e4741d35c4cf3a31358c56e65b30c; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:07.627310  8552 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:07.642378 30820 tablet_replica.cc:333] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.643652 30820 raft_consensus.cc:2243] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.644698 30820 raft_consensus.cc:2272] T ef2f2222a1ac4f6f923fdee4de1f48dc P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.652221 30820 tablet_replica.cc:333] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.670818 30820 raft_consensus.cc:2243] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.694394 30820 raft_consensus.cc:2272] T 0e910ef3c0af41bf8d4fba5315d68eef P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.697484 30820 tablet_replica.cc:333] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.720605 30820 raft_consensus.cc:2243] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:07.742863 30820 raft_consensus.cc:2272] T 2c59a3d3077c4b8db6380a834f6ce3c0 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.752724 30820 tablet_replica.cc:333] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.780519 30820 raft_consensus.cc:2243] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.800200 30820 raft_consensus.cc:2272] T 478b63c892a745d09e9508bc9cc5321f P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.822420 30820 tablet_replica.cc:333] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.823449 30820 raft_consensus.cc:2243] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.833887 30820 raft_consensus.cc:2272] T 826cffce8bb540c8beb56cc00a702782 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.854945 30820 tablet_replica.cc:333] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.871990 30820 raft_consensus.cc:2243] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.873366 30820 raft_consensus.cc:2272] T 7c52a06827f24bb6bc1851474fc80e53 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.986626 30820 tablet_replica.cc:333] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:07.988897 30820 raft_consensus.cc:2243] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.017387 30820 raft_consensus.cc:2272] T c6c31e616d2c4874ac5a8e29453cc51b P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.040395 30820 tablet_replica.cc:333] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.041266 30820 raft_consensus.cc:2243] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.042209 30820 raft_consensus.cc:2272] T 81129181ae3246c1bfb02954120c9b91 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.056849 30820 tablet_replica.cc:333] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.058866 30820 raft_consensus.cc:2243] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:08.080981 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 71387us
I20260709 12:21:08.080965 30820 raft_consensus.cc:2272] T 1da57956732d44d1b912fb270de7d63f P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.092204 30820 tablet_replica.cc:333] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.093091 30820 raft_consensus.cc:2243] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.128437 30820 raft_consensus.cc:2272] T 2f93a3ac5b02416a8d1d2547d9a9c726 P f05e4741d35c4cf3a31358c56e65b30c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.134521 30820 tablet_replica.cc:333] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.148322 30820 raft_consensus.cc:2243] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.160718 30820 raft_consensus.cc:2272] T f953fa305a42492da9de0fa8c18c43eb P f05e4741d35c4cf3a31358c56e65b30c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.163404 30820 tablet_replica.cc:333] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.164570 30820 raft_consensus.cc:2243] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.167779 30820 raft_consensus.cc:2272] T 6d27358a7bd3493a8877bd9ce8ae9055 P f05e4741d35c4cf3a31358c56e65b30c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.170522 30820 tablet_replica.cc:333] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.193171 30820 raft_consensus.cc:2243] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.194481 30820 raft_consensus.cc:2272] T 58d45de750cd4fd1a0cdd13ad4f731fb P f05e4741d35c4cf3a31358c56e65b30c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.206342 30820 tablet_replica.cc:333] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.225473 30820 raft_consensus.cc:2243] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.226253 30820 raft_consensus.cc:2272] T d2f941a3c6cb4a188198a804d603db3b P f05e4741d35c4cf3a31358c56e65b30c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.246510 30820 tablet_replica.cc:333] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.267294 30820 raft_consensus.cc:2243] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.269346 30820 raft_consensus.cc:2272] T 775fd46a41044b1ea06bc796ef1b233b P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.293655 30820 tablet_replica.cc:333] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.294512 30820 raft_consensus.cc:2243] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.299057 30820 raft_consensus.cc:2272] T 12a3b65ef9024a64b336b00ad51e294b P f05e4741d35c4cf3a31358c56e65b30c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.301789 30820 tablet_replica.cc:333] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.316148 30820 raft_consensus.cc:2243] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.316920 30820 raft_consensus.cc:2272] T 8f4287254496472398f69d3a10b48d77 P f05e4741d35c4cf3a31358c56e65b30c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.327904 30820 tablet_replica.cc:333] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.361389 30820 raft_consensus.cc:2243] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.362622 30820 raft_consensus.cc:2272] T a7f1c1160177412eb547493e1f3c561f P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.371160 30820 tablet_replica.cc:333] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.374476 30820 raft_consensus.cc:2243] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.375345 30820 raft_consensus.cc:2272] T 51b3fd343e81436fa5bea82486abcc64 P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.391368 30820 tablet_replica.cc:333] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.392222 30820 raft_consensus.cc:2243] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.399751 30820 raft_consensus.cc:2272] T fbb363b96dbf430998ab5641d3b03f46 P f05e4741d35c4cf3a31358c56e65b30c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.405074 30820 tablet_replica.cc:333] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.419152 30820 raft_consensus.cc:2243] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.433328 30820 raft_consensus.cc:2272] T 88a84a8ea71c4258b378603a0827a768 P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.437812 30820 tablet_replica.cc:333] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.440596  8593 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.449831  8593 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:08.451975  8593 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:08.454242 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:08.455163 30820 raft_consensus.cc:2243] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:08.456388 30820 raft_consensus.cc:2272] T 0d6c0fabd5a9486f95f088218e4ba65d P f05e4741d35c4cf3a31358c56e65b30c [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:08.463393 32084 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.463979 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.465166  8593 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:08.470546  8591 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.471109  8591 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.473122  8591 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:08.476830 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:21:08.478219 32623 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.479267 32623 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.480454  8587 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:08.481132  8587 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.481669  8587 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.483853  8587 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:08.483870 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:08.486169 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:21:08.493289 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.498633 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.503428  8556 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:08.511036  8592 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.511739  8592 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:08.517236 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:08.524668 32010 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.525233 32010 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.527215 30820 tablet_replica.cc:333] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c: stopping tablet replica
I20260709 12:21:08.520604  8592 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:08.529199  8560 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:08.531275 30820 raft_consensus.cc:2243] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.547014  8592 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.548100  8592 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.550710  8592 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:08.557355 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:08.558535 30820 raft_consensus.cc:2272] T 0292f21aab4a4843b2b424c3497ddced P f05e4741d35c4cf3a31358c56e65b30c [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:08.562477 32010 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.562980 32010 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.564502  8560 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:08.571609  8588 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.572273  8588 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:08.574522  8588 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:08.575618 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:08.576354 32042 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:21:08.577744 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:08.578544  8588 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:21:08.579047  8588 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.579459  8588 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:21:08.580650 32084 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:08.592149 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.601639  8588 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:08.604213  8588 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:08.605099  8587 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.605648  8587 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.607918 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:08.608572 32042 raft_consensus.cc:3060] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:21:08.611663 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.621598  8587 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:08.622112 32042 raft_consensus.cc:2468] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:21:08.624387 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:08.625391 32084 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:08.625886 32042 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:21:08.626495  8588 raft_consensus.cc:2804] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:21:08.627596 32623 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.629004 32623 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.629099  8588 raft_consensus.cc:697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:08.630059  8587 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:08.630499  8587 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.631033  8587 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:21:08.626806 32084 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.630338  8588 consensus_queue.cc:237] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:08.644120  8587 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:08.661749 32623 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.667249 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:08.667261  8587 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:08.668129 32042 raft_consensus.cc:3060] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:08.675565 32042 raft_consensus.cc:2468] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:21:08.689723 32623 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:21:08.691627 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.693817  8556 raft_consensus.cc:2804] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:08.694449  8556 raft_consensus.cc:697] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:08.694555  8591 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.695206  8591 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.695375  8556 consensus_queue.cc:237] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.697359  8591 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:08.706633 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.709343 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:21:08.718961 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.721086 31870 catalog_manager.cc:5697] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 23 to 24, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.722090  8592 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.730904  8592 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.733260  8592 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:08.736573 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.737282 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:08.737962 32654 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:21:08.723662  8536 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.739468  8536 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.729926 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.741271  8591 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:21:08.745494 32011 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:08.749471 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:08.750139 32654 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 18.
I20260709 12:21:08.751616 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:08.752635  8592 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:08.753057  8592 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.753412  8592 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:21:08.761644 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.762123  8560 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:21:08.763077  8560 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.763504  8560 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:21:08.742096  8536 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:08.776863  8560 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.784746  8560 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:08.795727 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8709b6ab6ebf4e279d1fe057a1ac58b5" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:08.796648 32654 raft_consensus.cc:3060] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:21:08.806562 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.806872 32654 raft_consensus.cc:2468] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:21:08.808115 32011 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:21:08.808373 32010 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.808990  8536 raft_consensus.cc:2804] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:21:08.809769  8536 raft_consensus.cc:697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:08.810871  8536 consensus_queue.cc:237] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:08.813289 32010 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:08.816856 32010 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.829381 31864 catalog_manager.cc:5697] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 22 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 23 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.839761  8597 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:21:08.840508 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.840415  8597 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.842914  8597 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:08.843801 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:08.844483 32654 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
W20260709 12:21:08.845636 32010 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.846261 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.847091  8597 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:08.847455  8597 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.847831  8597 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:21:08.857113 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.870574  8592 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.909174  8592 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:08.915212 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:21:08.916086 32010 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:08.922024 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.922640 32653 raft_consensus.cc:3060] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:08.924635  8597 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.931039 32653 raft_consensus.cc:2468] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:08.932734 32011 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.943262  8592 raft_consensus.cc:2804] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:21:08.944664 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 52870.4us
W20260709 12:21:08.947455 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:08.953466 32010 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.954695  8597 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:08.956367 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:08.944347  8592 raft_consensus.cc:697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:08.953709 31869 catalog_manager.cc:5697] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 18 to 19, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 19 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.960148  8592 consensus_queue.cc:237] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.943262  8601 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.971920  8601 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:08.976979  8601 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:08.981777 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:08.982461 32119 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
W20260709 12:21:08.983198 32010 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:08.990000 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:08.993052  8593 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:08.993597  8593 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:08.996919  8593 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:08.997907 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:21:08.998739 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 53026.2us
W20260709 12:21:09.001541 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.003558  8536 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:21:09.004052  8536 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.004433  8536 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:09.007200 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca59f0a11104ab882fac419c003e8b7" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:09.007766 32653 raft_consensus.cc:3060] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:21:09.017412 32084 leader_election.cc:336] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:09.017910 32084 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.017153 32653 raft_consensus.cc:2468] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:09.019650  8593 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:09.020895 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.021507 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.023334 32011 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.035948  8606 raft_consensus.cc:2804] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:21:09.036581  8606 raft_consensus.cc:697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:09.037391  8606 consensus_queue.cc:237] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.037899 32042 raft_consensus.cc:1275] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:21:09.041239  8588 consensus_queue.cc:1048] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:21:09.045080 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.049818 31935 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:09.035894  8536 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:21:09.067028 32010 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:09.077854 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.080778 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.083243 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bd59e23a6d4e1f9674d7662259db2f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:09.083834 32119 raft_consensus.cc:3060] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:09.086410  8536 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.087774  8593 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.088304  8593 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:09.061758 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.090368 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.091714 32119 raft_consensus.cc:2468] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:21:09.090052  8604 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.092760  8604 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.095899  8604 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.095521 31869 catalog_manager.cc:5697] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 27 to 28, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 28 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:09.103246 32623 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:09.103981 31869 catalog_manager.cc:5697] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 27 to 28, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 28 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.096377 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:09.098549 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.108316  8604 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.108881  8604 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.098830  8593 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:09.100036 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:09.112211 32654 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 26.
I20260709 12:21:09.112305 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:09.100904 32013 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.112907 32115 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:21:09.114485 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:09.114574 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:09.115310  8556 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:21:09.115931  8556 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.117147  8579 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:21:09.117406  8556 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:09.117682  8579 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.118109  8579 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:09.119148  8536 raft_consensus.cc:2804] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:21:09.122170  8536 raft_consensus.cc:697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
W20260709 12:21:09.125046 32623 leader_election.cc:336] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:09.126250  8579 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.128110  8536 consensus_queue.cc:237] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.133108 32119 raft_consensus.cc:2393] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 in current term 27: Already voted for candidate 38c927f0ee104b57972899f5b27aa99b in this term.
I20260709 12:21:09.138233 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:21:09.143146 32084 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:09.142688 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.137050  8604 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.142438 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:09.140300  8556 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.140753  8603 raft_consensus.cc:3060] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:09.153410 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:09.153949 32119 raft_consensus.cc:3060] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:09.155572  8556 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.161180 32119 raft_consensus.cc:2468] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:21:09.162281 32626 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:09.163105  8556 raft_consensus.cc:2804] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:21:09.163597  8556 raft_consensus.cc:697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:09.164115  8606 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.164672  8606 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.167757  8556 consensus_queue.cc:237] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:09.169685 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.170279 32623 leader_election.cc:336] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:09.142729  8579 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:09.179589 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:09.189896 32084 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:09.191262 32010 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:09.191885 32010 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.194587  8592 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:09.181248  8606 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.210425  8603 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:21:09.213177 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.220902 32653 raft_consensus.cc:2468] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:21:09.225741 32085 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.228358  8579 raft_consensus.cc:2804] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:21:09.228915  8579 raft_consensus.cc:697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:09.229813  8579 consensus_queue.cc:237] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:09.234371 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.234992 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.250543 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.251174 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.251600 31864 catalog_manager.cc:5697] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 26 to 27, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 27 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.253823 31870 catalog_manager.cc:5697] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 26 to 27, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 27 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.265740 31869 catalog_manager.cc:5697] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 24 to 25, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 25 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:09.261929 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:09.269816 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.276603  8579 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.277197  8579 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.281116  8579 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:09.283015 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:21:09.294605 32084 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:09.295053 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.295755  8611 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:09.319211 32039 raft_consensus.cc:1275] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:21:09.325747 31936 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 66679.2us
I20260709 12:21:09.330670  8611 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.331331  8611 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.334796 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:09.335783  8611 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:09.354635 32654 raft_consensus.cc:1275] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:21:09.357389  8606 consensus_queue.cc:1048] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:21:09.362226  8579 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.362826  8579 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.365259  8579 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:09.379390  8604 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.379984  8604 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.382186  8604 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.384372 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:09.385062 32115 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:21:09.385294 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:09.386112 32039 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:21:09.386238 32626 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:09.388384  8604 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:21:09.388918  8604 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.389458  8604 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:21:09.387725 32084 leader_election.cc:336] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:09.402943 32084 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:09.412578 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:09.408268  8604 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.418697  8604 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:09.422626  8593 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:09.422678 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:09.423254  8593 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.423707  8593 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:21:09.427583 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.413007  8611 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:09.431347 32115 raft_consensus.cc:3060] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:21:09.433568 32084 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:09.436111  8556 consensus_queue.cc:1048] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:09.440080 32115 raft_consensus.cc:2468] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 26.
I20260709 12:21:09.441052  8580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:47544 (request call id 1958) took 2782 ms. Trace:
I20260709 12:21:09.443557  8580 rpcz_store.cc:276] 0709 12:21:06.658140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:06.658632 (+   492us) service_pool.cc:224] Handling call
0709 12:21:06.858492 (+199860us) ts_tablet_manager.cc:934] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
0709 12:21:09.440959 (+2582467us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":39338,"tablet-copy.queue_time_us":174272,"thread_start_us":178748,"threads_started":2}
W20260709 12:21:09.445565 32624 connection.cc:466] server connection from 127.0.0.1:47544 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:47544 (request call id 1958) could send its response
I20260709 12:21:09.445624 32626 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:09.452720  8603 raft_consensus.cc:2804] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:21:09.454561  8603 raft_consensus.cc:697] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:09.455510  8603 consensus_queue.cc:237] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.473747  8593 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.480830 32653 raft_consensus.cc:1275] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:21:09.483521 32623 leader_election.cc:336] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:09.483886 32653 raft_consensus.cc:1275] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:21:09.486537  8601 consensus_queue.cc:1048] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:21:09.488816  8601 consensus_queue.cc:1048] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:21:09.496228  8593 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:09.498744 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:09.499377 32039 raft_consensus.cc:3060] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:09.509168 32039 raft_consensus.cc:2468] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:21:09.512706 32115 raft_consensus.cc:1275] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:21:09.558982 32623 leader_election.cc:336] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:09.559804 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 44735.3us
W20260709 12:21:09.561538 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 46780.3us
I20260709 12:21:09.575508 32084 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:09.594970  8627 raft_consensus.cc:2804] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:21:09.607172  8601 consensus_queue.cc:1048] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:21:09.622582  8611 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.623176  8611 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.639384  8588 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.640347  8588 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.643280  8588 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:09.647423  8611 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:09.680243  8606 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.681000  8606 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.683523  8606 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:09.687633 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 106520us
W20260709 12:21:09.701857 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 118857us
I20260709 12:21:09.703588  8627 raft_consensus.cc:697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:09.708384 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:09.709108 32119 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 27.
I20260709 12:21:09.713310 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:09.714236 32039 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:21:09.715476 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:09.720809 32112 raft_consensus.cc:1275] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:21:09.730505 31869 catalog_manager.cc:5697] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 25 to 26, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 26 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.722699  8611 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.751770  8611 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:09.754068  8611 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:09.770352 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 50535.7us
I20260709 12:21:09.772343  8597 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.773164 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:09.773015  8597 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.773742  8627 consensus_queue.cc:237] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:09.815666 32653 raft_consensus.cc:1275] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
W20260709 12:21:09.801920 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 90609.6us
I20260709 12:21:09.819308 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:09.820572  8593 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:09.854956 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:09.855916 32042 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
I20260709 12:21:09.863096  8628 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.863719  8628 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.865825  8628 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:09.868255 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.875030 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:09.858417  8593 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
W20260709 12:21:09.899312 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:09.882011  8638 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:09.899781  8593 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:09.913966  8643 consensus_queue.cc:1048] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.033s
I20260709 12:21:09.904521  8638 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.935348  8638 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:09.943835  8597 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:09.950320 32084 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:09.958905  8637 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:09.959618  8637 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:09.991170 31864 catalog_manager.cc:5697] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 27 to 28, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 28 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:09.993201 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.996292 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 274156us
I20260709 12:21:10.001667  8637 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:10.023846 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:10.024606 32042 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:21:10.027637 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:10.028296 32654 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:21:10.038514 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:10.039435  8592 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:10.039848  8592 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:10.040215  8592 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:10.044652 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:10.045327 32119 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:10.047624 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:10.058269  8630 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:10.059214  8630 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:10.059609  8630 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:10.060127  8593 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:10.010016  8638 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:10.067670  8634 consensus_queue.cc:1048] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:21:10.068326  8638 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:10.071724 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:10.072885  8631 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:10.073307  8631 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election (detected failure of leader f05e4741d35c4cf3a31358c56e65b30c)
I20260709 12:21:10.073668  8631 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:10.075572 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d45de750cd4fd1a0cdd13ad4f731fb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:10.076179 32119 raft_consensus.cc:3060] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:10.062264  8593 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:10.086122 32084 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:10.087406 32084 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:10.088101 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:10.091773  8592 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:10.092166  8630 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:10.094732 32119 raft_consensus.cc:2468] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:10.095338  8592 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:10.099015 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb363b96dbf430998ab5641d3b03f46" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:10.099618 32653 raft_consensus.cc:3060] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:10.095541  8631 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:10.096174  8630 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:10.098362 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.101938 32084 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:10.097836 32013 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:10.102284 32112 raft_consensus.cc:1275] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:21:10.100811 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 66287.4us
I20260709 12:21:10.101514  8611 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:10.115319 32653 raft_consensus.cc:2468] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:21:10.116494 32011 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:10.118683 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:10.119323 32039 raft_consensus.cc:3060] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:10.120107  8603 consensus_queue.cc:1048] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:10.121592 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:10.122340  8634 raft_consensus.cc:2764] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:21:10.124294  8640 raft_consensus.cc:2804] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:10.124867  8640 raft_consensus.cc:697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:10.128381 32039 raft_consensus.cc:2468] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 29.
I20260709 12:21:10.135432 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:10.136811 32039 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:10.116140  8638 raft_consensus.cc:2804] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:21:10.125800  8640 consensus_queue.cc:237] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:10.148294 32039 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:21:10.150576  8638 raft_consensus.cc:697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:10.172974 32084 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:21:10.177449 32010 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:10.178467 32010 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:10.179865 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.179562  8638 consensus_queue.cc:237] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:21:10.180570 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.181057 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.181635 32010 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:10.184860 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:21:10.187727 32084 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:10.188553 32084 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:10.195657  8630 raft_consensus.cc:2804] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:10.196262  8630 raft_consensus.cc:697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:10.197108  8630 consensus_queue.cc:237] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:10.197613  8639 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:10.198159  8639 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:10.199057  8639 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:10.208488  8631 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:10.209547 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:10.210196 32112 raft_consensus.cc:3060] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:21:10.213251 32010 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:10.214432 32010 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:10.223869 32112 raft_consensus.cc:2468] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 29.
I20260709 12:21:10.225361 32013 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:10.235471  8592 raft_consensus.cc:2804] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:10.237960  8592 raft_consensus.cc:697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 29 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:10.238849  8592 consensus_queue.cc:237] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:21:10.288344 32010 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:10.356433 30820 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 31993)
I20260709 12:21:10.351976 31864 catalog_manager.cc:5697] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 21 to 22, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:10.375751  8664 log.cc:927] Time spent T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:21:10.399837 31869 catalog_manager.cc:5697] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 28 to 29, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 29 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:10.409309 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.401983 31869 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 23, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:10.413723 32039 raft_consensus.cc:1275] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:21:10.415134  8630 consensus_queue.cc:1048] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:21:10.417076 31864 catalog_manager.cc:5697] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 27 to 28, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 28 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:10.430763 31864 catalog_manager.cc:5697] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 28 to 29, leader changed from f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 29 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:10.510416 32115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33516
I20260709 12:21:10.511073 32115 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Received request to transfer leadership
I20260709 12:21:10.524241 32115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33516
I20260709 12:21:10.524895 32115 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Received request to transfer leadership
I20260709 12:21:10.547489 32653 raft_consensus.cc:1275] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:21:10.550252  8640 consensus_queue.cc:1048] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:21:10.574363 32115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33516
I20260709 12:21:10.574950 32115 raft_consensus.cc:606] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Received request to transfer leadership
W20260709 12:21:10.586629 31918 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:21:10.587061 31918 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 0d6c0fabd5a9486f95f088218e4ba65d already in progress
W20260709 12:21:10.587363 31918 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 0d6c0fabd5a9486f95f088218e4ba65d already in progress
W20260709 12:21:10.623595 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.624753 32085 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d6c0fabd5a9486f95f088218e4ba65d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.641969 32039 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:21:10.643546  8627 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:10.659255  8639 raft_consensus.cc:993] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: : Instructing follower 3836f9b5cd36488da56d4540dc9d80ec to start an election
I20260709 12:21:10.659804  8639 raft_consensus.cc:1081] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Signalling peer 3836f9b5cd36488da56d4540dc9d80ec to start an election
W20260709 12:21:10.669028 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.673759 32084 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.676326 32042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d"
dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
 from {username='slave'} at 127.0.0.1:58398
I20260709 12:21:10.677946 32042 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:10.678370 32042 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:21:10.686879 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.699409 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.706672 32042 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
W20260709 12:21:10.721248 32010 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:10.721462 32115 raft_consensus.cc:1275] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:21:10.722364 32042 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Requested vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:10.723708  8640 consensus_queue.cc:1048] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:10.727384 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:10.727953 32115 raft_consensus.cc:3055] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:21:10.728247 32115 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:10.729023 32115 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:10.731499 32115 raft_consensus.cc:3060] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:21:10.782651 32115 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 24.
I20260709 12:21:10.809613 32013 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:10.811259  8631 raft_consensus.cc:2804] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:21:10.811756  8631 raft_consensus.cc:697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:10.812626  8631 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:10.817282 32117 raft_consensus.cc:1275] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:21:10.818770  8592 consensus_queue.cc:1048] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:10.865602 32039 raft_consensus.cc:1275] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:21:10.869370  8639 consensus_queue.cc:1048] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:10.888100 31869 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 23 to 24, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 24 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:11.161352 32084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111) [suppressed 353 similar messages]
W20260709 12:21:11.195530 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.251334 32011 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d6c0fabd5a9486f95f088218e4ba65d. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.332082 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.357286 30820 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:11.366433 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.376520 32116 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:21:11.378610  8678 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:21:11.380659 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.430980 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.450639 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.455529 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.465564 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.466059 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.475970 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.494796 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.495410 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.647964 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.628923 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.660620 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.685528 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.704159 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:11.717237  8679 ts_tablet_manager.cc:933] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:11.726392 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.728928 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.755695 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.760596 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.761040 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:11.770888 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:11.779620  8679 tablet_copy_client.cc:323] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.2:40293
W20260709 12:21:11.782198 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:11.783780 32117 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:11.803685 32057 tablet_copy_service.cc:140] P 3836f9b5cd36488da56d4540dc9d80ec: Received BeginTabletCopySession request for tablet 0d6c0fabd5a9486f95f088218e4ba65d from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:54754)
I20260709 12:21:11.804339 32057 tablet_copy_service.cc:161] P 3836f9b5cd36488da56d4540dc9d80ec: Beginning new tablet copy session on tablet 0d6c0fabd5a9486f95f088218e4ba65d from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:54754: session id = ec7f26ea46944ea498ad253b1b1819b0-0d6c0fabd5a9486f95f088218e4ba65d
W20260709 12:21:11.811506 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:11.814464 32039 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:21:11.816284  8630 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:21:11.825178 32085 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399): Couldn't send request to peer ec7f26ea46944ea498ad253b1b1819b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b88d8f995fcc4faea19a190f8dd669ab. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.842909 32057 tablet_copy_source_session.cc:215] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:11.856137  8679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d6c0fabd5a9486f95f088218e4ba65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:11.862562  8611 raft_consensus.cc:2955] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
W20260709 12:21:11.867594 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.868182 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:11.870107 32038 raft_consensus.cc:2955] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:11.898411  8679 tablet_copy_client.cc:806] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:11.899281  8679 tablet_copy_client.cc:670] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
W20260709 12:21:11.916872 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:11.929561  8679 tablet_copy_client.cc:538] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:21:11.930106 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.930480 31864 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index -1 to 15, NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) added. New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } } }
I20260709 12:21:11.949625  8679 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
W20260709 12:21:11.958850 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:12.088477 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:12.122628  8679 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=1 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:21:12.128788 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:12.143493 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.144984 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:12.146265  8679 tablet_bootstrap.cc:492] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:21:12.147234  8679 ts_tablet_manager.cc:1403] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.198s	user 0.061s	sys 0.026s
I20260709 12:21:12.151050  8679 raft_consensus.cc:359] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:12.152909  8679 raft_consensus.cc:740] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:21:12.153797  8679 consensus_queue.cc:260] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:12.166500 32660 raft_consensus.cc:1217] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Deduplicated request from leader. Original: 23.16->[24.17-24.17]   Dedup: 24.17->[]
W20260709 12:21:12.182512 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:12.188853  8679 ts_tablet_manager.cc:1434] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:21:12.191798 32057 tablet_copy_service.cc:342] P 3836f9b5cd36488da56d4540dc9d80ec: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0d6c0fabd5a9486f95f088218e4ba65d received from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:12.192283 32057 tablet_copy_service.cc:434] P 3836f9b5cd36488da56d4540dc9d80ec: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-0d6c0fabd5a9486f95f088218e4ba65d on tablet 0d6c0fabd5a9486f95f088218e4ba65d with peer ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:21:12.244505 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.357911 30820 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 31993)
I20260709 12:21:12.405397  8679 ts_tablet_manager.cc:933] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: Initiating tablet copy from peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:12.422734  8679 tablet_copy_client.cc:323] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Beginning tablet copy session from remote peer at address 127.30.25.3:35255
I20260709 12:21:12.425097 32130 tablet_copy_service.cc:140] P 38c927f0ee104b57972899f5b27aa99b: Received BeginTabletCopySession request for tablet b88d8f995fcc4faea19a190f8dd669ab from peer ec7f26ea46944ea498ad253b1b1819b0 ({username='slave'} at 127.0.0.1:34592)
I20260709 12:21:12.425652 32130 tablet_copy_service.cc:161] P 38c927f0ee104b57972899f5b27aa99b: Beginning new tablet copy session on tablet b88d8f995fcc4faea19a190f8dd669ab from peer ec7f26ea46944ea498ad253b1b1819b0 at {username='slave'} at 127.0.0.1:34592: session id = ec7f26ea46944ea498ad253b1b1819b0-b88d8f995fcc4faea19a190f8dd669ab
I20260709 12:21:12.435480 32130 tablet_copy_source_session.cc:215] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:12.458294  8679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88d8f995fcc4faea19a190f8dd669ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:12.494750 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.505244  8679 tablet_copy_client.cc:806] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:12.506008  8679 tablet_copy_client.cc:670] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:12.514451  8679 tablet_copy_client.cc:538] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:12.532301  8679 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap starting.
W20260709 12:21:12.534471 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.672953  8606 raft_consensus.cc:1064] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:21:12.676999  8606 consensus_queue.cc:237] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 16, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:12.701190 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 51536.3us
W20260709 12:21:12.763756 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.773212 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 84033.7us
W20260709 12:21:12.850479 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 71493.6us
I20260709 12:21:12.886390 32115 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 12:21:12.888036 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 104617us
I20260709 12:21:12.896369  8700 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:21:12.927943 32653 raft_consensus.cc:1275] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEARNER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:21:12.931406  8679 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:12.956029  8679 tablet_bootstrap.cc:492] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: Bootstrap complete.
I20260709 12:21:12.958983  8679 ts_tablet_manager.cc:1403] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: Time spent bootstrapping tablet: real 0.427s	user 0.082s	sys 0.065s
I20260709 12:21:12.960455  8640 consensus_queue.cc:1048] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:21:12.967356 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:12.969427 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:13.018358  8606 raft_consensus.cc:2955] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Committing config change with OpId 24.18: config changed from index 15 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:13.000679  8679 raft_consensus.cc:359] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:13.021930  8679 raft_consensus.cc:740] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Initialized, Role: LEARNER
I20260709 12:21:13.023006  8679 consensus_queue.cc:260] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: NON_VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: true } }
I20260709 12:21:13.030800 32660 raft_consensus.cc:2955] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Committing config change with OpId 24.18: config changed from index 15 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:13.039912 32115 raft_consensus.cc:2955] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Committing config change with OpId 24.18: config changed from index 15 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:21:13.056056 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:13.057925 31864 catalog_manager.cc:5697] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 15 to 18, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.056197  8679 ts_tablet_manager.cc:1434] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: Time spent starting tablet: real 0.088s	user 0.004s	sys 0.004s
I20260709 12:21:13.075680 32130 tablet_copy_service.cc:342] P 38c927f0ee104b57972899f5b27aa99b: Request end of tablet copy session ec7f26ea46944ea498ad253b1b1819b0-b88d8f995fcc4faea19a190f8dd669ab received from {username='slave'} at 127.0.0.1:34592
I20260709 12:21:13.076193 32130 tablet_copy_service.cc:434] P 38c927f0ee104b57972899f5b27aa99b: ending tablet copy session ec7f26ea46944ea498ad253b1b1819b0-b88d8f995fcc4faea19a190f8dd669ab on tablet b88d8f995fcc4faea19a190f8dd669ab with peer ec7f26ea46944ea498ad253b1b1819b0
W20260709 12:21:13.092943 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:13.093794 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:13.164979 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:13.358531 30820 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:13.405783 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:13.593763 32660 raft_consensus.cc:1217] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Deduplicated request from leader. Original: 26.14->[26.15-26.15]   Dedup: 26.15->[]
W20260709 12:21:13.657346 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.672545 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.827102 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.834581 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.899009 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.930279 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.944016 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.953303 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.967262 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.986886 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:13.987572 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.026263 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.092579 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:14.117358  8711 raft_consensus.cc:1064] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: attempting to promote NON_VOTER ec7f26ea46944ea498ad253b1b1819b0 to VOTER
I20260709 12:21:14.120412  8711 consensus_queue.cc:237] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 13, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:14.131922 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.133085 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.153489 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:14.162125 32660 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEARNER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:21:14.163770  8699 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:14.167064 32038 raft_consensus.cc:1275] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:21:14.179533 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.189601  8699 consensus_queue.cc:1048] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:21:14.200196 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:14.213142 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.218545 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.220573 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:14.260632  8699 raft_consensus.cc:2955] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:14.263731 32038 raft_consensus.cc:2955] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:21:14.272516 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:14.297785 32653 raft_consensus.cc:2955] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
I20260709 12:21:14.305785 31869 catalog_manager.cc:5697] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: config changed from index 15 to 16, ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) changed from NON_VOTER to VOTER. New cstate: current_term: 26 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } } }
W20260709 12:21:14.318025 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.339370 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.359601 30820 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:14.392027 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.446956 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.454777 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.461645 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.503954 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.518288 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.532667 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.542456 32626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 46293.9us
W20260709 12:21:14.602663 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:14.622932 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.807091 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.885753 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.891410 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.036514 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:15.253985 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:15.328372 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.360412 30820 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:15.400780 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.464584 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.560199 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.710182 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.767587 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.978726 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 64389.3us
W20260709 12:21:16.074503 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.110491 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.194991 32623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111) [suppressed 441 similar messages]
W20260709 12:21:16.226907 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.240823 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.267820 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.335673 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.362095 30820 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:16.403365 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.416777 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.556180 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.563146 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.565975 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.569166 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.574512 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.581128 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.694000 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.715889 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:16.730513 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.733882 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.756497 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.784693 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.833702 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.845331 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.854135 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.868649 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.870863 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.907922 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.916234 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:16.942394 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.007624 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.117581 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:17.140420 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:17.153065 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:17.293404 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:17.363472 30820 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:17.511415 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.512890 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:17.515373 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:17.558137 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.607875 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.689925 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.836175 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.853566 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:18.201395 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:18.226419 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:18.264549 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:18.364010 30820 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:18.420351 32142 debug-util.cc:398] Leaking SignalData structure 0x7b0800227c00 after lost signal to thread 30823
W20260709 12:21:18.421564 32142 debug-util.cc:398] Leaking SignalData structure 0x7b0800407b80 after lost signal to thread 31904
W20260709 12:21:18.422955 32142 debug-util.cc:398] Leaking SignalData structure 0x7b0800227660 after lost signal to thread 32071
W20260709 12:21:18.424108 32142 debug-util.cc:398] Leaking SignalData structure 0x7b08002273e0 after lost signal to thread 32145
W20260709 12:21:18.425338 32142 debug-util.cc:398] Leaking SignalData structure 0x7b080015daa0 after lost signal to thread 32684
W20260709 12:21:18.680549 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:18.764277 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 55350.6us
W20260709 12:21:18.810623 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:18.834262 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:18.976194 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:18.976928 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.050841 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.062292 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.052237 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.138792 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.226150 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.226822 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.235400 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.246201 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.265920 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.268605 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.325788 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.330209 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.348649 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.358889 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.364744 30820 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:19.388967 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.395895 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.396471 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:19.415980 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.423054 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.422703 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.485932 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.658196 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.670542 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.717799 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.720204 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:19.772461 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.803126 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:20.075582 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.157054 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:20.177426 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:20.207232 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.232652 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 55515.4us
W20260709 12:21:20.233963 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.236001 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:20.236552 32013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35255, user_credentials={real_user=slave}} blocked reactor thread for 58326.4us
W20260709 12:21:20.263420 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.366170 30820 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:20.647174 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.762825 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.763811 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.766825 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.929560 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.084298 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 59352.6us
W20260709 12:21:21.202328 32084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111) [suppressed 435 similar messages]
W20260709 12:21:21.205770 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.367097 30820 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:21.367772 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.408098 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.408756 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.410221 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.421610 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:21.424145 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.591851 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.708828 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.712448 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:21.712952 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.731297 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.733457 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:21.781682 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.807735 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.821519 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:21.885365 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.885911 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.890290 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.898874 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:21.914466 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:21.935794 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.119146 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.178571 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.219545 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:40293, user_credentials={real_user=slave}} blocked reactor thread for 62004.4us
W20260709 12:21:22.248140 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.248875 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.252349 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.253012 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.259676 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:22.274173 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.320731 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.349282 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.368192 30820 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 31993)
W20260709 12:21:22.375926 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.413380 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.422163 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.550761 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.679092 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.701768 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:22.801046 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:23.594738 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:23.902810  8770 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:23.903990  8770 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:23.907173  8770 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:24.050413  8769 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.051208  8769 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:21:24.053241 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:24.059499 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:24.062137  8769 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.063397  8770 raft_consensus.cc:493] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.064217  8770 raft_consensus.cc:515] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.066973  8770 leader_election.cc:290] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:24.104285 32010 consensus_peers.cc:597] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:24.119189  8770 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.120051  8770 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.123121  8770 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.128198  8769 raft_consensus.cc:493] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.128914  8769 raft_consensus.cc:515] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:21:24.129698 32010 leader_election.cc:336] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:24.130216 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:24.131266  8752 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:24.131424  8774 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.132045  8774 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.132522  8769 leader_election.cc:290] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.133463 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16a6933cca8f4bce92ae16ff487ca7c1" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:24.171675  8774 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:24.762668 32085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.007s	sys 0.002s
W20260709 12:21:24.763163 32085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.007s	sys 0.002s
I20260709 12:21:24.767108  8781 raft_consensus.cc:493] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.767498  8773 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.767845  8781 raft_consensus.cc:515] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.768150  8773 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.770052  8781 leader_election.cc:290] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.770215  8773 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.858148  8776 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:24.858886  8776 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.861521  8776 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.865619  8783 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.866429  8783 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.872198  8783 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.875754  8784 raft_consensus.cc:493] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.876502  8784 raft_consensus.cc:515] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.878957  8784 leader_election.cc:290] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.888629  8782 raft_consensus.cc:493] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.889274  8773 raft_consensus.cc:493] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.889374  8782 raft_consensus.cc:515] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.890022  8773 raft_consensus.cc:515] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.892549  8782 leader_election.cc:290] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.892798  8773 leader_election.cc:290] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.893260  8782 raft_consensus.cc:493] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.888811  8781 raft_consensus.cc:493] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.894810  8781 raft_consensus.cc:515] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.895610  8784 raft_consensus.cc:493] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.896175  8784 raft_consensus.cc:515] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.897548  8781 leader_election.cc:290] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.898480  8784 leader_election.cc:290] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.893899  8782 raft_consensus.cc:515] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.902338  8782 leader_election.cc:290] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.899912  8785 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:21:24.904211 32084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.315s	user 0.029s	sys 0.083s
I20260709 12:21:24.903976  8785 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:24.904636 32084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.315s	user 0.031s	sys 0.084s
I20260709 12:21:24.907812  8781 raft_consensus.cc:493] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.909406  8782 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.910108  8782 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.908370  8781 raft_consensus.cc:515] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.912838  8782 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.915318  8781 leader_election.cc:290] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.915894  8784 raft_consensus.cc:493] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.916455  8784 raft_consensus.cc:515] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:24.920394  8782 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
W20260709 12:21:24.921252 32681 debug-util.cc:398] Leaking SignalData structure 0x7b0800152c80 after lost signal to thread 30823
I20260709 12:21:24.921187  8782 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:24.922899 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08002df2c0 after lost signal to thread 31904
I20260709 12:21:24.924170  8782 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:24.924389 32681 debug-util.cc:398] Leaking SignalData structure 0x7b0800245220 after lost signal to thread 32071
I20260709 12:21:24.921335  8788 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.926580  8782 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.927155  8782 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.929306  8782 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:24.926062 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08003331c0 after lost signal to thread 32145
W20260709 12:21:24.930924 32681 debug-util.cc:398] Leaking SignalData structure 0x7b08001bac40 after lost signal to thread 32684
I20260709 12:21:24.932047  8785 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.925601  8788 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.936919  8784 leader_election.cc:290] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.944492  8789 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:24.979502  8784 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.945179  8789 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:24.982502  8787 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.990229  8785 raft_consensus.cc:493] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:24.982544  8784 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:24.990916  8785 raft_consensus.cc:515] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.991879  8789 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:24.992861  8784 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.993911  8776 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:24.994546  8776 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:24.996759  8776 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:24.998216  8785 leader_election.cc:290] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:24.989959  8787 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.001493  8787 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.003414  8788 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.010682  8783 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:25.010915  8776 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:25.012013  8787 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.017200  8776 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.019496  8776 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:25.019999  8776 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.020426  8776 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.017612  8787 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.022471  8789 raft_consensus.cc:493] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.023068  8789 raft_consensus.cc:515] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.024098  8787 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.015084  8783 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.026437  8783 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:25.026990  8776 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.016009  8794 raft_consensus.cc:493] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.028076  8794 raft_consensus.cc:515] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } attrs { replace: true } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.028945  8790 raft_consensus.cc:493] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.029490  8790 raft_consensus.cc:515] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.031152  8794 leader_election.cc:290] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.031657  8790 leader_election.cc:290] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.032032  8756 raft_consensus.cc:493] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.032785  8756 raft_consensus.cc:515] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.036644  8785 raft_consensus.cc:493] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.037420 32655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.037616 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a06d9856d134874b3144fd2abdbfc71" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.037235  8785 raft_consensus.cc:515] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:25.038120 32655 raft_consensus.cc:2468] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 26.
I20260709 12:21:25.038967 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.039306  8787 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.039635 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.039887  8787 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.040248  8785 leader_election.cc:290] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.040997 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.041703 32654 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:21:25.041917 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3695041c497f47468894b0baf52fcfd0" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.042680  8788 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.043284 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c52a06827f24bb6bc1851474fc80e53" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.043367  8788 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.040266 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.044659 32652 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 21.
I20260709 12:21:25.040330 32657 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:25.045579  8788 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.046633  8784 raft_consensus.cc:493] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.047112  8784 raft_consensus.cc:515] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:25.042073  8787 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.049403  8784 leader_election.cc:290] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.046103  8788 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.050594  8788 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.052309  8783 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.053153  8776 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.053062  8783 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.053704  8776 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.055387  8783 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.052266  8789 leader_election.cc:290] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.057387  8776 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.054212  8788 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:25.064082 32010 consensus_peers.cc:597] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.068101 32010 leader_election.cc:336] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.069173 32010 leader_election.cc:336] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.070739 32010 consensus_peers.cc:597] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.071190 32010 consensus_peers.cc:597] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:25.071774 32010 leader_election.cc:336] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.072158 32010 leader_election.cc:304] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.073622  8784 raft_consensus.cc:2749] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:25.032162  8790 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.061609 32115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47764 (request call id 4783) took 1446 ms. Trace:
I20260709 12:21:25.081847 32115 rpcz_store.cc:276] 0709 12:21:23.614640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:25.060510 (+1445870us) service_pool.cc:224] Handling call
0709 12:21:25.060880 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:25.061103 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:25.061140 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:25.061195 (+    55us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:21:25.061240 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:25.061303 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:25.061426 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:25.082736  8794 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.080695 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.081270 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.080111 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.083604  8790 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.097791  8784 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.098500  8776 raft_consensus.cc:493] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.098423  8784 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.099102  8776 raft_consensus.cc:515] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.099510  8785 raft_consensus.cc:493] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.099673  8790 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.100076  8785 raft_consensus.cc:515] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.100595  8784 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.101279  8776 leader_election.cc:290] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.094560 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02ef57fadd044f68ba212dc2fe410f83" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.109223  8785 leader_election.cc:290] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.095525 32653 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 20.
I20260709 12:21:25.109385 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.096007 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.107124  8791 raft_consensus.cc:493] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.110944  8791 raft_consensus.cc:515] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:25.113296  8791 leader_election.cc:290] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.107836 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.107930 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.096526 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.120996  8794 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.123565  8756 leader_election.cc:290] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.127676  8794 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.129071 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.166715 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.128448 32046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad86bf28242f4576af3db683944d53a3" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.130483 32040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.130561 32045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.130853 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.131546 32039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ee45b18b714479af8291d9056afb9e" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.132020 32043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.132356 32047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "826cffce8bb540c8beb56cc00a702782" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.154484 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f78e3424864d319ee8da87f4848fe9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.145414 32113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.144263 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ae9fcf8eb748c6b0cb9703adc9f338" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.131592 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
W20260709 12:21:25.132481 32010 leader_election.cc:336] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.133729  8798 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.156411 32115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.140928 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.157194 32119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.133700 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.158170 32121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:25.158921 32084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 67386.1us
I20260709 12:21:25.159843 32118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.159900 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2f941a3c6cb4a188198a804d603db3b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.163722 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.168147 32040 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:21:25.175021 32114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.175638 32041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.177222 32120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88d8f995fcc4faea19a190f8dd669ab" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.178278 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "786db0eb7e9b431ba90c5ad880c46c9e" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.175040 32038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f1c1160177412eb547493e1f3c561f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.175056 32044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.180943 32044 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 27.
I20260709 12:21:25.175741 32114 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 19.
I20260709 12:21:25.182416  8798 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.185092  8798 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:25.185555 32010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 53109.1us
W20260709 12:21:25.186543 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:25.187165 32010 leader_election.cc:336] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.187671 32010 leader_election.cc:336] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.190749 32011 leader_election.cc:304] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.191725  8756 raft_consensus.cc:2749] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:25.192778 32011 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:25.193611  8756 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:25.194186  8756 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
W20260709 12:21:25.196298 32010 leader_election.cc:336] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.196950 32010 leader_election.cc:336] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.197602 32010 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.198069 32010 leader_election.cc:336] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.194641  8756 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:21:25.198849 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.199673 32010 consensus_peers.cc:597] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.200126 32010 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.198680 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 62686.3us
I20260709 12:21:25.202597 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e910ef3c0af41bf8d4fba5315d68eef" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.216642 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81129181ae3246c1bfb02954120c9b91" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.218290 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b3fd343e81436fa5bea82486abcc64" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.219878  8794 raft_consensus.cc:493] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.220456 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12a3b65ef9024a64b336b00ad51e294b" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:25.220853 32623 leader_election.cc:336] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.221568 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:25.220469  8794 raft_consensus.cc:515] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:25.222108 32623 leader_election.cc:336] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.193732 32011 leader_election.cc:304] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.228595  8794 leader_election.cc:290] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:25.230562 32013 leader_election.cc:304] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:21:25.230950 32084 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.231690 32084 leader_election.cc:336] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.232242 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:25.232741 32084 leader_election.cc:336] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.233295 32084 leader_election.cc:336] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.233814 32084 leader_election.cc:336] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.234341 32084 leader_election.cc:336] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.234829 32084 leader_election.cc:336] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.235332 32623 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.235414 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:25.236099 32084 leader_election.cc:336] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.236626 32084 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.237293 32626 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.238234 32626 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.238617  8796 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:25.241587  8756 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.242698 32042 raft_consensus.cc:1240] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:21:25.246034 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.246656  8797 raft_consensus.cc:2749] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:25.246878  8789 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.247002  8792 raft_consensus.cc:2749] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:25.247663 32011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 54048.8us
I20260709 12:21:25.248291 32084 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:25.248723 32013 leader_election.cc:304] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.249110  8789 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:21:25.249528  8789 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.249577  8792 raft_consensus.cc:2749] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:25.249997  8789 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:21:25.251391 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.255124  8798 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:25.257298 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c59a3d3077c4b8db6380a834f6ce3c0" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.248327 32011 leader_election.cc:304] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.258759  8792 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:25.262079 32626 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.262964  8792 raft_consensus.cc:2749] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:25.264324  8756 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.261039 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.264243 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.265560 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.270947 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.271688 32084 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:25.272111  8782 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:25.272539  8781 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:25.260352 32013 leader_election.cc:304] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.267593  8789 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.268473  8794 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:25.268605 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d27358a7bd3493a8877bd9ce8ae9055" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.272965  8781 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.274557  8781 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:25.275192  8792 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:25.275661  8792 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.276062  8792 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:25.281513  8791 raft_consensus.cc:2749] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:25.282047 32013 leader_election.cc:304] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.283175  8791 raft_consensus.cc:2749] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:25.283458 32013 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:25.283910  8789 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.284770  8797 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:21:25.289583  8797 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:25.290035  8797 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:21:25.290472 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.291352  8798 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:21:25.292644 32623 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.292956 32011 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.296535  8791 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:25.296962  8791 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.297312  8791 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:25.297497 32011 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.298367  8785 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:25.284895 32013 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.293498  8798 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:25.294384 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.294726  8782 consensus_queue.cc:1059] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:25.308365 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.321998  8788 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:25.322296  8782 raft_consensus.cc:3055] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:21:25.318969 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f93a3ac5b02416a8d1d2547d9a9c726" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.322990  8782 raft_consensus.cc:740] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:25.301506  8797 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.302376 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:25.323947  8782 consensus_queue.cc:260] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.337746  8782 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:21:25.309264 31901 debug-util.cc:398] Leaking SignalData structure 0x7b0800212c20 after lost signal to thread 30823
I20260709 12:21:25.338421  8797 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:25.339301 31901 debug-util.cc:398] Leaking SignalData structure 0x7b08003e11c0 after lost signal to thread 31904
W20260709 12:21:25.340698 31901 debug-util.cc:398] Leaking SignalData structure 0x7b08003297c0 after lost signal to thread 32071
W20260709 12:21:25.341938 31901 debug-util.cc:398] Leaking SignalData structure 0x7b080040a060 after lost signal to thread 32145
I20260709 12:21:25.342298 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.343458 32085 leader_election.cc:304] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.344936 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.305491  8791 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.347086  8791 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.325832 32653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2cc7077f01430da5a31385d674e266" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.327278 32657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e828882e6b1940ffb33d4c2f76b47fe9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.286571  8781 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.328850 32623 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.329142 32652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775fd46a41044b1ea06bc796ef1b233b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.329520  8796 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:25.352639  8798 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.352212 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:25.355258 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6c31e616d2c4874ac5a8e29453cc51b" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:25.355949 31901 debug-util.cc:398] Leaking SignalData structure 0x7b08001c7360 after lost signal to thread 32684
I20260709 12:21:25.356282 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec" is_pre_election: true
I20260709 12:21:25.330511 32659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.331596 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1612f8e575bd46cf8069fa1c21ca192b" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.332284 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4bfb2be45e4fc38e6622722ab10735" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.332581 32011 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.333596 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.334926 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f953fa305a42492da9de0fa8c18c43eb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.335597 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.345077  8773 raft_consensus.cc:2749] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:25.357268 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:25.289737  8792 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.358592 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:21:25.324575 32084 leader_election.cc:336] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:25.358183 32623 leader_election.cc:336] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.362478 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:25.364701 32623 leader_election.cc:304] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.361835  8791 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:21:25.365861  8791 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:25.366130 32013 leader_election.cc:304] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.366241  8791 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:25.366648 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.369253 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "193a74284231469f9229c961dfaff799" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.360711  8781 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.372953  8796 raft_consensus.cc:2749] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:25.374269 32623 leader_election.cc:336] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.375851 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0292f21aab4a4843b2b424c3497ddced" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.374157  8791 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.376269  8797 raft_consensus.cc:2749] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.376487 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:25.377491 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c82ba0f778d4ff98ba9cc8255f28e9d" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:25.378336 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.381335 32623 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.382750 32654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.384351 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.366787  8792 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.386579 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:25.385258  8791 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:25.388024 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.390735  8796 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:25.392030 32623 leader_election.cc:336] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.392087 32042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "3836f9b5cd36488da56d4540dc9d80ec"
I20260709 12:21:25.394240 32656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.397274 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.399168 32011 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.396512 32623 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.399487  8791 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:25.399623 32042 raft_consensus.cc:1240] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 20. Current term is 21. Ops: []
I20260709 12:21:25.402772  8796 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:25.390076 32116 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:21:25.401057 32040 raft_consensus.cc:1240] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 28. Current term is 29. Ops: []
W20260709 12:21:25.404407 32623 leader_election.cc:336] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.404991 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.405704  8792 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.407845 32040 raft_consensus.cc:1240] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Rejecting Update request from peer 38c927f0ee104b57972899f5b27aa99b for earlier term 21. Current term is 22. Ops: []
I20260709 12:21:25.405938 32013 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:25.410054  8788 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:25.410583  8788 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:25.411579  8788 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:25.424125 32010 consensus_peers.cc:597] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:25.425729  8789 consensus_queue.cc:1059] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:25.426982  8782 raft_consensus.cc:3055] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:21:25.427419  8782 raft_consensus.cc:740] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:25.428344  8782 consensus_queue.cc:260] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.406991 32011 leader_election.cc:304] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.433424  8786 consensus_queue.cc:1059] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:21:25.433838 32011 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.433907  8797 raft_consensus.cc:2749] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.434742  8781 raft_consensus.cc:3055] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:21:25.434840  8792 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:25.434998  8782 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:25.435276  8781 raft_consensus.cc:740] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:25.436188  8796 raft_consensus.cc:2749] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:25.436306  8781 consensus_queue.cc:260] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.436705 32011 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.437606  8792 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:21:25.437781 32623 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.438092  8781 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:25.439328 32011 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.439679  8789 consensus_queue.cc:1059] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:25.460659  8789 raft_consensus.cc:3055] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:21:25.461062  8789 raft_consensus.cc:740] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:25.463102 32011 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.463920  8792 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.464354 32011 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.464582  8792 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:21:25.465338  8792 raft_consensus.cc:2749] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:21:25.461915  8789 consensus_queue.cc:260] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.466565  8789 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:21:25.456678 32085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:39399, user_credentials={real_user=slave}} blocked reactor thread for 109042us
I20260709 12:21:25.482245 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:25.489650 32085 leader_election.cc:304] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.491490  8773 raft_consensus.cc:2749] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:25.492834 32085 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.502022 32623 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:21:25.505903 32623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:40903, user_credentials={real_user=slave}} blocked reactor thread for 68147.2us
I20260709 12:21:25.505967  8773 raft_consensus.cc:2749] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:25.508016  8776 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:25.510699 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:25.512157 31864 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec reported cstate change: term changed from 21 to 22, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2). New cstate: current_term: 22 leader_uuid: "3836f9b5cd36488da56d4540dc9d80ec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:25.513790 32010 consensus_peers.cc:597] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.514350 32010 consensus_peers.cc:597] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:25.507157 32085 leader_election.cc:304] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.511149 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.515774  8789 raft_consensus.cc:2749] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:25.516144  8776 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:25.519827 32085 leader_election.cc:304] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.522820 32085 leader_election.cc:304] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.525221 32040 raft_consensus.cc:1240] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 19. Current term is 20. Ops: []
I20260709 12:21:25.523823  8773 raft_consensus.cc:2749] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.530381 32085 leader_election.cc:304] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.542095  8773 raft_consensus.cc:2749] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:25.530045  8789 raft_consensus.cc:2749] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:25.545759 32085 leader_election.cc:304] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.547905  8804 raft_consensus.cc:2749] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:25.549319 32085 leader_election.cc:304] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.549804 32623 leader_election.cc:304] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:25.550438  8789 raft_consensus.cc:2749] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:25.550652 32085 leader_election.cc:304] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.551587  8804 raft_consensus.cc:2749] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:25.553308 32085 leader_election.cc:304] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: ec7f26ea46944ea498ad253b1b1819b0, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:25.558179  8804 raft_consensus.cc:2749] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:25.558486  8801 raft_consensus.cc:2749] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:25.559741 32085 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.560645  8804 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:21:25.575438 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:25.576081  8801 consensus_queue.cc:1059] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:21:25.576293  8804 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:21:25.590468  8776 raft_consensus.cc:3055] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:21:25.590960  8776 raft_consensus.cc:740] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:25.591912  8776 consensus_queue.cc:260] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.593845  8776 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:21:25.623154 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:25.623811 32117 raft_consensus.cc:1240] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 27. Current term is 28. Ops: []
I20260709 12:21:25.628314 32112 raft_consensus.cc:1240] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Rejecting Update request from peer ec7f26ea46944ea498ad253b1b1819b0 for earlier term 23. Current term is 24. Ops: []
I20260709 12:21:25.648617  8783 consensus_queue.cc:1059] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:21:25.654135  8806 raft_consensus.cc:3055] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:21:25.656342  8806 raft_consensus.cc:740] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:21:25.652323 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.657737 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:25.662449  8806 consensus_queue.cc:260] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.663812  8806 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:25.654201  8783 consensus_queue.cc:1059] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:21:25.677580  8801 raft_consensus.cc:3055] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:21:25.678035  8801 raft_consensus.cc:740] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:25.679085  8801 consensus_queue.cc:260] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:25.680557  8801 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:21:25.697206 32010 consensus_peers.cc:597] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:25.784016 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.788683 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:25.789222 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.789667 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:25.836253 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.857511 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:25.876443 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
I20260709 12:21:25.946201 32112 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:25.952389  8790 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:21:25.956436 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.965021 32010 consensus_peers.cc:597] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:25.968027 32010 consensus_peers.cc:597] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:26.014655 32651 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:21:26.026230 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.032269 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:26.035175 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.035331 32010 consensus_peers.cc:597] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.036180 32068 debug-util.cc:398] Leaking SignalData structure 0x7b0800252180 after lost signal to thread 30823
W20260709 12:21:26.037410 32068 debug-util.cc:398] Leaking SignalData structure 0x7b080016a680 after lost signal to thread 31904
W20260709 12:21:26.038776 32068 debug-util.cc:398] Leaking SignalData structure 0x7b0800216800 after lost signal to thread 32071
W20260709 12:21:26.039896 32068 debug-util.cc:398] Leaking SignalData structure 0x7b08001d15a0 after lost signal to thread 32145
W20260709 12:21:26.041016 32068 debug-util.cc:398] Leaking SignalData structure 0x7b080025cd20 after lost signal to thread 32684
W20260709 12:21:26.041294 32068 debug-util.cc:398] Leaking SignalData structure 0x7b0800275560 after lost signal to thread 32685
I20260709 12:21:26.050757 32651 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Refusing update from remote peer 3836f9b5cd36488da56d4540dc9d80ec: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:26.056670  8790 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:21:26.077476 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.163107 32010 consensus_peers.cc:597] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:26.208070 32084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111) [suppressed 347 similar messages]
W20260709 12:21:26.234674 32010 consensus_peers.cc:597] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:26.283039 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
W20260709 12:21:26.301137 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.306627 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.331068 32623 consensus_peers.cc:597] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.342932 32623 consensus_peers.cc:597] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.379758 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.368453 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.399554 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.406373 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.453827 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.472425 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:26.478168 30820 tablet_replica.cc:333] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.480877 30820 raft_consensus.cc:2243] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.481583 30820 raft_consensus.cc:2272] T 08b3d325c1944e45a59e666e91d0dbb6 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.484620 30820 tablet_replica.cc:333] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.485438 30820 raft_consensus.cc:2243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 LEADER]: Raft consensus shutting down.
W20260709 12:21:26.486121 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:26.491052 30820 raft_consensus.cc:2272] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.500139 30820 tablet_replica.cc:333] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.504215 30820 raft_consensus.cc:2243] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.505389 30820 raft_consensus.cc:2272] T 6679c928e92e4f799b88357707650b60 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.508555 30820 tablet_replica.cc:333] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.509397 30820 raft_consensus.cc:2243] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.510653 30820 raft_consensus.cc:2272] T 78b16558212549a98cd68f4e744eb410 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.577455 30820 tablet_replica.cc:333] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.579154 30820 raft_consensus.cc:2243] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.602880 30820 raft_consensus.cc:2272] T 2ff18ae9165241a3937358efc94d1309 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.627434 30820 tablet_replica.cc:333] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.640566 30820 raft_consensus.cc:2243] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.641461 30820 raft_consensus.cc:2272] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:26.648393 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.656517 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.657408 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:26.659510 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.661509 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.658977 30820 tablet_replica.cc:333] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.668010  8819 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:26.670159  8819 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.672319  8819 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:26.667373 30820 raft_consensus.cc:2243] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.673403 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:21:26.668368 32084 consensus_peers.cc:597] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.673413 30820 raft_consensus.cc:2272] T 16a6933cca8f4bce92ae16ff487ca7c1 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.686587 30820 tablet_replica.cc:333] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.687454 30820 raft_consensus.cc:2243] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:21:26.690475 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.691107 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:26.692641 32084 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.693202 32084 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:26.694531 30820 raft_consensus.cc:2272] T 8ca59f0a11104ab882fac419c003e8b7 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.694612  8819 raft_consensus.cc:2749] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:26.697458 30820 tablet_replica.cc:333] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.698350 30820 raft_consensus.cc:2243] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.699512 30820 raft_consensus.cc:2272] T d3949314523941858ae6d6a4fd770fb8 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.714635 30820 tablet_replica.cc:333] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.715773 30820 raft_consensus.cc:2243] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.717181 30820 raft_consensus.cc:2272] T 1a06d9856d134874b3144fd2abdbfc71 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.720064 30820 tablet_replica.cc:333] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.721006 30820 raft_consensus.cc:2243] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.721716 30820 raft_consensus.cc:2272] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.725080 30820 tablet_replica.cc:333] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.726069 30820 raft_consensus.cc:2243] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.727047 30820 raft_consensus.cc:2272] T b88d8f995fcc4faea19a190f8dd669ab P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.743399 30820 tablet_replica.cc:333] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.744390 30820 raft_consensus.cc:2243] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.746047 30820 raft_consensus.cc:2272] T 58ee45b18b714479af8291d9056afb9e P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:26.747628 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:26.749425 30820 tablet_replica.cc:333] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.750492 30820 raft_consensus.cc:2243] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.752146 30820 raft_consensus.cc:2272] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P 3836f9b5cd36488da56d4540dc9d80ec [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.755137 30820 tablet_replica.cc:333] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.756408 30820 raft_consensus.cc:2243] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.757644 30820 raft_consensus.cc:2272] T c50b710409be4b8986d1a0b839db7004 P 3836f9b5cd36488da56d4540dc9d80ec [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.765690 30820 tablet_replica.cc:333] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
W20260709 12:21:26.777267 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:26.781596 30820 raft_consensus.cc:2243] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.796911 30820 raft_consensus.cc:2272] T 422b4d293d444b06bb00c3f5b66a6f25 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.806476 30820 tablet_replica.cc:333] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.809252 30820 raft_consensus.cc:2243] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:26.814200 30820 raft_consensus.cc:2272] T ad86bf28242f4576af3db683944d53a3 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.823539 30820 tablet_replica.cc:333] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.824374  8796 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:26.825030  8796 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:26.828413 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:26.828632 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:26.829383 32117 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:21:26.830747 32626 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:21:26.831297 32623 leader_election.cc:336] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.831635  8820 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:21:26.832082  8820 raft_consensus.cc:493] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:26.832463  8820 raft_consensus.cc:3060] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:26.835819  8796 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.843703  8820 raft_consensus.cc:515] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.845104 30820 raft_consensus.cc:2243] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:26.846768 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:26.875237 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6679c928e92e4f799b88357707650b60" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:21:26.877044 32623 leader_election.cc:336] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.887859  8796 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:26.888445  8796 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.890527  8796 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.894106 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:26.894654 32623 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.894821 32116 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 29.
I20260709 12:21:26.895251  8796 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:26.895718  8796 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.902506 32626 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:26.903659  8776 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:26.904062  8776 raft_consensus.cc:493] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:26.904438  8776 raft_consensus.cc:3060] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:21:26.904552 32623 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.906006  8796 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.910506  8816 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:26.910977  8816 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.913117  8816 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.914686 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:26.915417 32116 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
W20260709 12:21:26.917467 32623 leader_election.cc:336] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.925524 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:26.926412  8796 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:21:26.926795  8796 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:26.927105  8796 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:26.928401 32117 raft_consensus.cc:2468] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:21:26.929447 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:26.930047 32116 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 19.
I20260709 12:21:26.931016 32626 leader_election.cc:304] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:26.931766  8816 raft_consensus.cc:2804] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:21:26.932112 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:26.932232  8816 raft_consensus.cc:697] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:26.868148 30820 raft_consensus.cc:2272] T 3e4bfb2be45e4fc38e6622722ab10735 P 3836f9b5cd36488da56d4540dc9d80ec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.933043  8806 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:21:26.933182  8816 consensus_queue.cc:237] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.934051  8806 raft_consensus.cc:493] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:26.934491  8806 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:26.914224  8776 raft_consensus.cc:515] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.937338  8820 leader_election.cc:290] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:26.939911 32010 consensus_peers.cc:597] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:26.941045 30820 tablet_replica.cc:333] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.941919 30820 raft_consensus.cc:2243] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.942729 30820 raft_consensus.cc:2272] T 3695041c497f47468894b0baf52fcfd0 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:26.943055  8796 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.945945 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b3d325c1944e45a59e666e91d0dbb6" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:26.946441  8776 leader_election.cc:290] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.942169  8806 raft_consensus.cc:515] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:26.956943  8796 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.957791  8806 leader_election.cc:290] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:26.948151 32623 leader_election.cc:336] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:26.964282 32623 leader_election.cc:336] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:26.964977 32623 leader_election.cc:336] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:26.969774 32117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:26.970504 32117 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:26.971627 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ed5d095c544f0abfe4e2c375b06cc" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:26.972308 32112 raft_consensus.cc:3060] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:21:26.949285 30820 tablet_replica.cc:333] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.982475 30820 raft_consensus.cc:2243] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:26.983785 32117 raft_consensus.cc:2468] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 25.
I20260709 12:21:26.999411 32626 leader_election.cc:304] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
W20260709 12:21:26.986038 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:27.000793  8822 raft_consensus.cc:2804] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:21:26.987514 30820 raft_consensus.cc:2272] T 1c3ed5d095c544f0abfe4e2c375b06cc P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.003468  8822 raft_consensus.cc:697] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:27.004932 30820 tablet_replica.cc:333] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:26.989719 32116 raft_consensus.cc:2468] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 29.
I20260709 12:21:26.990598  8818 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:27.008852  8818 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.009187 32626 leader_election.cc:304] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:27.011109 30820 raft_consensus.cc:2243] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.011585  8818 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:26.997210 32112 raft_consensus.cc:2468] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:21:27.018117 32626 leader_election.cc:304] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:27.017616 31870 catalog_manager.cc:5697] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 20 to 21, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 21 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:27.018466 32623 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.021493  8806 raft_consensus.cc:2804] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:21:27.021682 30820 raft_consensus.cc:2272] T e0bd59e23a6d4e1f9674d7662259db2f P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.022045  8806 raft_consensus.cc:697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:27.023013  8806 consensus_queue.cc:237] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.004514  8822 consensus_queue.cc:237] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.012617  8820 raft_consensus.cc:2804] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:27.028299  8820 raft_consensus.cc:697] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:27.029258  8820 consensus_queue.cc:237] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.012208 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:27.044793 32116 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:21:27.046083 32626 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:27.047830  8820 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:27.048316  8820 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:27.051885  8820 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:27.084641  8784 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:27.085214  8784 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.087533  8784 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.089960 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:27.090965 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:27.091593 32116 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:21:27.092648 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
I20260709 12:21:27.093649  8784 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:27.094017  8784 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:27.094362  8784 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:27.095103 30820 tablet_replica.cc:333] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.095934 30820 raft_consensus.cc:2243] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.096984 30820 raft_consensus.cc:2272] T 478b63c892a745d09e9508bc9cc5321f P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.102874  8784 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.108794  8820 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.110725 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:27.111373 32112 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:27.111672 32658 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 22.
I20260709 12:21:27.113952 30820 tablet_replica.cc:333] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.114943 30820 raft_consensus.cc:2243] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.116029 30820 raft_consensus.cc:2272] T 826cffce8bb540c8beb56cc00a702782 P 3836f9b5cd36488da56d4540dc9d80ec [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.118844 32112 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:21:27.120146 32013 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b; no voters: 
W20260709 12:21:27.120187 32623 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.121062  8793 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:27.121608  8793 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Becoming Leader. State: Replica: 3836f9b5cd36488da56d4540dc9d80ec, State: Running, Role: LEADER
I20260709 12:21:27.121721  8784 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 23 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.122915  8793 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.123657  8820 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:27.125151 30820 tablet_replica.cc:333] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.125381 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:27.125897 32658 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:27.126024 30820 raft_consensus.cc:2243] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.126935 30820 raft_consensus.cc:2272] T 7c52a06827f24bb6bc1851474fc80e53 P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.129684 30820 tablet_replica.cc:333] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.130822 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:27.134554  8781 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:27.134905 32112 raft_consensus.cc:2368] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23.
I20260709 12:21:27.135236  8781 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.136273 32626 leader_election.cc:400] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23.
I20260709 12:21:27.136579 32626 leader_election.cc:403] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:21:27.137873  8781 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.140456 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:27.141325 32660 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 28.
I20260709 12:21:27.142617 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:27.145515  8817 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:27.145965  8817 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:27.149405  8817 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:21:27.153873 32084 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.160092  8784 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:27.160704  8784 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.163733  8784 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.165078  8790 raft_consensus.cc:493] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:27.165215 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:27.165771  8790 raft_consensus.cc:515] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.168115  8790 leader_election.cc:290] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:27.169426 32010 consensus_peers.cc:597] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:27.170573 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:27.171233 32660 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 28.
I20260709 12:21:27.174089 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:27.175382 32011 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:27.177449 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b83e7300b3fe4d21a97d941dee98fa81" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:27.178272  8791 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:27.181156 32112 raft_consensus.cc:2368] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 24. Current term is 25.
I20260709 12:21:27.181123 32658 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3836f9b5cd36488da56d4540dc9d80ec in term 23.
I20260709 12:21:27.180723  8791 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Starting leader election (detected failure of leader ec7f26ea46944ea498ad253b1b1819b0)
I20260709 12:21:27.186069  8820 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ec7f26ea46944ea498ad253b1b1819b0 for earlier term 22. Current term is 23.
I20260709 12:21:27.186744  8791 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:27.186367 32013 leader_election.cc:400] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 24. Current term is 25.
I20260709 12:21:27.191296 32013 leader_election.cc:403] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:27.192598  8784 raft_consensus.cc:3060] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:21:27.204823  8817 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.207643 32116 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 28. Current term is 29.
I20260709 12:21:27.210633 32013 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 pre-election: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 28. Current term is 29.
I20260709 12:21:27.217299  8791 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.219022 30820 raft_consensus.cc:2243] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.232713 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:27.233386 32658 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:27.237861 30820 pending_rounds.cc:70] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Trying to abort 1 pending ops.
I20260709 12:21:27.241346 32658 raft_consensus.cc:2468] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 29.
I20260709 12:21:27.242774 32085 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:27.244020  8791 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Requested vote from peers 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.238343 30820 pending_rounds.cc:77] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec: Aborting op as it isn't in flight: id { term: 23 index: 18 } timestamp: 7305624318501773312 op_type: NO_OP noop_request { }
I20260709 12:21:27.236863  8817 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.244994 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:27.245620  8804 raft_consensus.cc:2804] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:27.246402  8804 raft_consensus.cc:697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:27.245398 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfedf707307443deb5a0681825db83cb" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
W20260709 12:21:27.245481  8828 consensus_queue.cc:1175] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3836f9b5cd36488da56d4540dc9d80ec" status { last_received { term: 23 index: 18 } last_committed_idx: 17 last_received_current_leader { term: 23 index: 18 } }
I20260709 12:21:27.247426  8804 consensus_queue.cc:237] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.245012 30820 raft_consensus.cc:2889] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:21:27.250965 30820 raft_consensus.cc:2272] T 6d02c13b00af439e946e76ef880668d9 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.250582 32658 raft_consensus.cc:2368] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 28. Current term is 29.
W20260709 12:21:27.244822 32084 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.252357 32011 leader_election.cc:400] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 28. Current term is 29.
I20260709 12:21:27.252662 32011 leader_election.cc:403] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
W20260709 12:21:27.255002 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:27.263175  8784 raft_consensus.cc:2749] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255) with higher term. Message: Invalid argument: T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 24. Current term is 25.
I20260709 12:21:27.263607  8791 raft_consensus.cc:3060] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:27.280718 30820 tablet_replica.cc:333] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.281925 30820 raft_consensus.cc:2243] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.283155 30820 raft_consensus.cc:2272] T 58d45de750cd4fd1a0cdd13ad4f731fb P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.285931 30820 tablet_replica.cc:333] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.294126  8791 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399) with higher term. Message: Invalid argument: T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 3836f9b5cd36488da56d4540dc9d80ec for earlier term 28. Current term is 29.
I20260709 12:21:27.286676 30820 raft_consensus.cc:2243] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.296079 30820 raft_consensus.cc:2272] T d2f941a3c6cb4a188198a804d603db3b P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.298799 31870 catalog_manager.cc:5697] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:27.300076 30820 tablet_replica.cc:333] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.300909 30820 raft_consensus.cc:2243] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.300412 31870 catalog_manager.cc:5697] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:27.302348 30820 raft_consensus.cc:2272] T 0d6c0fabd5a9486f95f088218e4ba65d P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.313007 30820 tablet_replica.cc:333] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.313970 30820 raft_consensus.cc:2243] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:27.316319 32010 consensus_peers.cc:597] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:27.302155 31870 catalog_manager.cc:5697] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 28 to 29, leader changed from 38c927f0ee104b57972899f5b27aa99b (127.30.25.3) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 29 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:27.317153  8784 raft_consensus.cc:493] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:27.317792  8784 raft_consensus.cc:515] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:27.322197 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da57956732d44d1b912fb270de7d63f" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:27.327544 31864 catalog_manager.cc:5697] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 27 to 29, leader changed from ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 29 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:27.329638 32010 leader_election.cc:336] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:27.330142 32010 leader_election.cc:304] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:27.331034  8784 leader_election.cc:290] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:27.331605  8793 raft_consensus.cc:2749] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:27.336030 30820 raft_consensus.cc:2272] T bfedf707307443deb5a0681825db83cb P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.338891 30820 tablet_replica.cc:333] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.339730 30820 raft_consensus.cc:2243] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.340425 30820 raft_consensus.cc:2272] T 0292f21aab4a4843b2b424c3497ddced P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.386703 30820 tablet_replica.cc:333] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.389586 30820 raft_consensus.cc:2243] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.390861 30820 raft_consensus.cc:2272] T 4633b71e43824d84895ff136e5765fb1 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.404889 30820 tablet_replica.cc:333] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.411633 32112 raft_consensus.cc:1275] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:21:27.414502 30820 raft_consensus.cc:2243] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.413208  8820 consensus_queue.cc:1048] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:21:27.415369 30820 raft_consensus.cc:2272] T c6c31e616d2c4874ac5a8e29453cc51b P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.419335 30820 tablet_replica.cc:333] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.424849 30820 raft_consensus.cc:2243] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.432430 30820 raft_consensus.cc:2272] T 71e0cf115549454c894335e70c43843a P 3836f9b5cd36488da56d4540dc9d80ec [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.453639 30820 tablet_replica.cc:333] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.454432 30820 raft_consensus.cc:2243] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:27.456585 32623 consensus_peers.cc:597] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.469070  8791 raft_consensus.cc:493] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:27.469754  8791 raft_consensus.cc:515] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:27.472064  8791 leader_election.cc:290] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:27.475809 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a84a8ea71c4258b378603a0827a768" candidate_uuid: "3836f9b5cd36488da56d4540dc9d80ec" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
W20260709 12:21:27.479043 32010 leader_election.cc:336] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:27.478047 30820 raft_consensus.cc:2272] T 02ef57fadd044f68ba212dc2fe410f83 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.479630 32013 leader_election.cc:304] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3836f9b5cd36488da56d4540dc9d80ec; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:27.480985  8791 raft_consensus.cc:2749] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:27.490571 30820 tablet_replica.cc:333] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.495915 30820 raft_consensus.cc:2243] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.496589 30820 raft_consensus.cc:2272] T 193a74284231469f9229c961dfaff799 P 3836f9b5cd36488da56d4540dc9d80ec [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.499557 30820 tablet_replica.cc:333] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.500396 30820 raft_consensus.cc:2243] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.515187 30820 raft_consensus.cc:2272] T 19357e2a1cf1408b9391be0c12c3b0e8 P 3836f9b5cd36488da56d4540dc9d80ec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.518080 30820 tablet_replica.cc:333] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.518857 30820 raft_consensus.cc:2243] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.519948 30820 raft_consensus.cc:2272] T 5b2dd1839ae848ae849e1ec74ce14112 P 3836f9b5cd36488da56d4540dc9d80ec [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:27.526186 32623 consensus_peers.cc:597] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.540808 32112 raft_consensus.cc:1275] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:21:27.542696  8776 consensus_queue.cc:1048] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:21:27.543311 32010 consensus_peers.cc:597] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:27.547843 30820 tablet_replica.cc:333] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.548988 30820 raft_consensus.cc:2243] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.555583 30820 raft_consensus.cc:2272] T 8f4287254496472398f69d3a10b48d77 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.563354 30820 tablet_replica.cc:333] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.564368 30820 raft_consensus.cc:2243] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.566200 30820 raft_consensus.cc:2272] T a7f1c1160177412eb547493e1f3c561f P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.568861 30820 tablet_replica.cc:333] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.569525 30820 raft_consensus.cc:2243] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.570087 30820 raft_consensus.cc:2272] T b83e7300b3fe4d21a97d941dee98fa81 P 3836f9b5cd36488da56d4540dc9d80ec [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.573004 30820 tablet_replica.cc:333] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.573705 30820 raft_consensus.cc:2243] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.578954  8804 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.579523  8804 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:27.581676  8804 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:27.585901 32084 leader_election.cc:336] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:27.586700 32084 leader_election.cc:336] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.587090 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:27.588644  8819 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:27.595259 30820 raft_consensus.cc:2272] T 15d2c5dc46254473902eca585b82958f P 3836f9b5cd36488da56d4540dc9d80ec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.602137 30820 tablet_replica.cc:333] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
W20260709 12:21:27.607625 32623 consensus_peers.cc:597] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.619385 30820 raft_consensus.cc:2243] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.620520 30820 raft_consensus.cc:2272] T ef2f2222a1ac4f6f923fdee4de1f48dc P 3836f9b5cd36488da56d4540dc9d80ec [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:27.625016 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:27.625734 30820 tablet_replica.cc:333] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.626612 30820 raft_consensus.cc:2243] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:27.627961 30820 raft_consensus.cc:2272] T fbb363b96dbf430998ab5641d3b03f46 P 3836f9b5cd36488da56d4540dc9d80ec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.630741 30820 tablet_replica.cc:333] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.631456 30820 raft_consensus.cc:2243] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.632040 30820 raft_consensus.cc:2272] T 1da57956732d44d1b912fb270de7d63f P 3836f9b5cd36488da56d4540dc9d80ec [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:27.636364 32623 consensus_peers.cc:597] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:27.643918 30820 tablet_replica.cc:333] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec: stopping tablet replica
I20260709 12:21:27.645532 30820 raft_consensus.cc:2243] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.646152 30820 raft_consensus.cc:2272] T 88a84a8ea71c4258b378603a0827a768 P 3836f9b5cd36488da56d4540dc9d80ec [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.655061 32112 raft_consensus.cc:1275] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:21:27.657001  8806 consensus_queue.cc:1048] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:27.666904 32116 raft_consensus.cc:1275] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:21:27.669257  8806 consensus_queue.cc:1048] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:21:27.723887 32084 consensus_peers.cc:597] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:27.739516 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:27.754623  8819 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.755236  8819 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:27.761924  8819 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:27.779126  8819 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.779704  8819 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.783283 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:27.802428  8819 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:27.810833 32084 leader_election.cc:336] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:27.814359 32084 leader_election.cc:336] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.816126 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:27.827732  8804 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:27.851703 32084 leader_election.cc:336] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.852293 32084 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:27.853094  8819 raft_consensus.cc:2749] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:27.858158  8806 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.858959  8806 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.861430  8806 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:27.869256 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:27.870528 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:27.871038 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:27.871948  8776 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:27.875512 32658 raft_consensus.cc:1275] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:21:27.877475  8819 consensus_queue.cc:1048] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:21:27.903549 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:27.937795  8776 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.938494  8776 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.940546  8776 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:27.965924  8776 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.966598  8776 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.970269 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:27.971078 32116 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 21.
I20260709 12:21:27.971863  8819 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.972553  8819 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:27.972833 32626 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:27.974468  8806 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:27.974915  8806 raft_consensus.cc:493] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:27.974998  8819 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:27.975391  8806 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:27.979573 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:27.978720  8776 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:27.981781 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:27.990159 32084 leader_election.cc:336] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:27.990682 32084 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:27.991411  8819 raft_consensus.cc:2749] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:27.994002 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:27.994562 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:27.995318  8820 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:27.999316  8806 raft_consensus.cc:515] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.004580 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3949314523941858ae6d6a4fd770fb8" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:28.005147 32112 raft_consensus.cc:3060] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:28.014744 32112 raft_consensus.cc:2468] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:21:28.015928  8806 leader_election.cc:290] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:28.016227 32626 leader_election.cc:304] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.017153  8806 raft_consensus.cc:2804] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:28.017958  8806 raft_consensus.cc:697] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
W20260709 12:21:28.018875 32623 leader_election.cc:336] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.018951  8806 consensus_queue.cc:237] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:28.026492 32623 leader_election.cc:336] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.046564 31864 catalog_manager.cc:5697] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 21 to 22, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 22 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:28.069355 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:28.115193  8806 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.115850  8806 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.116494  8820 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.117017  8820 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.118156  8806 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:28.119050  8820 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:28.119284  8806 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.119784  8806 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.121910  8806 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:28.126268 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:28.126984 32112 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 22.
I20260709 12:21:28.128228 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.129074  8806 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:21:28.129714 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:28.129644  8806 raft_consensus.cc:493] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.130398  8806 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:28.137388  8820 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.138046  8820 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.140362  8820 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:28.143185  8806 raft_consensus.cc:515] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.149593  8819 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.150620  8819 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.152807  8819 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:28.154428 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:21:28.157063 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.157480 32084 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.160133 32084 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:28.161432  8819 raft_consensus.cc:2749] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:21:28.160133 32623 leader_election.cc:336] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.165359 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.167979 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:28.168494 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.169307  8820 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:28.159255  8806 leader_election.cc:290] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:28.159500 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b4d293d444b06bb00c3f5b66a6f25" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
W20260709 12:21:28.170536 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:28.170691 32112 raft_consensus.cc:3060] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:28.170930 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.180989  8820 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:28.181800 32112 raft_consensus.cc:2468] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 23.
I20260709 12:21:28.183019 32626 leader_election.cc:304] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.183841  8820 raft_consensus.cc:2804] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:28.184355  8820 raft_consensus.cc:697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:28.185170  8820 consensus_queue.cc:237] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:28.193171 32623 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:28.203308 32623 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.203801 32623 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.204595  8820 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:28.209049 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:28.210248 32623 leader_election.cc:336] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.214048  8806 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.214699  8806 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.216804  8806 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:28.219590 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:28.220320 32112 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 32.
I20260709 12:21:28.221596 32626 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.217491 31864 catalog_manager.cc:5697] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 22 to 23, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 23 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:28.223771  8822 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:21:28.224187  8822 raft_consensus.cc:493] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.224560  8822 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:21:28.257472 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:28.258236 32623 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.260725  8822 raft_consensus.cc:515] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.265306  8817 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.278627  8817 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:28.280676  8817 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:28.289639 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50b710409be4b8986d1a0b839db7004" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b"
I20260709 12:21:28.290230 32116 raft_consensus.cc:3060] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:21:28.293794 32623 leader_election.cc:336] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.297745 32116 raft_consensus.cc:2468] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 33.
I20260709 12:21:28.305742 32626 leader_election.cc:304] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec
I20260709 12:21:28.306710  8820 raft_consensus.cc:2804] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:21:28.308015  8820 raft_consensus.cc:697] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 33 LEADER]: Becoming Leader. State: Replica: ec7f26ea46944ea498ad253b1b1819b0, State: Running, Role: LEADER
I20260709 12:21:28.310019  8822 leader_election.cc:290] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 33 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:28.311563  8820 consensus_queue.cc:237] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:28.318863 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:28.319440 32084 consensus_peers.cc:597] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:28.316133  8822 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.326671  8822 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.330762  8822 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:28.336145 32623 leader_election.cc:336] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.339250 32084 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:28.339922 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:28.340392 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:28.341446 32084 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.342130 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.348279  8817 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:28.349431  8817 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.349896  8817 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.352567  8817 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:28.353613 31864 catalog_manager.cc:5697] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 reported cstate change: term changed from 32 to 33, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4). New cstate: current_term: 33 leader_uuid: "ec7f26ea46944ea498ad253b1b1819b0" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:28.354961 32658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
W20260709 12:21:28.374993 32084 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.375442 32084 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, ec7f26ea46944ea498ad253b1b1819b0
I20260709 12:21:28.376132  8817 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:28.384387 32084 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:28.438735 32084 consensus_peers.cc:597] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:28.443790 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:28.461660  8817 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.462519  8817 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:28.465029  8817 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:28.471292 32084 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:28.473836  8817 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:21:28.473994 32084 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.474613 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.474645  8817 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:28.475404  8819 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:28.477348  8817 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:28.483970 32084 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:28.486992 32084 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.487388 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.488427  8817 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:28.489696  8822 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.490265  8822 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.492523  8822 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:28.496527 32623 leader_election.cc:336] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.498855  8822 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.499563  8822 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.505558 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:28.506565  8822 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:28.510668 32623 leader_election.cc:336] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.511133 32623 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:28.515940  8820 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:28.518818 32116 raft_consensus.cc:1275] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:21:28.520473  8822 consensus_queue.cc:1048] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:28.530043  8776 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.530920  8776 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.533651  8776 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:28.536912 32623 leader_election.cc:336] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.537798 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:28.539752 32626 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:28.540021  8820 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.542495  8820 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.545923  8820 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:28.546451  8776 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:21:28.548830 32623 leader_election.cc:336] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.561462 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:28.606124 32623 consensus_peers.cc:597] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:28.619299 32623 consensus_peers.cc:597] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:28.653683 32116 raft_consensus.cc:1275] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:21:28.655308  8822 consensus_queue.cc:1048] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:28.676139  8776 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.676687  8776 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.680172  8776 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
W20260709 12:21:28.701963 32623 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.744611  8819 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.745235  8819 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:28.752655  8819 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:28.754417 32623 consensus_peers.cc:597] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:28.769062 32084 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.769796 32084 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:28.770175 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.771039  8819 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:28.785420  8817 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:28.785975  8817 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.789911 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:28.792359 32651 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:21:28.793819 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.795634  8817 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:28.796269  8817 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:21:28.796612  8817 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:28.796972  8817 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:21:28.799923  8806 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.800554  8806 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.804442  8804 raft_consensus.cc:493] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.804746 32116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d6c0fabd5a9486f95f088218e4ba65d" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:28.805044  8804 raft_consensus.cc:515] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:28.805501 32116 raft_consensus.cc:2468] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec7f26ea46944ea498ad253b1b1819b0 in term 24.
I20260709 12:21:28.807205  8804 leader_election.cc:290] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:28.812188  8806 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:28.819929 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:28.825090 32623 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.841451  8817 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.849788 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02c13b00af439e946e76ef880668d9" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:28.850575 32651 raft_consensus.cc:3060] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:21:28.857920 32651 raft_consensus.cc:2468] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 24.
I20260709 12:21:28.877447 32085 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.878352  8819 raft_consensus.cc:2804] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:21:28.887491  8817 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:28.891069 32084 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:28.892781 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:28.894843  8819 raft_consensus.cc:697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
W20260709 12:21:28.895955 32084 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:28.896857 32084 leader_election.cc:336] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.896934  8819 consensus_queue.cc:237] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.893291  8806 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.897390 32084 leader_election.cc:304] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:28.899052  8806 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.909355  8806 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:28.912734  8851 raft_consensus.cc:2749] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:28.913123 32112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "ec7f26ea46944ea498ad253b1b1819b0" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "38c927f0ee104b57972899f5b27aa99b" is_pre_election: true
I20260709 12:21:28.914837  8851 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.915354  8851 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.917459  8851 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:28.920037 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:28.921263 32651 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 22.
W20260709 12:21:28.924216 32623 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.924799 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:28.928435 32085 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
W20260709 12:21:28.929502 32084 consensus_peers.cc:597] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:28.929705  8822 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:21:28.929870  8804 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:28.930505  8804 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.930894  8804 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:28.933862 32112 raft_consensus.cc:1275] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 33 FOLLOWER]: Refusing update from remote peer ec7f26ea46944ea498ad253b1b1819b0: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:21:28.935480  8822 consensus_queue.cc:1048] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:21:28.930022 32084 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:28.942523  8804 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.945996  8851 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.946617  8851 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.948696  8851 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:28.950227 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:28.950853 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 30.
W20260709 12:21:28.951225 32623 consensus_peers.cc:597] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:28.951922 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.952802  8851 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:21:28.953384  8851 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:28.953807  8851 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:21:28.957386 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:28.964926 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633b71e43824d84895ff136e5765fb1" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:28.965592 32651 raft_consensus.cc:3060] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:28.973912 32651 raft_consensus.cc:2468] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 23.
I20260709 12:21:28.979398 32085 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:28.980739  8817 raft_consensus.cc:2804] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:28.974404  8804 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:28.981302  8817 raft_consensus.cc:697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:28.962132  8851 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.982396  8817 consensus_queue.cc:237] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:28.985107 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b2dd1839ae848ae849e1ec74ce14112" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:28.985690 32651 raft_consensus.cc:3060] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:21:28.986553 31864 catalog_manager.cc:5697] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:28.992897 30820 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 32068)
I20260709 12:21:28.990363  8851 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
I20260709 12:21:29.010138 32651 raft_consensus.cc:2468] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 31.
I20260709 12:21:29.011427 32085 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:29.030856  8819 raft_consensus.cc:2804] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:21:29.031430  8819 raft_consensus.cc:697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:29.033457  8819 consensus_queue.cc:237] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:29.034013 32084 consensus_peers.cc:597] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.035604 32084 consensus_peers.cc:597] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.041635 32084 leader_election.cc:336] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:29.042780 32084 consensus_peers.cc:597] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.044492 32084 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:29.054476 32623 consensus_peers.cc:597] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:29.060989 31864 catalog_manager.cc:5697] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 22 to 23, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 23 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:29.062827 31864 catalog_manager.cc:5697] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 30 to 31, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 31 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:29.068948  8819 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.069507  8819 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:29.070351 32084 leader_election.cc:336] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:29.071158 32084 consensus_peers.cc:597] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.071542 32084 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:29.076522 32084 consensus_peers.cc:597] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.076962 32084 consensus_peers.cc:597] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:29.078632  8776 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.079262  8776 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:29.081389  8776 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:29.084127 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0" is_pre_election: true
I20260709 12:21:29.084832 32651 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 20.
I20260709 12:21:29.086071 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:29.086977  8817 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:21:29.087374  8817 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.087734  8817 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:29.079571  8819 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:29.093631 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:29.096347 32084 leader_election.cc:336] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:29.097357  8817 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:29.101079 32651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d2c5dc46254473902eca585b82958f" candidate_uuid: "38c927f0ee104b57972899f5b27aa99b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ec7f26ea46944ea498ad253b1b1819b0"
I20260709 12:21:29.101671 32651 raft_consensus.cc:3060] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:29.116403 32651 raft_consensus.cc:2468] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c927f0ee104b57972899f5b27aa99b in term 21.
I20260709 12:21:29.119043 32085 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 
I20260709 12:21:29.120045  8819 raft_consensus.cc:2804] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:21:29.120568  8819 raft_consensus.cc:697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Becoming Leader. State: Replica: 38c927f0ee104b57972899f5b27aa99b, State: Running, Role: LEADER
I20260709 12:21:29.121481  8819 consensus_queue.cc:237] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:29.126842  8817 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: Requested vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), ec7f26ea46944ea498ad253b1b1819b0 (127.30.25.4:39399)
W20260709 12:21:29.133371 32084 leader_election.cc:336] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:29.163383 31864 catalog_manager.cc:5697] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b reported cstate change: term changed from 20 to 21, leader changed from 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2) to 38c927f0ee104b57972899f5b27aa99b (127.30.25.3). New cstate: current_term: 21 leader_uuid: "38c927f0ee104b57972899f5b27aa99b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:29.174357 32084 consensus_peers.cc:597] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.196132 32084 consensus_peers.cc:597] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.298554 32084 consensus_peers.cc:597] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:29.308046 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.311261 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.319905 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:29.369141 32623 leader_election.cc:336] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:29.369601 32623 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.370522  8822 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:21:29.371220 32623 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:29.371683 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:21:29.372534 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:29.372382  8822 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:29.373217 32623 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:29.373613 32623 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b, ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.374292  8822 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:29.374653 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:29.375104 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.375775  8822 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:29.378152 32623 leader_election.cc:336] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:29.378562 32623 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
W20260709 12:21:29.379489 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.380054 32623 leader_election.cc:336] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:29.380388 32623 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.381150  8820 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:29.381790  8820 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:29.385183 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:29.393234  8819 raft_consensus.cc:493] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.393803  8819 raft_consensus.cc:515] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:29.395984  8819 leader_election.cc:290] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:29.402024 32084 leader_election.cc:336] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:29.402673 32084 leader_election.cc:336] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:29.403041 32084 leader_election.cc:304] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.403723  8819 raft_consensus.cc:2749] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:29.451376 32651 raft_consensus.cc:1275] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:21:29.452893  8819 consensus_queue.cc:1048] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:21:29.500490 32084 consensus_peers.cc:597] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:29.500977 32084 consensus_peers.cc:597] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:29.507246 32084 consensus_peers.cc:597] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:29.526829 32651 raft_consensus.cc:1275] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:21:29.528394  8852 consensus_queue.cc:1048] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:29.570988  8822 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.571521  8822 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:29.573597  8822 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:29.575845 32084 consensus_peers.cc:597] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:29.576327 32084 consensus_peers.cc:597] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:29.582430 32623 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.586822 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:29.587636 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:29.588048 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.588779  8822 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:29.605584  8852 raft_consensus.cc:493] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.606215  8852 raft_consensus.cc:515] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:29.608515  8852 leader_election.cc:290] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:29.614984 32084 leader_election.cc:336] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:29.615727 32084 leader_election.cc:336] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:29.616110 32084 leader_election.cc:304] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.616806  8852 raft_consensus.cc:2749] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:29.644425 32651 raft_consensus.cc:1275] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:21:29.646064  8819 consensus_queue.cc:1048] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:21:29.695086 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.703387 32623 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:29.747325 32658 raft_consensus.cc:1275] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Refusing update from remote peer 38c927f0ee104b57972899f5b27aa99b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:21:29.748577  8851 consensus_queue.cc:1048] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:29.755151  8820 raft_consensus.cc:493] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:29.755698  8820 raft_consensus.cc:515] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:29.757881  8820 leader_election.cc:290] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:29.777328 32623 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:29.778173 32623 leader_election.cc:336] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:29.784706 32623 leader_election.cc:304] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:29.785478  8822 raft_consensus.cc:2749] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:29.820917 32623 consensus_peers.cc:597] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:30.003348  8822 raft_consensus.cc:493] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.003918  8822 raft_consensus.cc:515] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:30.006378  8822 leader_election.cc:290] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:30.015951 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:30.016695 32623 leader_election.cc:336] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.017077 32623 leader_election.cc:304] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.017813  8822 raft_consensus.cc:2749] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:30.053294  8820 raft_consensus.cc:493] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.053834  8820 raft_consensus.cc:515] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:30.057603  8820 leader_election.cc:290] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:30.063716 32084 consensus_peers.cc:597] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:30.074544 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:30.075259 32623 leader_election.cc:336] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.075639 32623 leader_election.cc:304] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.077394  8865 raft_consensus.cc:2749] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:30.085430 32623 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:30.089802  8852 raft_consensus.cc:493] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.090399  8852 raft_consensus.cc:515] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:30.092460  8852 leader_election.cc:290] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:30.097592  8852 raft_consensus.cc:493] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:21:30.097887 32084 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:30.098069  8852 raft_consensus.cc:515] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
W20260709 12:21:30.098567 32084 leader_election.cc:336] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.099097 32084 leader_election.cc:304] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.100438  8852 leader_election.cc:290] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:30.102145  8852 raft_consensus.cc:2749] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:30.109450  8865 raft_consensus.cc:493] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.110054  8865 raft_consensus.cc:515] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:30.112258  8865 leader_election.cc:290] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:30.119022 32084 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:30.121814 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:30.122812 32623 leader_election.cc:336] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:30.125530 32623 leader_election.cc:304] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.126278  8865 raft_consensus.cc:2749] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:21:30.124325 32084 leader_election.cc:336] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.127818 32084 leader_election.cc:304] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.130339  8852 raft_consensus.cc:2749] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:30.135602 32623 consensus_peers.cc:597] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:30.137322 32084 consensus_peers.cc:597] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:30.138664 32623 consensus_peers.cc:597] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:30.166698 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
W20260709 12:21:30.171594 32623 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:30.223229 32623 consensus_peers.cc:597] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:30.262665 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
W20260709 12:21:30.298074 32626 consensus_peers.cc:597] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.307523  8852 raft_consensus.cc:493] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.308120  8852 raft_consensus.cc:515] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:30.310192  8852 leader_election.cc:290] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:30.324873 32084 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:30.327291 32084 leader_election.cc:336] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.327723 32084 leader_election.cc:304] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.328415  8852 raft_consensus.cc:2749] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:30.334508 32626 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.336495 32084 consensus_peers.cc:597] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:21:30.338616 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:30.344595 30820 tablet_replica.cc:333] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.346525 30820 raft_consensus.cc:2243] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.347149 30820 raft_consensus.cc:2272] T 08b3d325c1944e45a59e666e91d0dbb6 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.349895 30820 tablet_replica.cc:333] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.350742 30820 raft_consensus.cc:2243] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.351828 30820 raft_consensus.cc:2272] T d1ae9fcf8eb748c6b0cb9703adc9f338 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.354369 30820 tablet_replica.cc:333] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.355120 30820 raft_consensus.cc:2243] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.355609 30820 raft_consensus.cc:2272] T 6679c928e92e4f799b88357707650b60 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.358019 30820 tablet_replica.cc:333] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.358803 30820 raft_consensus.cc:2243] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.359354 30820 raft_consensus.cc:2272] T 2ff18ae9165241a3937358efc94d1309 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.361853 30820 tablet_replica.cc:333] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.362634 30820 raft_consensus.cc:2243] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.363157 30820 raft_consensus.cc:2272] T 74fd2ead38d34aac9ef68364e1f4a2f8 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.365767 30820 tablet_replica.cc:333] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.366482 30820 raft_consensus.cc:2243] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.367094 30820 raft_consensus.cc:2272] T 1612f8e575bd46cf8069fa1c21ca192b P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.369712 30820 tablet_replica.cc:333] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
W20260709 12:21:30.370286 32626 consensus_peers.cc:597] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.370529 30820 raft_consensus.cc:2243] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.371632 30820 raft_consensus.cc:2272] T 16a6933cca8f4bce92ae16ff487ca7c1 P 38c927f0ee104b57972899f5b27aa99b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.374339 30820 tablet_replica.cc:333] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.375087 30820 raft_consensus.cc:2243] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.375617 30820 raft_consensus.cc:2272] T 2c59a3d3077c4b8db6380a834f6ce3c0 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.378196 30820 tablet_replica.cc:333] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.378994 30820 raft_consensus.cc:2243] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.379498 30820 raft_consensus.cc:2272] T 0c82ba0f778d4ff98ba9cc8255f28e9d P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.382000 30820 tablet_replica.cc:333] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.382805 30820 raft_consensus.cc:2243] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.383380 30820 raft_consensus.cc:2272] T 9d2cc7077f01430da5a31385d674e266 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.385978 30820 tablet_replica.cc:333] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.386790 30820 raft_consensus.cc:2243] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.387851 30820 raft_consensus.cc:2272] T f3f78e3424864d319ee8da87f4848fe9 P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.390517 30820 tablet_replica.cc:333] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.391295 30820 raft_consensus.cc:2243] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.391824 30820 raft_consensus.cc:2272] T d3949314523941858ae6d6a4fd770fb8 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.394263 30820 tablet_replica.cc:333] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.395035 30820 raft_consensus.cc:2243] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.396517 30820 raft_consensus.cc:2272] T b88d8f995fcc4faea19a190f8dd669ab P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.399188 30820 tablet_replica.cc:333] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.399916 30820 raft_consensus.cc:2243] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.400415 30820 raft_consensus.cc:2272] T ad86bf28242f4576af3db683944d53a3 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.402786 30820 tablet_replica.cc:333] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.403486 30820 raft_consensus.cc:2243] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.404186 30820 raft_consensus.cc:2272] T 1c3ed5d095c544f0abfe4e2c375b06cc P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.406441 30820 tablet_replica.cc:333] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.407207 30820 raft_consensus.cc:2243] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.407733 30820 raft_consensus.cc:2272] T e0bd59e23a6d4e1f9674d7662259db2f P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.410032 30820 tablet_replica.cc:333] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.410807 30820 raft_consensus.cc:2243] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.411372 30820 raft_consensus.cc:2272] T e828882e6b1940ffb33d4c2f76b47fe9 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.413815 30820 tablet_replica.cc:333] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.414486 30820 raft_consensus.cc:2243] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.415488 30820 raft_consensus.cc:2272] T 6d02c13b00af439e946e76ef880668d9 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.418133 30820 tablet_replica.cc:333] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.419036 30820 raft_consensus.cc:2243] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.419620 30820 raft_consensus.cc:2272] T 422b4d293d444b06bb00c3f5b66a6f25 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.422044 30820 tablet_replica.cc:333] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.422778 30820 raft_consensus.cc:2243] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.423403 30820 raft_consensus.cc:2272] T ef2f2222a1ac4f6f923fdee4de1f48dc P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.425949 30820 tablet_replica.cc:333] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.426776 30820 raft_consensus.cc:2243] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.427901 30820 raft_consensus.cc:2272] T 0e910ef3c0af41bf8d4fba5315d68eef P 38c927f0ee104b57972899f5b27aa99b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.430089 32626 consensus_peers.cc:597] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.430733 32626 consensus_peers.cc:597] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.430784 30820 tablet_replica.cc:333] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.431202  8851 raft_consensus.cc:493] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.431737 30820 raft_consensus.cc:2243] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.431720  8851 raft_consensus.cc:515] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } }
I20260709 12:21:30.432473 30820 raft_consensus.cc:2272] T c50b710409be4b8986d1a0b839db7004 P 38c927f0ee104b57972899f5b27aa99b [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.433744  8851 leader_election.cc:290] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:30.435057 30820 tablet_replica.cc:333] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.435844 30820 raft_consensus.cc:2243] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.436991 30820 raft_consensus.cc:2272] T 786db0eb7e9b431ba90c5ad880c46c9e P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.438901 32084 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:30.439569 32084 leader_election.cc:336] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.439733 30820 tablet_replica.cc:333] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.439957 32084 leader_election.cc:304] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38c927f0ee104b57972899f5b27aa99b; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.440611 30820 raft_consensus.cc:2243] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.440640  8851 raft_consensus.cc:2749] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:30.442274 30820 raft_consensus.cc:2272] T 4633b71e43824d84895ff136e5765fb1 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.445305 30820 tablet_replica.cc:333] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.446030 30820 raft_consensus.cc:2243] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.447235 30820 raft_consensus.cc:2272] T c6c31e616d2c4874ac5a8e29453cc51b P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.449921 30820 tablet_replica.cc:333] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.450667 30820 raft_consensus.cc:2243] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:21:30.451025 32626 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.451767 30820 raft_consensus.cc:2272] T 193a74284231469f9229c961dfaff799 P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.454406 30820 tablet_replica.cc:333] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.455157 30820 raft_consensus.cc:2243] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.456202 30820 raft_consensus.cc:2272] T 88a84a8ea71c4258b378603a0827a768 P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.458750 30820 tablet_replica.cc:333] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.459488 30820 raft_consensus.cc:2243] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.460119 30820 raft_consensus.cc:2272] T f953fa305a42492da9de0fa8c18c43eb P 38c927f0ee104b57972899f5b27aa99b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.462635 30820 tablet_replica.cc:333] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.463369 30820 raft_consensus.cc:2243] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.463954 30820 raft_consensus.cc:2272] T 58d45de750cd4fd1a0cdd13ad4f731fb P 38c927f0ee104b57972899f5b27aa99b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.466470 30820 tablet_replica.cc:333] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.467163 30820 raft_consensus.cc:2243] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.467761 30820 raft_consensus.cc:2272] T 6d27358a7bd3493a8877bd9ce8ae9055 P 38c927f0ee104b57972899f5b27aa99b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.470396 30820 tablet_replica.cc:333] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.471095 30820 raft_consensus.cc:2243] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.472110 30820 raft_consensus.cc:2272] T d2f941a3c6cb4a188198a804d603db3b P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.474824 30820 tablet_replica.cc:333] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.475519 30820 raft_consensus.cc:2243] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.476064 30820 raft_consensus.cc:2272] T 775fd46a41044b1ea06bc796ef1b233b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.477196 32626 consensus_peers.cc:597] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.479045 30820 tablet_replica.cc:333] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.479929 30820 raft_consensus.cc:2243] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.480947 30820 raft_consensus.cc:2272] T 15d2c5dc46254473902eca585b82958f P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.483480 32626 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.483676 30820 tablet_replica.cc:333] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.484530 30820 raft_consensus.cc:2243] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.485109 30820 raft_consensus.cc:2272] T b83e7300b3fe4d21a97d941dee98fa81 P 38c927f0ee104b57972899f5b27aa99b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.488479 30820 tablet_replica.cc:333] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.489203 30820 raft_consensus.cc:2243] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.490242 30820 raft_consensus.cc:2272] T 51b3fd343e81436fa5bea82486abcc64 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.492875 30820 tablet_replica.cc:333] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.493557 30820 raft_consensus.cc:2243] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.494613 30820 raft_consensus.cc:2272] T bfedf707307443deb5a0681825db83cb P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.497016 30820 tablet_replica.cc:333] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.497684 30820 raft_consensus.cc:2243] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.498539 30820 raft_consensus.cc:2272] T 0d6c0fabd5a9486f95f088218e4ba65d P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.501107 30820 tablet_replica.cc:333] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.501785 30820 raft_consensus.cc:2243] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.502831 30820 raft_consensus.cc:2272] T 81129181ae3246c1bfb02954120c9b91 P 38c927f0ee104b57972899f5b27aa99b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.505465 30820 tablet_replica.cc:333] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
W20260709 12:21:30.505702 32626 consensus_peers.cc:597] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.506134 30820 raft_consensus.cc:2243] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.507306 30820 raft_consensus.cc:2272] T 1da57956732d44d1b912fb270de7d63f P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.511435 32626 consensus_peers.cc:597] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:30.513940 30820 tablet_replica.cc:333] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.515251 30820 raft_consensus.cc:2243] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.515789 30820 raft_consensus.cc:2272] T 2f93a3ac5b02416a8d1d2547d9a9c726 P 38c927f0ee104b57972899f5b27aa99b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.517910 30820 tablet_replica.cc:333] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.518474 30820 raft_consensus.cc:2243] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.518970 30820 raft_consensus.cc:2272] T 3e4bfb2be45e4fc38e6622722ab10735 P 38c927f0ee104b57972899f5b27aa99b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.520929 30820 tablet_replica.cc:333] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.521502 30820 raft_consensus.cc:2243] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.522300 30820 raft_consensus.cc:2272] T 5b2dd1839ae848ae849e1ec74ce14112 P 38c927f0ee104b57972899f5b27aa99b [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.524680 30820 tablet_replica.cc:333] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.525182 30820 raft_consensus.cc:2243] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.525954 30820 raft_consensus.cc:2272] T 12a3b65ef9024a64b336b00ad51e294b P 38c927f0ee104b57972899f5b27aa99b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.527673 30820 tablet_replica.cc:333] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.528119 30820 raft_consensus.cc:2243] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.528716 30820 raft_consensus.cc:2272] T a7f1c1160177412eb547493e1f3c561f P 38c927f0ee104b57972899f5b27aa99b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.530637 30820 tablet_replica.cc:333] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b: stopping tablet replica
I20260709 12:21:30.531077 30820 raft_consensus.cc:2243] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.531910 30820 raft_consensus.cc:2272] T 02ef57fadd044f68ba212dc2fe410f83 P 38c927f0ee104b57972899f5b27aa99b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.545521 32626 consensus_peers.cc:597] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.547248 32626 consensus_peers.cc:597] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.599449 32626 consensus_peers.cc:597] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.603735 32626 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.613016 30820 connection.cc:466] server connection from 127.0.0.1:34592 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34592 (request call id 2646) could send its response
W20260709 12:21:30.614440 32626 consensus_peers.cc:597] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.699416 32626 consensus_peers.cc:597] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.790856 32623 consensus_peers.cc:597] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:30.857872 32626 consensus_peers.cc:597] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Couldn't send request to peer 38c927f0ee104b57972899f5b27aa99b. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:30.918119 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:21:30.921281  8865 raft_consensus.cc:493] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:30.921871  8865 raft_consensus.cc:515] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:30.923978  8865 leader_election.cc:290] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:30.935169 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:30.935829 32623 leader_election.cc:336] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:30.936224 32623 leader_election.cc:304] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:30.937027  8867 raft_consensus.cc:2749] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:31.131686 32623 consensus_peers.cc:597] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:31.168876 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:21:31.185485 32623 consensus_peers.cc:597] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:21:31.211722 32623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111) [suppressed 584 similar messages]
W20260709 12:21:31.219050 32623 consensus_peers.cc:597] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:31.219899  8867 raft_consensus.cc:493] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:31.220474  8867 raft_consensus.cc:515] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.222640  8867 leader_election.cc:290] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:31.231720 32623 leader_election.cc:336] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:31.236696 32623 leader_election.cc:336] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.237198 32623 leader_election.cc:304] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.237927  8867 raft_consensus.cc:2749] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:31.246402 32623 consensus_peers.cc:597] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:21:31.315595 32623 consensus_peers.cc:597] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:31.325502  8867 raft_consensus.cc:493] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:31.327039  8867 raft_consensus.cc:515] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.329101  8867 leader_election.cc:290] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:31.338622 32623 leader_election.cc:336] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:31.339560 32623 leader_election.cc:336] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.339972 32623 leader_election.cc:304] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.340684  8867 raft_consensus.cc:2749] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:31.343842  8822 raft_consensus.cc:493] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:31.344352  8822 raft_consensus.cc:515] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.346376  8822 leader_election.cc:290] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903)
W20260709 12:21:31.357118 32623 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:31.357978 32623 leader_election.cc:336] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.358387 32623 leader_election.cc:304] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.359097  8867 raft_consensus.cc:2749] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:31.378340 32623 consensus_peers.cc:597] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:31.431299  8822 raft_consensus.cc:493] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.432978  8822 raft_consensus.cc:515] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.436488  8822 leader_election.cc:290] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.445303 32626 leader_election.cc:336] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
W20260709 12:21:31.449488 32623 consensus_peers.cc:597] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:31.450292 32623 leader_election.cc:336] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.450795 32623 leader_election.cc:304] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:31.451774  8822 raft_consensus.cc:2749] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:31.456501 32623 consensus_peers.cc:597] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:31.459926 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
W20260709 12:21:31.470260 32623 consensus_peers.cc:597] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:31.471956 32623 consensus_peers.cc:597] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:21:31.483162  8822 raft_consensus.cc:493] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:31.482914  8870 raft_consensus.cc:493] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
I20260709 12:21:31.484509  8822 raft_consensus.cc:515] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } attrs { replace: true } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.484760  8870 raft_consensus.cc:515] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.487561  8822 leader_election.cc:290] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.496287 32626 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.499878  8870 leader_election.cc:290] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
W20260709 12:21:31.501632 32623 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.502038 32623 leader_election.cc:304] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.502735  8870 raft_consensus.cc:2749] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:31.503257 32623 leader_election.cc:336] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
W20260709 12:21:31.506532 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.506709  8870 raft_consensus.cc:493] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3836f9b5cd36488da56d4540dc9d80ec)
W20260709 12:21:31.507283 32623 leader_election.cc:336] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.507727 32623 leader_election.cc:304] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.507901  8870 raft_consensus.cc:515] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.508562  8822 raft_consensus.cc:2749] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:31.513381  8870 leader_election.cc:290] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293)
I20260709 12:21:31.514400  8870 raft_consensus.cc:493] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.515290  8870 raft_consensus.cc:515] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.520962  8870 leader_election.cc:290] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.523537 32626 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
W20260709 12:21:31.525312 32623 leader_election.cc:336] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:31.526304 32623 leader_election.cc:336] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.526763 32623 leader_election.cc:304] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.527734  8870 raft_consensus.cc:2749] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:21:31.531152 32623 leader_election.cc:336] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.531611 32623 leader_election.cc:304] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:31.532474  8870 raft_consensus.cc:2749] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:31.549623 32623 consensus_peers.cc:597] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:31.552867  8870 raft_consensus.cc:493] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.555400  8870 raft_consensus.cc:515] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:31.563508 32626 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.561125  8870 leader_election.cc:290] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.565917 32623 leader_election.cc:336] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.566493 32623 leader_election.cc:304] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:31.567950  8870 raft_consensus.cc:2749] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:21:31.601065 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Couldn't send request to peer 3836f9b5cd36488da56d4540dc9d80ec. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:31.609620  8870 raft_consensus.cc:493] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.611001  8870 raft_consensus.cc:515] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
W20260709 12:21:31.620486 32626 leader_election.cc:336] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.626286  8870 leader_election.cc:290] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.628525 32623 leader_election.cc:336] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.629139 32623 leader_election.cc:304] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.630255  8870 raft_consensus.cc:2749] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:31.658809  8822 raft_consensus.cc:493] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.659462  8822 raft_consensus.cc:515] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.661316  8822 leader_election.cc:290] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.666448 32623 leader_election.cc:336] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
W20260709 12:21:31.668588 32626 leader_election.cc:336] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.668655 30820 tablet_server.cc:179] TabletServer@127.30.25.4:0 shutting down...
I20260709 12:21:31.669126 32626 leader_election.cc:304] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.670538  8822 raft_consensus.cc:2749] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:31.742632 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:31.745507 30820 tablet_replica.cc:333] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
W20260709 12:21:31.745463 32623 consensus_peers.cc:597] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 -> Peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Couldn't send request to peer f05e4741d35c4cf3a31358c56e65b30c. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:31.745335  8822 raft_consensus.cc:493] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.746913  8822 raft_consensus.cc:515] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.747217 30820 raft_consensus.cc:2243] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.748688 30820 raft_consensus.cc:2272] T b88d8f995fcc4faea19a190f8dd669ab P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.748859  8822 leader_election.cc:290] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:31.753358 30820 tablet_replica.cc:333] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.754459 30820 raft_consensus.cc:2243] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.755724  8870 raft_consensus.cc:493] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.756556 30820 raft_consensus.cc:2272] T d3949314523941858ae6d6a4fd770fb8 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.756512  8870 raft_consensus.cc:515] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.759028  8870 leader_election.cc:290] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.759902 32626 leader_election.cc:336] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.761040 30820 tablet_replica.cc:333] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
W20260709 12:21:31.761219 32623 leader_election.cc:336] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.761787 32623 leader_election.cc:304] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.762372 30820 raft_consensus.cc:2243] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.762542  8870 raft_consensus.cc:2749] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:31.763038  8822 raft_consensus.cc:493] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.763682  8822 raft_consensus.cc:515] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3836f9b5cd36488da56d4540dc9d80ec" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 40293 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.764597 30820 raft_consensus.cc:2272] T 1a06d9856d134874b3144fd2abdbfc71 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.765842  8822 leader_election.cc:290] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
W20260709 12:21:31.768329 32626 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.769240 30820 tablet_replica.cc:333] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.770499 30820 raft_consensus.cc:2243] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 LEADER]: Raft consensus shutting down.
W20260709 12:21:31.771770 32626 leader_election.cc:336] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.772300 30820 raft_consensus.cc:2272] T 08b3d325c1944e45a59e666e91d0dbb6 P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:31.774148 32623 leader_election.cc:336] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.774662 32623 leader_election.cc:304] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:31.775435  8822 raft_consensus.cc:2749] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:31.777508 30820 tablet_replica.cc:333] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.778832 30820 raft_consensus.cc:2243] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:31.780534 32623 leader_election.cc:336] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3836f9b5cd36488da56d4540dc9d80ec (127.30.25.2:40293): Network error: Client connection negotiation failed: client connection to 127.30.25.2:40293: connect: Connection refused (error 111)
I20260709 12:21:31.780745 30820 raft_consensus.cc:2272] T 58ee45b18b714479af8291d9056afb9e P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.781000 32623 leader_election.cc:304] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 3836f9b5cd36488da56d4540dc9d80ec, 38c927f0ee104b57972899f5b27aa99b
I20260709 12:21:31.781736  8822 raft_consensus.cc:2749] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:31.784608 30820 tablet_replica.cc:333] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.785486 30820 raft_consensus.cc:2243] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.786859 30820 raft_consensus.cc:2272] T 786db0eb7e9b431ba90c5ad880c46c9e P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.790180 30820 tablet_replica.cc:333] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.791091 30820 raft_consensus.cc:2243] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.792604 30820 raft_consensus.cc:2272] T c50b710409be4b8986d1a0b839db7004 P ec7f26ea46944ea498ad253b1b1819b0 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.796435 30820 tablet_replica.cc:333] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.797097 30820 raft_consensus.cc:2243] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.798470 30820 raft_consensus.cc:2272] T 19357e2a1cf1408b9391be0c12c3b0e8 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.801367 30820 tablet_replica.cc:333] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.801977 30820 raft_consensus.cc:2243] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.803026 30820 raft_consensus.cc:2272] T 193a74284231469f9229c961dfaff799 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.806113 30820 tablet_replica.cc:333] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.806834 30820 raft_consensus.cc:2243] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.808579 30820 raft_consensus.cc:2272] T 6679c928e92e4f799b88357707650b60 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.812459 30820 tablet_replica.cc:333] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.813233 30820 raft_consensus.cc:2243] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.814795 30820 raft_consensus.cc:2272] T 1612f8e575bd46cf8069fa1c21ca192b P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.817910 30820 tablet_replica.cc:333] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.818639 30820 raft_consensus.cc:2243] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.819967 30820 raft_consensus.cc:2272] T b83e7300b3fe4d21a97d941dee98fa81 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.822440 30820 tablet_replica.cc:333] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.823149 30820 raft_consensus.cc:2243] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.824308 30820 raft_consensus.cc:2272] T 81129181ae3246c1bfb02954120c9b91 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.828255 30820 tablet_replica.cc:333] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.828894 30820 raft_consensus.cc:2243] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.830184 30820 raft_consensus.cc:2272] T 2f93a3ac5b02416a8d1d2547d9a9c726 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.833043 30820 tablet_replica.cc:333] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.833606 30820 raft_consensus.cc:2243] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.834699 30820 raft_consensus.cc:2272] T 0c82ba0f778d4ff98ba9cc8255f28e9d P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.835953  8822 raft_consensus.cc:493] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.836515  8822 raft_consensus.cc:515] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.837425 30820 tablet_replica.cc:333] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.838068 30820 raft_consensus.cc:2243] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.838817 30820 raft_consensus.cc:2272] T 826cffce8bb540c8beb56cc00a702782 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.841629 30820 tablet_replica.cc:333] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.842592 30820 raft_consensus.cc:2243] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.842689  8822 leader_election.cc:290] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:31.844116 30820 raft_consensus.cc:2272] T 7c52a06827f24bb6bc1851474fc80e53 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:31.844646 32626 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
W20260709 12:21:31.845155 32623 leader_election.cc:336] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.845585 32623 leader_election.cc:304] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.846285  8822 raft_consensus.cc:2749] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:31.847330  8871 raft_consensus.cc:493] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 38c927f0ee104b57972899f5b27aa99b)
I20260709 12:21:31.848773  8871 raft_consensus.cc:515] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f05e4741d35c4cf3a31358c56e65b30c" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 40903 } } peers { permanent_uuid: "38c927f0ee104b57972899f5b27aa99b" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35255 } } peers { permanent_uuid: "ec7f26ea46944ea498ad253b1b1819b0" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 39399 } attrs { promote: false } }
I20260709 12:21:31.847383 30820 tablet_replica.cc:333] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.850476 30820 raft_consensus.cc:2243] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.851255  8871 leader_election.cc:290] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903), 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255)
I20260709 12:21:31.851934 30820 raft_consensus.cc:2272] T 8f4287254496472398f69d3a10b48d77 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:31.855726 32626 leader_election.cc:336] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 38c927f0ee104b57972899f5b27aa99b (127.30.25.3:35255): Network error: Client connection negotiation failed: client connection to 127.30.25.3:35255: connect: Connection refused (error 111)
I20260709 12:21:31.856980 30820 tablet_replica.cc:333] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
W20260709 12:21:31.857700 32623 leader_election.cc:336] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f05e4741d35c4cf3a31358c56e65b30c (127.30.25.1:40903): Network error: Client connection negotiation failed: client connection to 127.30.25.1:40903: connect: Connection refused (error 111)
I20260709 12:21:31.858044 30820 raft_consensus.cc:2243] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.858126 32623 leader_election.cc:304] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec7f26ea46944ea498ad253b1b1819b0; no voters: 38c927f0ee104b57972899f5b27aa99b, f05e4741d35c4cf3a31358c56e65b30c
I20260709 12:21:31.858934 30820 raft_consensus.cc:2272] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.859334  8871 raft_consensus.cc:2706] T f3f78e3424864d319ee8da87f4848fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Received pre-election callback for term 27 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 12:21:31.862092 30820 tablet_replica.cc:333] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.862788 30820 raft_consensus.cc:2243] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.863296 30820 raft_consensus.cc:2272] T 15d2c5dc46254473902eca585b82958f P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.865525 30820 tablet_replica.cc:333] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.866065 30820 raft_consensus.cc:2243] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.867152 30820 raft_consensus.cc:2272] T 9d2cc7077f01430da5a31385d674e266 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.869563 30820 tablet_replica.cc:333] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.870260 30820 raft_consensus.cc:2243] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.871881 30820 raft_consensus.cc:2272] T 1c3ed5d095c544f0abfe4e2c375b06cc P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.875039 30820 tablet_replica.cc:333] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.875646 30820 raft_consensus.cc:2243] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.876909 30820 raft_consensus.cc:2272] T 3695041c497f47468894b0baf52fcfd0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.879199 30820 tablet_replica.cc:333] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.879750 30820 raft_consensus.cc:2243] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.880365 30820 raft_consensus.cc:2272] T 78b16558212549a98cd68f4e744eb410 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.882637 30820 tablet_replica.cc:333] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.883210 30820 raft_consensus.cc:2243] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.883986 30820 raft_consensus.cc:2272] T 8ca59f0a11104ab882fac419c003e8b7 P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.886972 30820 tablet_replica.cc:333] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.888069 30820 raft_consensus.cc:2243] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.889569 30820 raft_consensus.cc:2272] T f953fa305a42492da9de0fa8c18c43eb P ec7f26ea46944ea498ad253b1b1819b0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.891793 30820 tablet_replica.cc:333] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.892383 30820 raft_consensus.cc:2243] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.892968 30820 raft_consensus.cc:2272] T 6d02c13b00af439e946e76ef880668d9 P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.895272 30820 tablet_replica.cc:333] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.895917 30820 raft_consensus.cc:2243] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.896643 30820 raft_consensus.cc:2272] T 0d6c0fabd5a9486f95f088218e4ba65d P ec7f26ea46944ea498ad253b1b1819b0 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.899461 30820 tablet_replica.cc:333] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.900087 30820 raft_consensus.cc:2243] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.900611 30820 raft_consensus.cc:2272] T 4633b71e43824d84895ff136e5765fb1 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.903234 30820 tablet_replica.cc:333] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.903780 30820 raft_consensus.cc:2243] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.904748 30820 raft_consensus.cc:2272] T 0292f21aab4a4843b2b424c3497ddced P ec7f26ea46944ea498ad253b1b1819b0 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.906930 30820 tablet_replica.cc:333] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.907488 30820 raft_consensus.cc:2243] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.914933 30820 raft_consensus.cc:2272] T bfedf707307443deb5a0681825db83cb P ec7f26ea46944ea498ad253b1b1819b0 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.917495 30820 tablet_replica.cc:333] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.918028 30820 raft_consensus.cc:2243] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.919176 30820 raft_consensus.cc:2272] T 3e4bfb2be45e4fc38e6622722ab10735 P ec7f26ea46944ea498ad253b1b1819b0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.921319 30820 tablet_replica.cc:333] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.921957 30820 raft_consensus.cc:2243] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.923512 30820 raft_consensus.cc:2272] T 775fd46a41044b1ea06bc796ef1b233b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.926404 30820 tablet_replica.cc:333] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.926939 30820 raft_consensus.cc:2243] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.927913 30820 raft_consensus.cc:2272] T 6d27358a7bd3493a8877bd9ce8ae9055 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.929953 30820 tablet_replica.cc:333] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.930675 30820 raft_consensus.cc:2243] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.931408 30820 raft_consensus.cc:2272] T 51b3fd343e81436fa5bea82486abcc64 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.934968 30820 tablet_replica.cc:333] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.935834 30820 raft_consensus.cc:2243] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.936440 30820 raft_consensus.cc:2272] T fbb363b96dbf430998ab5641d3b03f46 P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.938671 30820 tablet_replica.cc:333] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.939153 30820 raft_consensus.cc:2243] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.939755 30820 raft_consensus.cc:2272] T 8709b6ab6ebf4e279d1fe057a1ac58b5 P ec7f26ea46944ea498ad253b1b1819b0 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.941752 30820 tablet_replica.cc:333] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.942293 30820 raft_consensus.cc:2243] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.943357 30820 raft_consensus.cc:2272] T 478b63c892a745d09e9508bc9cc5321f P ec7f26ea46944ea498ad253b1b1819b0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.945864 30820 tablet_replica.cc:333] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.946502 30820 raft_consensus.cc:2243] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.947466 30820 raft_consensus.cc:2272] T 2c59a3d3077c4b8db6380a834f6ce3c0 P ec7f26ea46944ea498ad253b1b1819b0 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.949599 30820 tablet_replica.cc:333] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.950078 30820 raft_consensus.cc:2243] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.950749 30820 raft_consensus.cc:2272] T 0e910ef3c0af41bf8d4fba5315d68eef P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.952991 30820 tablet_replica.cc:333] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.953498 30820 raft_consensus.cc:2243] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.954172 30820 raft_consensus.cc:2272] T 12a3b65ef9024a64b336b00ad51e294b P ec7f26ea46944ea498ad253b1b1819b0 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.956686 30820 tablet_replica.cc:333] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.957173 30820 raft_consensus.cc:2243] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.957643 30820 raft_consensus.cc:2272] T 5b2dd1839ae848ae849e1ec74ce14112 P ec7f26ea46944ea498ad253b1b1819b0 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.959690 30820 tablet_replica.cc:333] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.960152 30820 raft_consensus.cc:2243] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.961125 30820 raft_consensus.cc:2272] T 422b4d293d444b06bb00c3f5b66a6f25 P ec7f26ea46944ea498ad253b1b1819b0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.963086 30820 tablet_replica.cc:333] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.963539 30820 raft_consensus.cc:2243] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.964537 30820 raft_consensus.cc:2272] T e828882e6b1940ffb33d4c2f76b47fe9 P ec7f26ea46944ea498ad253b1b1819b0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.966552 30820 tablet_replica.cc:333] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0: stopping tablet replica
I20260709 12:21:31.966993 30820 raft_consensus.cc:2243] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.967689 30820 raft_consensus.cc:2272] T 71e0cf115549454c894335e70c43843a P ec7f26ea46944ea498ad253b1b1819b0 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:32.475659 30820 tablet_server.cc:196] TabletServer@127.30.25.4:0 shutdown complete.
I20260709 12:21:32.583748 30820 master.cc:562] Master@127.30.25.62:33695 shutting down...
W20260709 12:21:32.624850 31918 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:21:32.625192 31918 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:21:32.625432 31918 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 12:21:32.630062 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:32.630780 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:32.631162 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P a0887d9cdc8044caa86af7acdb454104: stopping tablet replica
I20260709 12:21:32.835871 30820 master.cc:584] Master@127.30.25.62:33695 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (294522 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:21:32.946621 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:37527
I20260709 12:21:32.947804 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:32.953652  8874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:32.953753  8873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:32.954509  8876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:32.956391 30820 server_base.cc:1061] running on GCE node
I20260709 12:21:32.960163 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:32.960352 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:32.960546 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599692960511 us; error 0 us; skew 500 ppm
I20260709 12:21:32.961059 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:32.963452 30820 webserver.cc:533] Webserver started at http://127.30.25.62:33931/ using document root <none> and password file <none>
I20260709 12:21:32.963991 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:32.964210 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:32.964507 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:32.965729 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-rldg"
I20260709 12:21:32.970136 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:32.973479  8881 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.974401 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:32.974753 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-rldg"
I20260709 12:21:32.975142 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:32.988788 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:32.989985 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:33.027817 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:37527
I20260709 12:21:33.027917  8932 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:37527 every 8 connection(s)
I20260709 12:21:33.031841  8933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.042296  8933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d: Bootstrap starting.
I20260709 12:21:33.047330  8933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.051569  8933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d: No bootstrap required, opened a new log
I20260709 12:21:33.053730  8933 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" member_type: VOTER }
I20260709 12:21:33.054116  8933 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.054288  8933 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a02ca4df2c94fc2b3ecca76eb0a1a9d, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.054945  8933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" member_type: VOTER }
I20260709 12:21:33.055365  8933 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:33.055568  8933 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:33.055774  8933 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.060191  8933 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" member_type: VOTER }
I20260709 12:21:33.060648  8933 leader_election.cc:304] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a02ca4df2c94fc2b3ecca76eb0a1a9d; no voters: 
I20260709 12:21:33.061697  8933 leader_election.cc:290] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:33.062033  8936 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.063500  8936 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 1 LEADER]: Becoming Leader. State: Replica: 7a02ca4df2c94fc2b3ecca76eb0a1a9d, State: Running, Role: LEADER
I20260709 12:21:33.064244  8936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" member_type: VOTER }
I20260709 12:21:33.064862  8933 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:33.070693  8937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" member_type: VOTER } }
I20260709 12:21:33.071368  8937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:33.073021  8944 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:33.072659  8938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a02ca4df2c94fc2b3ecca76eb0a1a9d. Latest consensus state: current_term: 1 leader_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a02ca4df2c94fc2b3ecca76eb0a1a9d" member_type: VOTER } }
I20260709 12:21:33.073493  8938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:33.078526  8944 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:33.082603 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:33.088050  8944 catalog_manager.cc:1383] Generated new cluster ID: a4a4e92d183143b0b6d7ba1d4d5a29a7
I20260709 12:21:33.088357  8944 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:33.123721  8944 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:33.125023  8944 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:33.140074  8944 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d: Generated new TSK 0
I20260709 12:21:33.140772  8944 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:33.149325 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:33.155668  8954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:33.156659  8955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:33.158365  8957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:33.159721 30820 server_base.cc:1061] running on GCE node
I20260709 12:21:33.162094 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:33.162289 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:33.162498 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599693162481 us; error 0 us; skew 500 ppm
I20260709 12:21:33.162992 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:33.165326 30820 webserver.cc:533] Webserver started at http://127.30.25.1:42335/ using document root <none> and password file <none>
I20260709 12:21:33.165769 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:33.165921 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:33.166141 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:33.167349 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "e47ee72c4e114aef9818c7c324ed1938"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.172086 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:21:33.175417  8962 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.176344 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:33.176647 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "e47ee72c4e114aef9818c7c324ed1938"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.176942 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:33.198757 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:33.199970 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:33.201467 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:33.203806 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:33.204051 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.204296 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:33.204479 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.246021 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:34461
I20260709 12:21:33.246136  9024 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:34461 every 8 connection(s)
I20260709 12:21:33.251151 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:33.259037  9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:33.259545  9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:33.262706  9032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:33.264000 30820 server_base.cc:1061] running on GCE node
I20260709 12:21:33.266940 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:33.267220 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:33.267459 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599693267441 us; error 0 us; skew 500 ppm
I20260709 12:21:33.268153 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:33.270473  9025 heartbeater.cc:344] Connected to a master server at 127.30.25.62:37527
I20260709 12:21:33.270862  9025 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:33.271168 30820 webserver.cc:533] Webserver started at http://127.30.25.2:42951/ using document root <none> and password file <none>
I20260709 12:21:33.271718 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:33.271667  9025 heartbeater.cc:507] Master 127.30.25.62:37527 requested a full tablet report, sending...
I20260709 12:21:33.272101 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:33.272409 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:33.273761 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "0ded781e200f40a1b78076731e570fb7"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.274065  8898 ts_manager.cc:194] Registered new tserver with Master: e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:33.275942  8898 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51526
I20260709 12:21:33.278936 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:21:33.282522  9037 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.283349 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:21:33.283677 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "0ded781e200f40a1b78076731e570fb7"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.283980 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:33.295277 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:33.296887 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:33.298418 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:33.300690 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:33.300920 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.301158 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:33.301360 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.341960 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:43631
I20260709 12:21:33.342059  9099 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:43631 every 8 connection(s)
I20260709 12:21:33.346936 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:33.354750  9103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:33.355998  9104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:33.359216 30820 server_base.cc:1061] running on GCE node
W20260709 12:21:33.359647  9106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:33.360291  9100 heartbeater.cc:344] Connected to a master server at 127.30.25.62:37527
I20260709 12:21:33.360679  9100 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:33.360746 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:33.361080 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:33.361263 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599693361246 us; error 0 us; skew 500 ppm
I20260709 12:21:33.361506  9100 heartbeater.cc:507] Master 127.30.25.62:37527 requested a full tablet report, sending...
I20260709 12:21:33.361913 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:33.363801  8898 ts_manager.cc:194] Registered new tserver with Master: 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:33.364850 30820 webserver.cc:533] Webserver started at http://127.30.25.3:33499/ using document root <none> and password file <none>
I20260709 12:21:33.365501 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:33.365733 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:33.365825  8898 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51538
I20260709 12:21:33.366148 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:33.367656 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.372267 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:21:33.375635  9111 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.376430 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:21:33.376788 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.377058 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:33.416250 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:33.417486 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:33.419009 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:33.421393 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:33.421624 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.421866 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:33.422044 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.462675 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:35913
I20260709 12:21:33.462785  9173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:35913 every 8 connection(s)
I20260709 12:21:33.467550 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:33.474865  9177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:33.476522  9178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:33.480607  9174 heartbeater.cc:344] Connected to a master server at 127.30.25.62:37527
W20260709 12:21:33.480728  9180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:33.480980 30820 server_base.cc:1061] running on GCE node
I20260709 12:21:33.481194  9174 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:33.482137 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:21:33.482223  9174 heartbeater.cc:507] Master 127.30.25.62:37527 requested a full tablet report, sending...
W20260709 12:21:33.482513 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:33.482762 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599693482744 us; error 0 us; skew 500 ppm
I20260709 12:21:33.483475 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:33.484846  8898 ts_manager.cc:194] Registered new tserver with Master: aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:33.486171 30820 webserver.cc:533] Webserver started at http://127.30.25.4:37693/ using document root <none> and password file <none>
I20260709 12:21:33.486650  8898 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51550
I20260709 12:21:33.486785 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:33.487126 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:33.487495 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:33.489041 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-3-root/instance:
uuid: "343eaafd515a4dfc9de3df299924f0d4"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.493762 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:21:33.497116  9185 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.497965 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:33.498278 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-3-root
uuid: "343eaafd515a4dfc9de3df299924f0d4"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:33.498661 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:33.518935 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:33.520223 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:33.521696 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:33.523921 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:33.524179 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.524447 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:33.524642 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:33.569453 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.4:38651
I20260709 12:21:33.569571  9247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.4:38651 every 8 connection(s)
I20260709 12:21:33.585129  9248 heartbeater.cc:344] Connected to a master server at 127.30.25.62:37527
I20260709 12:21:33.585764  9248 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:33.586611  9248 heartbeater.cc:507] Master 127.30.25.62:37527 requested a full tablet report, sending...
I20260709 12:21:33.588424  8898 ts_manager.cc:194] Registered new tserver with Master: 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:33.588745 30820 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015083126s
I20260709 12:21:33.589857  8898 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51564
I20260709 12:21:33.615394  8898 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51574:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:21:33.856446  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 45855a20f5a4404cb8ba32f93c294288 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:33.857553  9212 tablet_service.cc:1511] Processing CreateTablet for tablet 2426191b92b948e4b607c976696af51f (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:33.859411  9065 tablet_service.cc:1511] Processing CreateTablet for tablet 45855a20f5a4404cb8ba32f93c294288 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:33.860782  9065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45855a20f5a4404cb8ba32f93c294288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.861536  9063 tablet_service.cc:1511] Processing CreateTablet for tablet 2426191b92b948e4b607c976696af51f (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:33.862066  9064 tablet_service.cc:1511] Processing CreateTablet for tablet bec4c3600c2a4b22876a710877cf531d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:33.862965  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2426191b92b948e4b607c976696af51f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.864188  9062 tablet_service.cc:1511] Processing CreateTablet for tablet d77c1d2e7167495fbe8556f3ba691a55 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:33.864346  9211 tablet_service.cc:1511] Processing CreateTablet for tablet c7f9c182103549958915a7a7ed1eab44 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:33.865556  9062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d77c1d2e7167495fbe8556f3ba691a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.867556  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec4c3600c2a4b22876a710877cf531d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.857970  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45855a20f5a4404cb8ba32f93c294288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.874761  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2426191b92b948e4b607c976696af51f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.879392  9211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f9c182103549958915a7a7ed1eab44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.884402  9210 tablet_service.cc:1511] Processing CreateTablet for tablet 149942a6b05d454496753897bc2ed0b8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:33.885869  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 149942a6b05d454496753897bc2ed0b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.893680  9139 tablet_service.cc:1511] Processing CreateTablet for tablet 45855a20f5a4404cb8ba32f93c294288 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:33.895180  9139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45855a20f5a4404cb8ba32f93c294288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.899457  9138 tablet_service.cc:1511] Processing CreateTablet for tablet bec4c3600c2a4b22876a710877cf531d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:33.900967  9138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec4c3600c2a4b22876a710877cf531d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.903474  9137 tablet_service.cc:1511] Processing CreateTablet for tablet c7f9c182103549958915a7a7ed1eab44 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:33.904937  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f9c182103549958915a7a7ed1eab44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.907488  9136 tablet_service.cc:1511] Processing CreateTablet for tablet d77c1d2e7167495fbe8556f3ba691a55 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:33.908866  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d77c1d2e7167495fbe8556f3ba691a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.912937  9268 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:33.912842  9061 tablet_service.cc:1511] Processing CreateTablet for tablet 149942a6b05d454496753897bc2ed0b8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:33.916702  9209 tablet_service.cc:1511] Processing CreateTablet for tablet 571bcffedf6f4da59792575fd301ed0d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:33.918113  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571bcffedf6f4da59792575fd301ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.924578  8990 tablet_service.cc:1511] Processing CreateTablet for tablet bec4c3600c2a4b22876a710877cf531d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:33.925666  8988 tablet_service.cc:1511] Processing CreateTablet for tablet c7f9c182103549958915a7a7ed1eab44 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:33.926641  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec4c3600c2a4b22876a710877cf531d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.927552  8989 tablet_service.cc:1511] Processing CreateTablet for tablet 2426191b92b948e4b607c976696af51f (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:33.928020  8987 tablet_service.cc:1511] Processing CreateTablet for tablet d77c1d2e7167495fbe8556f3ba691a55 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:33.928931  8989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2426191b92b948e4b607c976696af51f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.930246  8986 tablet_service.cc:1511] Processing CreateTablet for tablet 149942a6b05d454496753897bc2ed0b8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:33.931721  8986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 149942a6b05d454496753897bc2ed0b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.938724  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d77c1d2e7167495fbe8556f3ba691a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.942754  8988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f9c182103549958915a7a7ed1eab44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.960208  9270 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:33.970990  9270 tablet_bootstrap.cc:654] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.984263  9272 tablet_bootstrap.cc:492] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:33.924432  9135 tablet_service.cc:1511] Processing CreateTablet for tablet 571bcffedf6f4da59792575fd301ed0d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:33.989252  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571bcffedf6f4da59792575fd301ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.925133  9061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 149942a6b05d454496753897bc2ed0b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.001983  8989 tablet_service.cc:1511] Processing CreateTablet for tablet 571bcffedf6f4da59792575fd301ed0d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:34.011436  8989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571bcffedf6f4da59792575fd301ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:33.919549  9268 tablet_bootstrap.cc:654] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.987564  9270 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:34.026578  9064 tablet_service.cc:1511] Processing CreateTablet for tablet 2290f27d484a4682a825fb8d1bb87924 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:34.027004  9270 ts_tablet_manager.cc:1403] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.011s
I20260709 12:21:34.028050  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2290f27d484a4682a825fb8d1bb87924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.031791  9270 raft_consensus.cc:359] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.036926  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 2290f27d484a4682a825fb8d1bb87924 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:34.038290  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2290f27d484a4682a825fb8d1bb87924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.044215  9062 tablet_service.cc:1511] Processing CreateTablet for tablet c70307298bd54a0a8f754e99eb9cfda5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:34.045684  9062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c70307298bd54a0a8f754e99eb9cfda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.033453  9270 raft_consensus.cc:385] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.046119  9270 raft_consensus.cc:740] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.047089  9270 consensus_queue.cc:260] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.047789  9136 tablet_service.cc:1511] Processing CreateTablet for tablet c70307298bd54a0a8f754e99eb9cfda5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:34.047931  9212 tablet_service.cc:1511] Processing CreateTablet for tablet ed17e2a536b342e3b9222a83b27741c8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:34.049204  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c70307298bd54a0a8f754e99eb9cfda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.049274  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed17e2a536b342e3b9222a83b27741c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.051956  9272 tablet_bootstrap.cc:654] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.051697  9064 tablet_service.cc:1511] Processing CreateTablet for tablet ed17e2a536b342e3b9222a83b27741c8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:34.053176  9211 tablet_service.cc:1511] Processing CreateTablet for tablet 2290f27d484a4682a825fb8d1bb87924 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:34.055387  9138 tablet_service.cc:1511] Processing CreateTablet for tablet c992391e9d4940a3b550c8c511b354d1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:34.063120  9138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c992391e9d4940a3b550c8c511b354d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.063169  9211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2290f27d484a4682a825fb8d1bb87924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.056095  8987 tablet_service.cc:1511] Processing CreateTablet for tablet ed17e2a536b342e3b9222a83b27741c8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:34.039227  9269 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:34.059039  9210 tablet_service.cc:1511] Processing CreateTablet for tablet c992391e9d4940a3b550c8c511b354d1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:34.067071  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed17e2a536b342e3b9222a83b27741c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.067299  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c992391e9d4940a3b550c8c511b354d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.059628  8986 tablet_service.cc:1511] Processing CreateTablet for tablet c70307298bd54a0a8f754e99eb9cfda5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:34.060225  8988 tablet_service.cc:1511] Processing CreateTablet for tablet c992391e9d4940a3b550c8c511b354d1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:34.071883  9065 tablet_service.cc:1511] Processing CreateTablet for tablet 31ddf308b7944a55845b113b19560440 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:34.074988  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed17e2a536b342e3b9222a83b27741c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.075274  9139 tablet_service.cc:1511] Processing CreateTablet for tablet 31ddf308b7944a55845b113b19560440 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:34.079634  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 31ddf308b7944a55845b113b19560440 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
W20260709 12:21:34.080782  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:34.081004  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ddf308b7944a55845b113b19560440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.081130  8988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c992391e9d4940a3b550c8c511b354d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.082417  8990 tablet_service.cc:1511] Processing CreateTablet for tablet fec75531e4924e7b9308bdd3a269b06e (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:34.083791  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fec75531e4924e7b9308bdd3a269b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.088542  9139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ddf308b7944a55845b113b19560440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.102299  9065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ddf308b7944a55845b113b19560440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.109092  9270 ts_tablet_manager.cc:1434] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.006s
I20260709 12:21:34.110249  9270 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:34.110353  9025 heartbeater.cc:499] Master 127.30.25.62:37527 was elected leader, sending a full tablet report...
I20260709 12:21:34.111014  8986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c70307298bd54a0a8f754e99eb9cfda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.109232  9061 tablet_service.cc:1511] Processing CreateTablet for tablet fec75531e4924e7b9308bdd3a269b06e (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:34.114714  9061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fec75531e4924e7b9308bdd3a269b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.114835  9209 tablet_service.cc:1511] Processing CreateTablet for tablet 72198f72a6de43099cba6530224ef11d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:34.116171  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72198f72a6de43099cba6530224ef11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.123250  9063 tablet_service.cc:1511] Processing CreateTablet for tablet 72198f72a6de43099cba6530224ef11d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:34.124223  9268 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:34.124632  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72198f72a6de43099cba6530224ef11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.124795  9268 ts_tablet_manager.cc:1403] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.212s	user 0.014s	sys 0.000s
I20260709 12:21:34.128010  9268 raft_consensus.cc:359] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:34.128275  8985 tablet_service.cc:1511] Processing CreateTablet for tablet 72198f72a6de43099cba6530224ef11d (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:34.128768  9268 raft_consensus.cc:385] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.129145  9268 raft_consensus.cc:740] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.129747  8985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72198f72a6de43099cba6530224ef11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.130144  9268 consensus_queue.cc:260] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:34.148276  9270 tablet_bootstrap.cc:654] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.156283  9268 ts_tablet_manager.cc:1434] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 12:21:34.157169  9100 heartbeater.cc:499] Master 127.30.25.62:37527 was elected leader, sending a full tablet report...
I20260709 12:21:34.162948  9268 tablet_bootstrap.cc:492] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:34.172545  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 8fdefe9cd9cb4f9f821cf256b4ec6f6e (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:34.173511  9061 tablet_service.cc:1511] Processing CreateTablet for tablet 2efbde85ba184e82a1ceb3e8ded5d1f5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:34.174055  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fdefe9cd9cb4f9f821cf256b4ec6f6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.174896  9061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efbde85ba184e82a1ceb3e8ded5d1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.204464  9272 tablet_bootstrap.cc:492] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:34.205014  9272 ts_tablet_manager.cc:1403] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.221s	user 0.013s	sys 0.007s
I20260709 12:21:34.208452  9272 raft_consensus.cc:359] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.209326  9272 raft_consensus.cc:385] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.209709  9272 raft_consensus.cc:740] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.175369  9135 tablet_service.cc:1511] Processing CreateTablet for tablet fec75531e4924e7b9308bdd3a269b06e (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:34.179759  8989 tablet_service.cc:1511] Processing CreateTablet for tablet 8fdefe9cd9cb4f9f821cf256b4ec6f6e (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:34.182806  9138 tablet_service.cc:1511] Processing CreateTablet for tablet 2efbde85ba184e82a1ceb3e8ded5d1f5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:34.183266  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 2efbde85ba184e82a1ceb3e8ded5d1f5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:34.211210  8989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fdefe9cd9cb4f9f821cf256b4ec6f6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.211990  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efbde85ba184e82a1ceb3e8ded5d1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.186162  9210 tablet_service.cc:1511] Processing CreateTablet for tablet 8fdefe9cd9cb4f9f821cf256b4ec6f6e (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:34.215154  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fdefe9cd9cb4f9f821cf256b4ec6f6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.188103  9268 tablet_bootstrap.cc:654] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.220024  9063 tablet_service.cc:1511] Processing CreateTablet for tablet 8984e03dbda34e0fb5f577335d3870ec (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:34.211210  9138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efbde85ba184e82a1ceb3e8ded5d1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.221529  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8984e03dbda34e0fb5f577335d3870ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.221007  9272 consensus_queue.cc:260] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.233302  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fec75531e4924e7b9308bdd3a269b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.200913  9269 tablet_bootstrap.cc:654] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.243402  8988 tablet_service.cc:1511] Processing CreateTablet for tablet 8984e03dbda34e0fb5f577335d3870ec (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:34.245153  9209 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc9d94cd3304974abe8834d2f696077 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:34.246686  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc9d94cd3304974abe8834d2f696077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.248099  9268 tablet_bootstrap.cc:492] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:34.248719  9268 ts_tablet_manager.cc:1403] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.086s	user 0.001s	sys 0.014s
W20260709 12:21:34.250743  9026 tablet.cc:2406] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:34.251993  9270 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:34.252556  9270 ts_tablet_manager.cc:1403] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.143s	user 0.013s	sys 0.013s
I20260709 12:21:34.253480  9268 raft_consensus.cc:359] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.245157  8988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8984e03dbda34e0fb5f577335d3870ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.256049  9174 heartbeater.cc:499] Master 127.30.25.62:37527 was elected leader, sending a full tablet report...
I20260709 12:21:34.256413  9268 raft_consensus.cc:385] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.257591  9272 ts_tablet_manager.cc:1434] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.052s	user 0.010s	sys 0.000s
I20260709 12:21:34.259859  9272 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:34.261515  9139 tablet_service.cc:1511] Processing CreateTablet for tablet 8984e03dbda34e0fb5f577335d3870ec (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:34.262382  9268 raft_consensus.cc:740] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.262956  8987 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc9d94cd3304974abe8834d2f696077 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:34.267552  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc9d94cd3304974abe8834d2f696077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.269131  9139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8984e03dbda34e0fb5f577335d3870ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.267416  9268 consensus_queue.cc:260] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.275931  9268 ts_tablet_manager.cc:1434] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:21:34.279966  9268 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:34.264678  9063 tablet_service.cc:1511] Processing CreateTablet for tablet 4cc9d94cd3304974abe8834d2f696077 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:34.286346  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc9d94cd3304974abe8834d2f696077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.282244  9272 tablet_bootstrap.cc:654] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.293357  9138 tablet_service.cc:1511] Processing CreateTablet for tablet bdb71a58f7994546bdcf177f0ec7b6b1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:34.293781  9270 raft_consensus.cc:359] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:34.294696  9270 raft_consensus.cc:385] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.294972  9138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb71a58f7994546bdcf177f0ec7b6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.295049  9270 raft_consensus.cc:740] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.277544  9269 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:34.296749  9269 ts_tablet_manager.cc:1403] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.258s	user 0.014s	sys 0.000s
I20260709 12:21:34.301239  9213 tablet_service.cc:1511] Processing CreateTablet for tablet bdb71a58f7994546bdcf177f0ec7b6b1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:34.302688  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb71a58f7994546bdcf177f0ec7b6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.307973  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 4507e0b0b1a74cf98c0b22397b2d56f1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:34.309556  9268 tablet_bootstrap.cc:654] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.314742  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4507e0b0b1a74cf98c0b22397b2d56f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.308218  9270 consensus_queue.cc:260] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:34.316687  9270 ts_tablet_manager.cc:1434] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.064s	user 0.005s	sys 0.005s
I20260709 12:21:34.312469  9269 raft_consensus.cc:359] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:34.317613  8988 tablet_service.cc:1511] Processing CreateTablet for tablet 1f703417858b43a1b0d7affdd86231f6 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:34.326844  8988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f703417858b43a1b0d7affdd86231f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.327008  8989 tablet_service.cc:1511] Processing CreateTablet for tablet bdb71a58f7994546bdcf177f0ec7b6b1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:34.326889  9270 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:34.329739  9276 raft_consensus.cc:493] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.330265  9276 raft_consensus.cc:515] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:34.334064  9276 leader_election.cc:290] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:34.335124  8989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb71a58f7994546bdcf177f0ec7b6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.324186  9269 raft_consensus.cc:385] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.339442  9269 raft_consensus.cc:740] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.362092  9065 tablet_service.cc:1511] Processing CreateTablet for tablet 1f703417858b43a1b0d7affdd86231f6 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:34.363538  9065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f703417858b43a1b0d7affdd86231f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.350195  9135 tablet_service.cc:1511] Processing CreateTablet for tablet 1f703417858b43a1b0d7affdd86231f6 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:34.367584  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f703417858b43a1b0d7affdd86231f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.376766  8987 tablet_service.cc:1511] Processing CreateTablet for tablet 39d47714f6214edbaabd7255530669f6 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:34.377813  9269 consensus_queue.cc:260] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:34.378170  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d47714f6214edbaabd7255530669f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.378435  9062 tablet_service.cc:1511] Processing CreateTablet for tablet 39d47714f6214edbaabd7255530669f6 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:34.379868  9062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d47714f6214edbaabd7255530669f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.383778  9211 tablet_service.cc:1511] Processing CreateTablet for tablet ed9ca59931ad482da231fc05861e3b93 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:34.385154  9211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed9ca59931ad482da231fc05861e3b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.387321  8985 tablet_service.cc:1511] Processing CreateTablet for tablet ed9ca59931ad482da231fc05861e3b93 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:34.388650  8985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed9ca59931ad482da231fc05861e3b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.399893  9064 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c48e852b04e0099cd6a98a6020814 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:34.401427  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c48e852b04e0099cd6a98a6020814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.402374  9061 tablet_service.cc:1511] Processing CreateTablet for tablet 4507e0b0b1a74cf98c0b22397b2d56f1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:34.403784  9061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4507e0b0b1a74cf98c0b22397b2d56f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.404198  9138 tablet_service.cc:1511] Processing CreateTablet for tablet ed9ca59931ad482da231fc05861e3b93 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:34.407243  9272 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:34.407835  9272 ts_tablet_manager.cc:1403] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.148s	user 0.011s	sys 0.005s
I20260709 12:21:34.357086  9210 tablet_service.cc:1511] Processing CreateTablet for tablet 4507e0b0b1a74cf98c0b22397b2d56f1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:34.409252  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4507e0b0b1a74cf98c0b22397b2d56f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.369786  9209 tablet_service.cc:1511] Processing CreateTablet for tablet 39d47714f6214edbaabd7255530669f6 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:34.410249  9138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed9ca59931ad482da231fc05861e3b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.410332  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39d47714f6214edbaabd7255530669f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.433248  9274 raft_consensus.cc:493] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.434078  9274 raft_consensus.cc:515] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.445669  9272 raft_consensus.cc:359] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.446533  9272 raft_consensus.cc:385] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.446832  9272 raft_consensus.cc:740] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.451159  9274 leader_election.cc:290] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:34.453568  9286 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.454115  9286 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:34.454738  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c48e852b04e0099cd6a98a6020814 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:34.466328  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c48e852b04e0099cd6a98a6020814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.467031  8990 tablet_service.cc:1511] Processing CreateTablet for tablet 45f96b48cfe14f749d63e8cb99981b8b (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:34.469193  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45f96b48cfe14f749d63e8cb99981b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.467622  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c48e852b04e0099cd6a98a6020814 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:34.459803  9269 ts_tablet_manager.cc:1434] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.162s	user 0.007s	sys 0.005s
I20260709 12:21:34.479310  9248 heartbeater.cc:499] Master 127.30.25.62:37527 was elected leader, sending a full tablet report...
I20260709 12:21:34.482121  9060 tablet_service.cc:1511] Processing CreateTablet for tablet 45f96b48cfe14f749d63e8cb99981b8b (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:34.483664  9060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45f96b48cfe14f749d63e8cb99981b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.495151  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c48e852b04e0099cd6a98a6020814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.502986  9270 tablet_bootstrap.cc:654] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.506659  9268 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:34.507207  9268 ts_tablet_manager.cc:1403] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.227s	user 0.005s	sys 0.011s
I20260709 12:21:34.509689  9269 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:34.518004  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:34.524712  9187 leader_election.cc:343] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.532402  9139 tablet_service.cc:1511] Processing CreateTablet for tablet 45f96b48cfe14f749d63e8cb99981b8b (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:34.533676  9211 tablet_service.cc:1511] Processing CreateTablet for tablet 44e11b03c5914827946641c953501ec8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:34.535095  9211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e11b03c5914827946641c953501ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.538066  9136 tablet_service.cc:1511] Processing CreateTablet for tablet 25135f5b10d44185b9d6be0f2fb5a02c (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:34.541196  8987 tablet_service.cc:1511] Processing CreateTablet for tablet 44e11b03c5914827946641c953501ec8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:34.542575  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e11b03c5914827946641c953501ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.542783  9064 tablet_service.cc:1511] Processing CreateTablet for tablet bcf8624d5330403dbdebaeaabf396c07 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:34.544553  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf8624d5330403dbdebaeaabf396c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.547344  8988 tablet_service.cc:1511] Processing CreateTablet for tablet 25135f5b10d44185b9d6be0f2fb5a02c (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:34.548692  8988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25135f5b10d44185b9d6be0f2fb5a02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.549363  9134 tablet_service.cc:1511] Processing CreateTablet for tablet bcf8624d5330403dbdebaeaabf396c07 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:34.551800  9212 tablet_service.cc:1511] Processing CreateTablet for tablet 25135f5b10d44185b9d6be0f2fb5a02c (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:34.553148  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25135f5b10d44185b9d6be0f2fb5a02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.555076  9063 tablet_service.cc:1511] Processing CreateTablet for tablet eb7ab6e4c736485fbc461ccb19ea580a (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:34.555356  9062 tablet_service.cc:1511] Processing CreateTablet for tablet 76d74a33940c4299aa703e8d872d9375 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:34.558970  9133 tablet_service.cc:1511] Processing CreateTablet for tablet eb7ab6e4c736485fbc461ccb19ea580a (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:34.560115  8986 tablet_service.cc:1511] Processing CreateTablet for tablet eb7ab6e4c736485fbc461ccb19ea580a (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:34.566422  8986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7ab6e4c736485fbc461ccb19ea580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.566160  8984 tablet_service.cc:1511] Processing CreateTablet for tablet 76d74a33940c4299aa703e8d872d9375 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:34.567600  8984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d74a33940c4299aa703e8d872d9375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.561600  9139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45f96b48cfe14f749d63e8cb99981b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.521911  9268 raft_consensus.cc:359] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.581645  9268 raft_consensus.cc:385] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.582050  8983 tablet_service.cc:1511] Processing CreateTablet for tablet 39aad2922e1f459a8fbb5dc6e3ac7d7b (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:34.582608  9208 tablet_service.cc:1511] Processing CreateTablet for tablet bcf8624d5330403dbdebaeaabf396c07 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:34.583614  8983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39aad2922e1f459a8fbb5dc6e3ac7d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.584059  9208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf8624d5330403dbdebaeaabf396c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.584193  8982 tablet_service.cc:1511] Processing CreateTablet for tablet 8f97ca50b64e4e5eb86f9392e8a6d6cd (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:34.585583  8982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f97ca50b64e4e5eb86f9392e8a6d6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.589236  9269 tablet_bootstrap.cc:654] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.576915  9286 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:34.593219  9059 tablet_service.cc:1511] Processing CreateTablet for tablet b81bd1db68c34b918042fd1f8a0d980c (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:34.595549  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2426191b92b948e4b607c976696af51f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:34.595902  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2426191b92b948e4b607c976696af51f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:34.596288  9000 raft_consensus.cc:2468] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:34.596377  9061 tablet_service.cc:1511] Processing CreateTablet for tablet 44e11b03c5914827946641c953501ec8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:34.597746  9039 leader_election.cc:304] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:34.601022  9276 raft_consensus.cc:2804] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:34.601166  9039 leader_election.cc:343] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.601446  9276 raft_consensus.cc:493] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.601907  9276 raft_consensus.cc:3060] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.562155  9272 consensus_queue.cc:260] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.604243  9272 ts_tablet_manager.cc:1434] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.196s	user 0.001s	sys 0.009s
I20260709 12:21:34.605070  9272 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:34.610822  9132 tablet_service.cc:1511] Processing CreateTablet for tablet 39aad2922e1f459a8fbb5dc6e3ac7d7b (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:34.582026  9268 raft_consensus.cc:740] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.612876  9268 consensus_queue.cc:260] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.631889  8989 tablet_service.cc:1511] Processing CreateTablet for tablet f78c5f0841e2455aa811815132278a1f (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:34.632787  9207 tablet_service.cc:1511] Processing CreateTablet for tablet 39aad2922e1f459a8fbb5dc6e3ac7d7b (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:34.635320  9270 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:34.635118  9206 tablet_service.cc:1511] Processing CreateTablet for tablet b81bd1db68c34b918042fd1f8a0d980c (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:34.636952  9210 tablet_service.cc:1511] Processing CreateTablet for tablet 76d74a33940c4299aa703e8d872d9375 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:34.644359  9058 tablet_service.cc:1511] Processing CreateTablet for tablet 8f97ca50b64e4e5eb86f9392e8a6d6cd (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:34.648003  9057 tablet_service.cc:1511] Processing CreateTablet for tablet f78c5f0841e2455aa811815132278a1f (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:34.635888  9270 ts_tablet_manager.cc:1403] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.309s	user 0.013s	sys 0.007s
I20260709 12:21:34.653031  9270 raft_consensus.cc:359] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:34.653759  9270 raft_consensus.cc:385] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.659614  9207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39aad2922e1f459a8fbb5dc6e3ac7d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.661300  8989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78c5f0841e2455aa811815132278a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.627127  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7ab6e4c736485fbc461ccb19ea580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.662825  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d74a33940c4299aa703e8d872d9375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.667043  9062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d74a33940c4299aa703e8d872d9375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.670770  9206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81bd1db68c34b918042fd1f8a0d980c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.654047  9270 raft_consensus.cc:740] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.676229  9059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81bd1db68c34b918042fd1f8a0d980c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.679790  9058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f97ca50b64e4e5eb86f9392e8a6d6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.679563  9209 tablet_service.cc:1511] Processing CreateTablet for tablet f78c5f0841e2455aa811815132278a1f (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:34.686784  9057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78c5f0841e2455aa811815132278a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.689864  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78c5f0841e2455aa811815132278a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.709059  9268 ts_tablet_manager.cc:1434] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.201s	user 0.044s	sys 0.048s
I20260709 12:21:34.711208  9268 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:34.702557  9134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf8624d5330403dbdebaeaabf396c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.729717  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 2e2c3b1da14244ec82ac5d25e49e0862 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:34.731128  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e2c3b1da14244ec82ac5d25e49e0862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.736254  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:34.716780  9061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e11b03c5914827946641c953501ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.738714  9270 consensus_queue.cc:260] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:34.745849  9269 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:34.754420  9269 ts_tablet_manager.cc:1403] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.245s	user 0.013s	sys 0.009s
I20260709 12:21:34.757687  9269 raft_consensus.cc:359] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:34.759217  9270 ts_tablet_manager.cc:1434] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.109s	user 0.007s	sys 0.000s
I20260709 12:21:34.702062  9276 raft_consensus.cc:515] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:34.771483  8987 tablet_service.cc:1511] Processing CreateTablet for tablet 2e2c3b1da14244ec82ac5d25e49e0862 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 12:21:34.773629  8963 leader_election.cc:343] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.774513  9132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39aad2922e1f459a8fbb5dc6e3ac7d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.800364  9294 raft_consensus.cc:493] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.800809  9270 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:34.800952  9294 raft_consensus.cc:515] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.776787  9269 raft_consensus.cc:385] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:34.806617  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e2c3b1da14244ec82ac5d25e49e0862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.814455  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2426191b92b948e4b607c976696af51f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:34.823611  9000 raft_consensus.cc:3060] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:34.824321  9269 raft_consensus.cc:740] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:34.810149  9133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7ab6e4c736485fbc461ccb19ea580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.826115  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25135f5b10d44185b9d6be0f2fb5a02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.826740  9269 consensus_queue.cc:260] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:34.860373  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2426191b92b948e4b607c976696af51f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:34.860782  9279 raft_consensus.cc:493] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.861322  9279 raft_consensus.cc:515] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:34.862630  9205 tablet_service.cc:1511] Processing CreateTablet for tablet 86996e32eee445b381d0391def34e7b1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:34.857556  9276 leader_election.cc:290] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:34.867933  9279 leader_election.cc:290] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:34.868700  9279 raft_consensus.cc:493] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.869299  9279 raft_consensus.cc:515] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:34.872974  9039 leader_election.cc:343] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.865159  9205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86996e32eee445b381d0391def34e7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.891713  8982 tablet_service.cc:1511] Processing CreateTablet for tablet 185f6cd295f64fcfa3bf18036f9f75aa (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:34.892464  9060 tablet_service.cc:1511] Processing CreateTablet for tablet 185f6cd295f64fcfa3bf18036f9f75aa (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:34.893146  8982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185f6cd295f64fcfa3bf18036f9f75aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.893814  9060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185f6cd295f64fcfa3bf18036f9f75aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.895515  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:34.897230  9039 leader_election.cc:343] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:34.902897  9269 ts_tablet_manager.cc:1434] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.148s	user 0.008s	sys 0.000s
I20260709 12:21:34.904042  9269 tablet_bootstrap.cc:492] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:34.909327  9294 leader_election.cc:290] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:34.912356  9138 tablet_service.cc:1511] Processing CreateTablet for tablet 8f97ca50b64e4e5eb86f9392e8a6d6cd (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:34.912561  9137 tablet_service.cc:1511] Processing CreateTablet for tablet b81bd1db68c34b918042fd1f8a0d980c (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:34.912561  9135 tablet_service.cc:1511] Processing CreateTablet for tablet 2e2c3b1da14244ec82ac5d25e49e0862 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:34.913818  9138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f97ca50b64e4e5eb86f9392e8a6d6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.914661  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e2c3b1da14244ec82ac5d25e49e0862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.915606  9131 tablet_service.cc:1511] Processing CreateTablet for tablet 86996e32eee445b381d0391def34e7b1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:34.916932  9131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86996e32eee445b381d0391def34e7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.918653  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81bd1db68c34b918042fd1f8a0d980c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.922756  9130 tablet_service.cc:1511] Processing CreateTablet for tablet 185f6cd295f64fcfa3bf18036f9f75aa (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:34.926371  9279 leader_election.cc:290] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:34.934413  9130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 185f6cd295f64fcfa3bf18036f9f75aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.936439  9204 tablet_service.cc:1511] Processing CreateTablet for tablet b6ee9ad5a10c4a739bf573cebe2cb609 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:34.937943  9204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ee9ad5a10c4a739bf573cebe2cb609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.902271  9272 tablet_bootstrap.cc:654] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.902268  9268 tablet_bootstrap.cc:654] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:34.871599  9059 tablet_service.cc:1511] Processing CreateTablet for tablet 86996e32eee445b381d0391def34e7b1 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:34.967303  9063 tablet_service.cc:1511] Processing CreateTablet for tablet b6ee9ad5a10c4a739bf573cebe2cb609 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:34.967300  8986 tablet_service.cc:1511] Processing CreateTablet for tablet b6ee9ad5a10c4a739bf573cebe2cb609 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:34.975144  8986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ee9ad5a10c4a739bf573cebe2cb609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.986720  9212 tablet_service.cc:1511] Processing CreateTablet for tablet 9418597cbc1943eda834dfaa445555a8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:34.988112  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9418597cbc1943eda834dfaa445555a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.991771  9000 raft_consensus.cc:2468] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:34.993011  9039 leader_election.cc:304] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:34.993532  9059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86996e32eee445b381d0391def34e7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:34.994277  9294 raft_consensus.cc:2804] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:34.994828  9294 raft_consensus.cc:697] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:34.995682  9294 consensus_queue.cc:237] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:35.000036  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:35.000303  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:35.000787  9074 raft_consensus.cc:2468] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:35.000907  9075 raft_consensus.cc:2468] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:34.994182  9270 tablet_bootstrap.cc:654] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.024076  8990 tablet_service.cc:1511] Processing CreateTablet for tablet 9418597cbc1943eda834dfaa445555a8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:35.024757  9112 leader_election.cc:304] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:35.025627  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9418597cbc1943eda834dfaa445555a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.025687  9279 raft_consensus.cc:2804] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.026023  9112 leader_election.cc:304] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:35.026432  9279 raft_consensus.cc:493] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.027001  9279 raft_consensus.cc:3060] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.027133  9297 raft_consensus.cc:2804] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.027684  9297 raft_consensus.cc:493] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.028072  9297 raft_consensus.cc:3060] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.024034  9274 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.029263  9274 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.029379  9064 tablet_service.cc:1511] Processing CreateTablet for tablet 30a7af654bae45a4a4bae112448a8932 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:35.031908  9274 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:35.032845  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a7af654bae45a4a4bae112448a8932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.045784  9297 raft_consensus.cc:515] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.048166  9297 leader_election.cc:290] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:35.051239  9207 tablet_service.cc:1511] Processing CreateTablet for tablet 30a7af654bae45a4a4bae112448a8932 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:35.052687  9207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a7af654bae45a4a4bae112448a8932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.055680  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:35.057232  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:35.057984  9269 tablet_bootstrap.cc:654] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:35.059144  9113 leader_election.cc:343] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.059229  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:35.059962  9074 raft_consensus.cc:3060] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.060098  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ee9ad5a10c4a739bf573cebe2cb609. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:35.060321  9113 leader_election.cc:343] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.073208  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.074573  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.076146  9147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.077561  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
W20260709 12:21:35.083628  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:35.090590  9279 raft_consensus.cc:515] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.110074  8984 tablet_service.cc:1511] Processing CreateTablet for tablet bae55b8962494f7582e7d5304cb39d13 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:35.110572  9058 tablet_service.cc:1511] Processing CreateTablet for tablet bae55b8962494f7582e7d5304cb39d13 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:35.111608  8984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae55b8962494f7582e7d5304cb39d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.111910  9058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae55b8962494f7582e7d5304cb39d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.114264  9149 raft_consensus.cc:2393] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:35.115288  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:35.115952  9075 raft_consensus.cc:3060] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.111305  9279 leader_election.cc:290] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:35.119635  9268 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:35.120216  9268 ts_tablet_manager.cc:1403] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.409s	user 0.017s	sys 0.022s
I20260709 12:21:35.123500  9075 raft_consensus.cc:2468] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:35.123581  9139 tablet_service.cc:1511] Processing CreateTablet for tablet 9418597cbc1943eda834dfaa445555a8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:35.131146  9136 tablet_service.cc:1511] Processing CreateTablet for tablet 30a7af654bae45a4a4bae112448a8932 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:35.133026  9268 raft_consensus.cc:359] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.140552  9268 raft_consensus.cc:385] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.144676  9268 raft_consensus.cc:740] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
W20260709 12:21:35.145215  8963 leader_election.cc:343] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.145769  8963 leader_election.cc:304] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.145591  9268 consensus_queue.cc:260] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.146616  9274 raft_consensus.cc:2749] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.137784  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a7af654bae45a4a4bae112448a8932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.148427  9268 ts_tablet_manager.cc:1434] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:21:35.151039  9268 tablet_bootstrap.cc:492] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:35.153012  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:35.154527  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
W20260709 12:21:35.155789  9113 leader_election.cc:343] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.165747  9139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9418597cbc1943eda834dfaa445555a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.167560  9269 tablet_bootstrap.cc:492] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:35.168115  9269 ts_tablet_manager.cc:1403] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.264s	user 0.009s	sys 0.004s
I20260709 12:21:35.170557  9112 leader_election.cc:304] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:35.171458  9279 raft_consensus.cc:2804] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.172003  9279 raft_consensus.cc:697] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:35.172894  9279 consensus_queue.cc:237] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:35.175104  9113 leader_election.cc:343] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:35.179186  8963 leader_election.cc:343] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.179253  9038 leader_election.cc:304] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.180989  9294 raft_consensus.cc:2749] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.183204  9074 raft_consensus.cc:2468] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:35.184477  9112 leader_election.cc:304] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:35.185627  9297 raft_consensus.cc:2804] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.191205  9135 tablet_service.cc:1511] Processing CreateTablet for tablet bae55b8962494f7582e7d5304cb39d13 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:35.193087  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae55b8962494f7582e7d5304cb39d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.194504  8897 catalog_manager.cc:5697] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.200947  9286 raft_consensus.cc:493] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.202183  9286 raft_consensus.cc:515] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:35.171121  9269 raft_consensus.cc:359] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.204319  9269 raft_consensus.cc:385] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.204391  9286 leader_election.cc:290] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:35.204718  9269 raft_consensus.cc:740] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.205586  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571bcffedf6f4da59792575fd301ed0d" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:35.205555  9269 consensus_queue.cc:260] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.209017  9272 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:35.209721  9272 ts_tablet_manager.cc:1403] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.605s	user 0.016s	sys 0.020s
W20260709 12:21:35.211227  9187 leader_election.cc:343] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:35.212050  9186 leader_election.cc:343] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.218153  9186 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:35.225083  8982 tablet_service.cc:1511] Processing CreateTablet for tablet 69899b55ef9441a39d4f4a192d838f17 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:35.235399  9210 tablet_service.cc:1511] Processing CreateTablet for tablet 69899b55ef9441a39d4f4a192d838f17 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:35.237349  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69899b55ef9441a39d4f4a192d838f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.238121  8982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69899b55ef9441a39d4f4a192d838f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.249804  9272 raft_consensus.cc:359] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.250663  9272 raft_consensus.cc:385] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.250975  9272 raft_consensus.cc:740] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.251950  9269 ts_tablet_manager.cc:1434] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.084s	user 0.005s	sys 0.003s
I20260709 12:21:35.251881  9272 consensus_queue.cc:260] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.253506  9269 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:35.255703  9286 raft_consensus.cc:2749] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.264460  9064 tablet_service.cc:1511] Processing CreateTablet for tablet 69899b55ef9441a39d4f4a192d838f17 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:35.265941  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69899b55ef9441a39d4f4a192d838f17. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:35.258492  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 46636.8us
I20260709 12:21:35.273151  9269 tablet_bootstrap.cc:654] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.284525  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:35.285490  9222 raft_consensus.cc:2468] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:35.287024  8964 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.288230  9274 raft_consensus.cc:2804] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.288631  9274 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.288481  9059 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa24aacdc954777b693378dd54a79de (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:35.289032  9274 raft_consensus.cc:3060] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.288690  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571bcffedf6f4da59792575fd301ed0d" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.289868  9059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa24aacdc954777b693378dd54a79de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.289933  8990 tablet_service.cc:1511] Processing CreateTablet for tablet ce01c7da004041a58a5a746d2f258ee9 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:35.290494  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa24aacdc954777b693378dd54a79de (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:35.291419  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce01c7da004041a58a5a746d2f258ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.291802  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa24aacdc954777b693378dd54a79de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:35.294137  9186 leader_election.cc:343] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.294178  9136 tablet_service.cc:1511] Processing CreateTablet for tablet ce01c7da004041a58a5a746d2f258ee9 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:35.296175  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce01c7da004041a58a5a746d2f258ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.297219  9212 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa24aacdc954777b693378dd54a79de (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:35.298506  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa24aacdc954777b693378dd54a79de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.294656  9186 leader_election.cc:304] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:35.299736  9286 raft_consensus.cc:2749] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.302664  9268 tablet_bootstrap.cc:654] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.305330  8987 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4462b201e045d69b521bfedf999c94 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:35.256201  9297 raft_consensus.cc:697] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:35.305714  9274 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.311014  9270 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:35.311702  9270 ts_tablet_manager.cc:1403] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.511s	user 0.017s	sys 0.008s
I20260709 12:21:35.317727  9270 raft_consensus.cc:359] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.319859  9270 raft_consensus.cc:385] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.320220  9270 raft_consensus.cc:740] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.320806  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4462b201e045d69b521bfedf999c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.329681  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:35.329906  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:35.330600  9222 raft_consensus.cc:3060] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:35.331295  8963 leader_election.cc:343] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.335070  9207 tablet_service.cc:1511] Processing CreateTablet for tablet ce01c7da004041a58a5a746d2f258ee9 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:35.334800  9297 consensus_queue.cc:237] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.336552  9207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce01c7da004041a58a5a746d2f258ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.340308  9222 raft_consensus.cc:2468] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:35.343034  8964 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.347177  9272 ts_tablet_manager.cc:1434] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.137s	user 0.012s	sys 0.006s
I20260709 12:21:35.348428  9272 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:35.348912  8895 catalog_manager.cc:5697] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:35.349582  9209 tablet_service.cc:1511] Processing CreateTablet for tablet 94657c9a3fe64516bd052cd08f818cfa (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:35.350934  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94657c9a3fe64516bd052cd08f818cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.353291  9135 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4462b201e045d69b521bfedf999c94 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:35.322410  9270 consensus_queue.cc:260] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.344727  9274 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:35.360921  9286 raft_consensus.cc:493] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.361629  9286 raft_consensus.cc:515] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.365645  9272 tablet_bootstrap.cc:654] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.359622  9274 raft_consensus.cc:2804] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.377471  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4462b201e045d69b521bfedf999c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.377720  9058 tablet_service.cc:1511] Processing CreateTablet for tablet 94657c9a3fe64516bd052cd08f818cfa (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:35.380481  9058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94657c9a3fe64516bd052cd08f818cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.383126  9132 tablet_service.cc:1511] Processing CreateTablet for tablet 98c726df727346b694e0509fae44d789 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:35.384523  9132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c726df727346b694e0509fae44d789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.384599  9268 tablet_bootstrap.cc:492] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:35.385144  9268 ts_tablet_manager.cc:1403] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.234s	user 0.018s	sys 0.025s
I20260709 12:21:35.378047  9063 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4462b201e045d69b521bfedf999c94 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:35.394619  9274 raft_consensus.cc:697] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:35.414122  9274 consensus_queue.cc:237] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.416443  9210 tablet_service.cc:1511] Processing CreateTablet for tablet 98c726df727346b694e0509fae44d789 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:35.427871  9210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c726df727346b694e0509fae44d789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.419449  9286 leader_election.cc:290] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:35.436425  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.436702  9130 tablet_service.cc:1511] Processing CreateTablet for tablet d3459d33314340e1b9fa3217c09920f5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:35.438005  9130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3459d33314340e1b9fa3217c09920f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.439795  9212 tablet_service.cc:1511] Processing CreateTablet for tablet d3459d33314340e1b9fa3217c09920f5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:35.441890  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3459d33314340e1b9fa3217c09920f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.442687  9064 tablet_service.cc:1511] Processing CreateTablet for tablet d3459d33314340e1b9fa3217c09920f5 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:35.444103  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4462b201e045d69b521bfedf999c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.448483  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3459d33314340e1b9fa3217c09920f5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:35.423223  8964 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 80295.5us
I20260709 12:21:35.464207  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45855a20f5a4404cb8ba32f93c294288" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:35.465410  9074 raft_consensus.cc:2393] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:35.465066  9268 raft_consensus.cc:359] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.467386  9186 leader_election.cc:304] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.468940  9286 raft_consensus.cc:3060] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.476686  9316 raft_consensus.cc:493] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.477608  9316 raft_consensus.cc:515] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.478791  9294 raft_consensus.cc:493] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.479683  9294 raft_consensus.cc:515] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.483690  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:35.484588  8999 raft_consensus.cc:2468] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:35.487499  9039 leader_election.cc:304] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:35.487613  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.488299  9149 raft_consensus.cc:2468] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:35.488442  9276 raft_consensus.cc:2804] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:35.488824  9276 raft_consensus.cc:493] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.489181  9276 raft_consensus.cc:3060] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.466117  9268 raft_consensus.cc:385] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.492533  9294 leader_election.cc:290] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:35.496335  9268 raft_consensus.cc:740] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.509464  8990 tablet_service.cc:1511] Processing CreateTablet for tablet 94657c9a3fe64516bd052cd08f818cfa (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:35.510221  9135 tablet_service.cc:1511] Processing CreateTablet for tablet 293b73b5cbf14c728935731f909f32b4 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:35.511654  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293b73b5cbf14c728935731f909f32b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.511835  9061 tablet_service.cc:1511] Processing CreateTablet for tablet ba2d24a7495d4a42b94f6f25809db7f8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:35.511950  9060 tablet_service.cc:1511] Processing CreateTablet for tablet 293b73b5cbf14c728935731f909f32b4 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:35.513206  9061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2d24a7495d4a42b94f6f25809db7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.514451  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94657c9a3fe64516bd052cd08f818cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.510407  9207 tablet_service.cc:1511] Processing CreateTablet for tablet ba2d24a7495d4a42b94f6f25809db7f8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:35.519285  9207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2d24a7495d4a42b94f6f25809db7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.519802  9060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293b73b5cbf14c728935731f909f32b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.521806  9276 raft_consensus.cc:515] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:35.525074  9039 consensus_peers.cc:597] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 -> Peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Couldn't send request to peer 343eaafd515a4dfc9de3df299924f0d4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:35.526350  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:35.526988  8999 raft_consensus.cc:3060] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.538118  8896 catalog_manager.cc:5697] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:35.538794  9268 consensus_queue.cc:260] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.539621  8999 raft_consensus.cc:2468] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:35.540938  9039 leader_election.cc:304] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:35.541742  9268 ts_tablet_manager.cc:1434] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.156s	user 0.009s	sys 0.004s
I20260709 12:21:35.541821  9294 raft_consensus.cc:2804] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.543100  9268 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:35.551769  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:35.552662  9074 raft_consensus.cc:2393] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:35.553304  9207 tablet_service.cc:1511] Processing CreateTablet for tablet f3f4d49acc064d6ea6cf83616117be16 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:35.554693  9269 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:35.555086  9207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f4d49acc064d6ea6cf83616117be16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.555267  9269 ts_tablet_manager.cc:1403] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.302s	user 0.003s	sys 0.026s
I20260709 12:21:35.556423  9136 tablet_service.cc:1511] Processing CreateTablet for tablet f3f4d49acc064d6ea6cf83616117be16 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:35.557821  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f4d49acc064d6ea6cf83616117be16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.559224  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.561038  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:35.562417  9269 raft_consensus.cc:359] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:35.563191  9269 raft_consensus.cc:385] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.564090  8982 tablet_service.cc:1511] Processing CreateTablet for tablet 98c726df727346b694e0509fae44d789 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:35.566107  8982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98c726df727346b694e0509fae44d789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.561832  9146 raft_consensus.cc:3060] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.569128  8963 leader_election.cc:304] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.570156  9314 raft_consensus.cc:3060] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.566638  8986 tablet_service.cc:1511] Processing CreateTablet for tablet 293b73b5cbf14c728935731f909f32b4 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:35.571918  8986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293b73b5cbf14c728935731f909f32b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.546635  9276 leader_election.cc:290] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:35.563496  9269 raft_consensus.cc:740] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.555174  9316 leader_election.cc:290] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:35.596472  9314 raft_consensus.cc:2749] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.606776  9268 tablet_bootstrap.cc:654] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.576448  8984 tablet_service.cc:1511] Processing CreateTablet for tablet ba2d24a7495d4a42b94f6f25809db7f8 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:35.612735  8984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2d24a7495d4a42b94f6f25809db7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.575448  9272 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:35.609390  9132 tablet_service.cc:1511] Processing CreateTablet for tablet 09b89dc1b3074a578d898694dd05a74a (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:35.613979  9132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09b89dc1b3074a578d898694dd05a74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.585316  9269 consensus_queue.cc:260] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:35.615285  8982 tablet_service.cc:1511] Processing CreateTablet for tablet f3f4d49acc064d6ea6cf83616117be16 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:35.618144  8982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f4d49acc064d6ea6cf83616117be16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.610814  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 09b89dc1b3074a578d898694dd05a74a (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:35.621469  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09b89dc1b3074a578d898694dd05a74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.628207  9297 consensus_queue.cc:1048] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:35.628907  9276 raft_consensus.cc:493] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.629498  9276 raft_consensus.cc:515] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.632809  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2290f27d484a4682a825fb8d1bb87924" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:35.584215  9270 ts_tablet_manager.cc:1434] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.272s	user 0.023s	sys 0.016s
W20260709 12:21:35.634359  9038 leader_election.cc:343] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.635627  9270 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:35.636435  9276 leader_election.cc:290] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:35.636654  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2290f27d484a4682a825fb8d1bb87924" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:35.638206  9039 leader_election.cc:343] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:35.638721  9039 leader_election.cc:304] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:35.639809  9276 raft_consensus.cc:2749] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.640654  9276 consensus_queue.cc:1048] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.646741  9272 ts_tablet_manager.cc:1403] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.299s	user 0.025s	sys 0.048s
I20260709 12:21:35.650012  8990 tablet_service.cc:1511] Processing CreateTablet for tablet 95536fefe39b43939fac53b1ab01dab0 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:35.651520  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95536fefe39b43939fac53b1ab01dab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.652122  9060 tablet_service.cc:1511] Processing CreateTablet for tablet 95536fefe39b43939fac53b1ab01dab0 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:35.653494  9060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95536fefe39b43939fac53b1ab01dab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.655627  9212 tablet_service.cc:1511] Processing CreateTablet for tablet 7931c26b00244fa1b699e0ce90f2b291 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:35.656384  9135 tablet_service.cc:1511] Processing CreateTablet for tablet 95536fefe39b43939fac53b1ab01dab0 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:35.612453  9059 tablet_service.cc:1511] Processing CreateTablet for tablet 09b89dc1b3074a578d898694dd05a74a (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:35.656996  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7931c26b00244fa1b699e0ce90f2b291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.658283  9059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09b89dc1b3074a578d898694dd05a74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.658236  9135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95536fefe39b43939fac53b1ab01dab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.506995  9286 raft_consensus.cc:2749] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:35.666692  9294 raft_consensus.cc:697] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:35.673655  9146 raft_consensus.cc:2468] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:35.704411  9064 tablet_service.cc:1511] Processing CreateTablet for tablet 7931c26b00244fa1b699e0ce90f2b291 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:35.705771  9064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7931c26b00244fa1b699e0ce90f2b291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.713966  8989 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd869b989df40608f17c7091c100fc7 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:35.715380  8989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd869b989df40608f17c7091c100fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.715448  9209 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd869b989df40608f17c7091c100fc7 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:35.716861  9209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd869b989df40608f17c7091c100fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.720079  9297 consensus_queue.cc:1048] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.724892  9270 tablet_bootstrap.cc:654] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.727835  9063 tablet_service.cc:1511] Processing CreateTablet for tablet 13219c476384456b808b69575b85edcd (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:35.729503  9063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13219c476384456b808b69575b85edcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.730238  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 13219c476384456b808b69575b85edcd (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:35.731297  9130 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd869b989df40608f17c7091c100fc7 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:35.731673  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13219c476384456b808b69575b85edcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.729573  9294 consensus_queue.cc:237] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:35.725911  8987 tablet_service.cc:1511] Processing CreateTablet for tablet 7931c26b00244fa1b699e0ce90f2b291 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:35.760115  8987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7931c26b00244fa1b699e0ce90f2b291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.766794  9130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd869b989df40608f17c7091c100fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.772082  9204 tablet_service.cc:1511] Processing CreateTablet for tablet 13219c476384456b808b69575b85edcd (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:35.772634  8984 tablet_service.cc:1511] Processing CreateTablet for tablet 013d7d1179ed4bffba8be127fe4a2d03 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:35.773494  9204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13219c476384456b808b69575b85edcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.773995  8984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013d7d1179ed4bffba8be127fe4a2d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.707723  9272 raft_consensus.cc:359] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.803033  9272 raft_consensus.cc:385] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.814519  9269 ts_tablet_manager.cc:1434] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.259s	user 0.009s	sys 0.017s
I20260709 12:21:35.817777  9269 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:35.809023  9272 raft_consensus.cc:740] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.819010  9272 consensus_queue.cc:260] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:35.822736  9137 tablet_service.cc:1511] Processing CreateTablet for tablet 013d7d1179ed4bffba8be127fe4a2d03 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:35.823679  9272 ts_tablet_manager.cc:1434] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.176s	user 0.004s	sys 0.006s
I20260709 12:21:35.824812  9137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013d7d1179ed4bffba8be127fe4a2d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.825776  9272 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:35.835579  9060 tablet_service.cc:1511] Processing CreateTablet for tablet 013d7d1179ed4bffba8be127fe4a2d03 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:35.849261  9060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013d7d1179ed4bffba8be127fe4a2d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.845562  9059 tablet_service.cc:1511] Processing CreateTablet for tablet 46df15ab8dec424780c28700160bbdb2 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:35.851056  9059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46df15ab8dec424780c28700160bbdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.851281  9268 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:35.851855  9268 ts_tablet_manager.cc:1403] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.309s	user 0.009s	sys 0.021s
I20260709 12:21:35.853020  9136 tablet_service.cc:1511] Processing CreateTablet for tablet 39459159540e4827b6f57bb550d02a30 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:35.835140  9213 tablet_service.cc:1511] Processing CreateTablet for tablet 46df15ab8dec424780c28700160bbdb2 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:35.840059  8986 tablet_service.cc:1511] Processing CreateTablet for tablet 46df15ab8dec424780c28700160bbdb2 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:35.856856  9212 tablet_service.cc:1511] Processing CreateTablet for tablet 39459159540e4827b6f57bb550d02a30 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:35.830273  9269 tablet_bootstrap.cc:654] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.864171  8990 tablet_service.cc:1511] Processing CreateTablet for tablet 39459159540e4827b6f57bb550d02a30 (DEFAULT_TABLE table=test-workload [id=a7d6e11453e14740ab1685ead8721e38]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:35.867864  9136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39459159540e4827b6f57bb550d02a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.868690  9146 raft_consensus.cc:3060] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.874836  9297 consensus_queue.cc:1048] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.861752  8986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46df15ab8dec424780c28700160bbdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.891613  9212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39459159540e4827b6f57bb550d02a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.900904  9213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46df15ab8dec424780c28700160bbdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.902863  8990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39459159540e4827b6f57bb550d02a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.923169  9297 consensus_queue.cc:1048] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.926813  9268 raft_consensus.cc:359] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:35.927636  9268 raft_consensus.cc:385] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.927930  9268 raft_consensus.cc:740] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.928781  9268 consensus_queue.cc:260] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:35.930779  9314 consensus_queue.cc:1048] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:35.933687  9268 ts_tablet_manager.cc:1434] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 12:21:35.947994  9270 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:35.948557  9270 ts_tablet_manager.cc:1403] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.313s	user 0.020s	sys 0.008s
I20260709 12:21:35.960196  8895 catalog_manager.cc:5697] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:35.942128  9268 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:35.978545  9272 tablet_bootstrap.cc:654] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.975636  9286 raft_consensus.cc:493] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.005111  9269 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:36.005614  9270 raft_consensus.cc:359] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.007043  9270 raft_consensus.cc:385] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.007329  9270 raft_consensus.cc:740] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.008083  9270 consensus_queue.cc:260] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:35.994477  9286 raft_consensus.cc:515] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.012424  9286 leader_election.cc:290] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:36.011976  8896 catalog_manager.cc:5697] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } }
I20260709 12:21:36.015020  9297 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.015607  9297 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.015779  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149942a6b05d454496753897bc2ed0b8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.016522  8999 raft_consensus.cc:2468] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:36.017925  9187 leader_election.cc:304] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.005676  9269 ts_tablet_manager.cc:1403] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.188s	user 0.015s	sys 0.003s
I20260709 12:21:36.023646  9269 raft_consensus.cc:359] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.024626  9269 raft_consensus.cc:385] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.024821  9270 ts_tablet_manager.cc:1434] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.003s
I20260709 12:21:36.026077  9270 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.062578  9268 tablet_bootstrap.cc:654] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.049947  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.047529  9314 consensus_queue.cc:1048] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:21:36.024995  9269 raft_consensus.cc:740] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.023774  9297 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:36.038534  9286 raft_consensus.cc:2804] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.030035  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:36.031860  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149942a6b05d454496753897bc2ed0b8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.083926  9074 raft_consensus.cc:2468] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:36.090576  9272 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.092614  9270 tablet_bootstrap.cc:654] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.092664  9272 ts_tablet_manager.cc:1403] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.267s	user 0.005s	sys 0.015s
I20260709 12:21:36.082876  9286 raft_consensus.cc:493] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.098189  9286 raft_consensus.cc:3060] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.101331  9272 raft_consensus.cc:359] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.102335  9272 raft_consensus.cc:385] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.102725  9272 raft_consensus.cc:740] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.103724  9272 consensus_queue.cc:260] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:36.104907  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:36.106359  9272 ts_tablet_manager.cc:1434] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:21:36.107510  9272 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.108880  9113 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:36.116050  9272 tablet_bootstrap.cc:654] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.106985  9269 consensus_queue.cc:260] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.124397  9269 ts_tablet_manager.cc:1434] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.104s	user 0.005s	sys 0.005s
I20260709 12:21:36.127008  9269 tablet_bootstrap.cc:492] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:36.125577  9270 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.130923  9270 ts_tablet_manager.cc:1403] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.027s
I20260709 12:21:36.145169  9341 raft_consensus.cc:2749] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:36.146090  9270 raft_consensus.cc:359] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.147001  9270 raft_consensus.cc:385] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.147313  9270 raft_consensus.cc:740] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.148093  9270 consensus_queue.cc:260] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.153518  9269 tablet_bootstrap.cc:654] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.154203  9270 ts_tablet_manager.cc:1434] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 12:21:36.155918  9270 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.156744  9286 raft_consensus.cc:515] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.159611  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149942a6b05d454496753897bc2ed0b8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:36.160209  8999 raft_consensus.cc:3060] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.167634  9270 tablet_bootstrap.cc:654] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.167764  8999 raft_consensus.cc:2468] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:36.169360  9187 leader_election.cc:304] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.170442  9323 raft_consensus.cc:2804] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.173138  9268 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:36.173906  9268 ts_tablet_manager.cc:1403] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.232s	user 0.008s	sys 0.008s
I20260709 12:21:36.177875  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149942a6b05d454496753897bc2ed0b8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:36.178571  9074 raft_consensus.cc:3060] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.185338  9268 raft_consensus.cc:359] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.186117  9268 raft_consensus.cc:385] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.186482  9268 raft_consensus.cc:740] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.186659  9074 raft_consensus.cc:2468] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:36.187366  9268 consensus_queue.cc:260] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.191234  9268 ts_tablet_manager.cc:1434] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 12:21:36.192642  9268 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:36.194108  9270 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.195055  9270 ts_tablet_manager.cc:1403] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.023s
I20260709 12:21:36.195749  9286 leader_election.cc:290] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:36.199051  9270 raft_consensus.cc:359] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.199491  9268 tablet_bootstrap.cc:654] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.199816  9270 raft_consensus.cc:385] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.200186  9270 raft_consensus.cc:740] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.201287  9270 consensus_queue.cc:260] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.202442  9272 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.202951  9272 ts_tablet_manager.cc:1403] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.096s	user 0.039s	sys 0.027s
I20260709 12:21:36.204598  9270 ts_tablet_manager.cc:1434] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:21:36.206442  9270 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.206163  9272 raft_consensus.cc:359] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.206971  9272 raft_consensus.cc:385] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.207296  9272 raft_consensus.cc:740] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.208062  9272 consensus_queue.cc:260] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.210359  9272 ts_tablet_manager.cc:1434] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:21:36.211814  9323 raft_consensus.cc:697] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:36.212749  9323 consensus_queue.cc:237] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.214226  9272 tablet_bootstrap.cc:492] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.215818  9270 tablet_bootstrap.cc:654] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.230935  9269 tablet_bootstrap.cc:492] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:36.231633  9269 ts_tablet_manager.cc:1403] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.105s	user 0.006s	sys 0.007s
I20260709 12:21:36.232491  9268 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:36.233199  9268 ts_tablet_manager.cc:1403] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.000s
I20260709 12:21:36.235685  9269 raft_consensus.cc:359] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.236479  9268 raft_consensus.cc:359] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.236779  9270 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.236743  9269 raft_consensus.cc:385] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.237294  9268 raft_consensus.cc:385] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.237560  9270 ts_tablet_manager.cc:1403] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.005s
I20260709 12:21:36.238301  9272 tablet_bootstrap.cc:654] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.237473  9269 raft_consensus.cc:740] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.240612  9268 raft_consensus.cc:740] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.240909  9270 raft_consensus.cc:359] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.242059  9270 raft_consensus.cc:385] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.242555  9270 raft_consensus.cc:740] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.241134  9269 consensus_queue.cc:260] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.243425  9270 consensus_queue.cc:260] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.244149  9323 raft_consensus.cc:493] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.244859  9323 raft_consensus.cc:515] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.244979  8895 catalog_manager.cc:5697] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.247591  9323 leader_election.cc:290] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.242030  9268 consensus_queue.cc:260] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.249221  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.249557  9269 ts_tablet_manager.cc:1434] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 12:21:36.250663  9269 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:36.250674  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
W20260709 12:21:36.252498  9186 leader_election.cc:343] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:36.253198  9186 leader_election.cc:343] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.253608  9186 leader_election.cc:304] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:36.254626  9270 ts_tablet_manager.cc:1434] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:21:36.254808  9323 raft_consensus.cc:2749] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.256853  9268 ts_tablet_manager.cc:1434] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 12:21:36.257761  9269 tablet_bootstrap.cc:654] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.258819  9270 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.257860  9268 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:36.265734  9270 tablet_bootstrap.cc:654] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.269937  9268 tablet_bootstrap.cc:654] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.286298  9294 consensus_queue.cc:1048] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.293955  9322 consensus_queue.cc:1048] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.298826  9272 tablet_bootstrap.cc:492] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.299803  9272 ts_tablet_manager.cc:1403] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.086s	user 0.030s	sys 0.019s
I20260709 12:21:36.303717  9269 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:36.304522  9269 ts_tablet_manager.cc:1403] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.006s
I20260709 12:21:36.307412  9272 raft_consensus.cc:359] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.308408  9272 raft_consensus.cc:385] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.308806  9270 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.308806  9272 raft_consensus.cc:740] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.309619  9270 ts_tablet_manager.cc:1403] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.023s
I20260709 12:21:36.310196  9272 consensus_queue.cc:260] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.313315  9272 ts_tablet_manager.cc:1434] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:21:36.313272  9270 raft_consensus.cc:359] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.314237  9270 raft_consensus.cc:385] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.314610  9270 raft_consensus.cc:740] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.315002  9272 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.316435  9269 raft_consensus.cc:359] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.317237  9269 raft_consensus.cc:385] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.317580  9269 raft_consensus.cc:740] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.318552  9270 consensus_queue.cc:260] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.321043  9270 ts_tablet_manager.cc:1434] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 12:21:36.321904  9322 raft_consensus.cc:493] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.321547  9269 consensus_queue.cc:260] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.322459  9322 raft_consensus.cc:515] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.323400  9272 tablet_bootstrap.cc:654] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.328444  9270 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.330147  9322 leader_election.cc:290] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.333369  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.334438  9000 raft_consensus.cc:2468] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:36.336251  9039 leader_election.cc:304] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.341982  9269 ts_tablet_manager.cc:1434] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 12:21:36.342698  9270 tablet_bootstrap.cc:654] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.346864  9268 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:36.347874  9268 ts_tablet_manager.cc:1403] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.002s
I20260709 12:21:36.343922  9269 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:36.353829  9268 raft_consensus.cc:359] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.354641  9268 raft_consensus.cc:385] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.354939  9268 raft_consensus.cc:740] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.357862  9268 consensus_queue.cc:260] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.360693  9269 tablet_bootstrap.cc:654] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.343317  9322 raft_consensus.cc:2804] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.361653  9322 raft_consensus.cc:493] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.362025  9322 raft_consensus.cc:3060] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.366912  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:36.368711  9269 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
W20260709 12:21:36.368947  9038 leader_election.cc:343] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.369469  9269 ts_tablet_manager.cc:1403] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.000s
I20260709 12:21:36.373627  9269 raft_consensus.cc:359] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.374650  9269 raft_consensus.cc:385] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.375083  9269 raft_consensus.cc:740] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.376027  9269 consensus_queue.cc:260] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.378603  9323 raft_consensus.cc:493] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.379354  9323 raft_consensus.cc:515] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.380832  9268 ts_tablet_manager.cc:1434] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:21:36.381863  9268 tablet_bootstrap.cc:492] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:36.383833  9270 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.384409  9270 ts_tablet_manager.cc:1403] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.003s
I20260709 12:21:36.387322  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2426191b92b948e4b607c976696af51f" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.387781  9323 leader_election.cc:290] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:36.390508  9322 raft_consensus.cc:515] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.390245  9270 raft_consensus.cc:359] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.391851  9270 raft_consensus.cc:385] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.392290  9270 raft_consensus.cc:740] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.393055  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2426191b92b948e4b607c976696af51f" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.394752  9186 leader_election.cc:304] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:36.395336  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
W20260709 12:21:36.396692  9038 leader_election.cc:343] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.396664  9323 raft_consensus.cc:2749] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.399061  9270 consensus_queue.cc:260] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.402247  9269 ts_tablet_manager.cc:1434] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 12:21:36.408643  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:36.409544  8999 raft_consensus.cc:3060] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.410414  9269 tablet_bootstrap.cc:492] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:36.417795  9322 leader_election.cc:290] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.420496  9367 raft_consensus.cc:493] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.421000  9367 raft_consensus.cc:515] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.421633  9269 tablet_bootstrap.cc:654] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.424165  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:36.425851  9223 raft_consensus.cc:2468] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:36.429122  9272 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.429724  9272 ts_tablet_manager.cc:1403] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.115s	user 0.051s	sys 0.048s
I20260709 12:21:36.430034  8999 raft_consensus.cc:2468] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:36.425052  9270 ts_tablet_manager.cc:1434] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.002s
I20260709 12:21:36.431600  9270 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.432920  9272 raft_consensus.cc:359] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.433652  9272 raft_consensus.cc:385] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.434024  9272 raft_consensus.cc:740] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.434906  9272 consensus_queue.cc:260] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.437188  9272 ts_tablet_manager.cc:1434] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:21:36.437834  9270 tablet_bootstrap.cc:654] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.438162  9272 tablet_bootstrap.cc:492] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.433694  9039 leader_election.cc:304] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:36.439668  9039 leader_election.cc:304] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:36.440448  9322 raft_consensus.cc:2804] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.440930  9322 raft_consensus.cc:697] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:36.441823  9364 raft_consensus.cc:2804] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.442648  9322 consensus_queue.cc:237] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.442229  9364 raft_consensus.cc:493] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.444506  9364 raft_consensus.cc:3060] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.450682  9367 leader_election.cc:290] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.451505  9272 tablet_bootstrap.cc:654] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.454988  9270 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.437887  9268 tablet_bootstrap.cc:654] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.455586  9316 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.456161  9316 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.458371  9316 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.459604  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.463521  9000 raft_consensus.cc:2468] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:36.463794  9272 tablet_bootstrap.cc:492] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.459766  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:36.464488  9272 ts_tablet_manager.cc:1403] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.011s
I20260709 12:21:36.464967  9146 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:36.459017  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.466173  8963 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.467283  9316 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.467665  9316 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.468005  9316 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.468135  9272 raft_consensus.cc:359] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.469095  9272 raft_consensus.cc:385] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.469449  9272 raft_consensus.cc:740] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.470439  9272 consensus_queue.cc:260] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.472591  9268 tablet_bootstrap.cc:492] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:36.472883  9272 ts_tablet_manager.cc:1434] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:21:36.473086  9268 ts_tablet_manager.cc:1403] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.003s
I20260709 12:21:36.474169  9272 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
W20260709 12:21:36.475705  8963 leader_election.cc:343] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.475874  9268 raft_consensus.cc:359] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.476634  9268 raft_consensus.cc:385] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.476914  9268 raft_consensus.cc:740] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.477617  9268 consensus_queue.cc:260] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.480028  9268 ts_tablet_manager.cc:1434] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:36.462706  9269 tablet_bootstrap.cc:492] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:36.480932  9272 tablet_bootstrap.cc:654] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.481145  9269 ts_tablet_manager.cc:1403] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.007s
I20260709 12:21:36.483970  9269 raft_consensus.cc:359] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.484817  9269 raft_consensus.cc:385] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.485170  9269 raft_consensus.cc:740] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.485999  9316 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.486254  9269 consensus_queue.cc:260] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.489128  9269 ts_tablet_manager.cc:1434] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:21:36.455608  9270 ts_tablet_manager.cc:1403] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 12:21:36.490156  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:36.491998  9269 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:36.492594  9223 raft_consensus.cc:3060] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.492647  9341 raft_consensus.cc:493] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.493703  9268 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:36.493328  9341 raft_consensus.cc:515] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.495435  9341 leader_election.cc:290] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.497293  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c992391e9d4940a3b550c8c511b354d1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:36.499284  8963 leader_election.cc:343] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.500181  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c992391e9d4940a3b550c8c511b354d1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.498077  9221 raft_consensus.cc:2468] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:36.501492  9269 tablet_bootstrap.cc:654] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.502233  9113 leader_election.cc:304] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:36.503492  9341 raft_consensus.cc:2804] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.503968  9341 raft_consensus.cc:493] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.504372  9341 raft_consensus.cc:3060] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.505844  9272 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.506409  9272 ts_tablet_manager.cc:1403] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.009s
I20260709 12:21:36.509203  9272 raft_consensus.cc:359] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.509867  9272 raft_consensus.cc:385] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.510180  9272 raft_consensus.cc:740] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.510023  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:36.510771  9146 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.511059  9297 raft_consensus.cc:493] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.511569  9297 raft_consensus.cc:515] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.513646  9297 leader_election.cc:290] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.514791  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571bcffedf6f4da59792575fd301ed0d" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:36.515401  9316 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.505498  9364 raft_consensus.cc:515] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.514859  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571bcffedf6f4da59792575fd301ed0d" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:36.515628  9221 raft_consensus.cc:2468] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:36.516255  8999 raft_consensus.cc:2468] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:36.517269  9113 leader_election.cc:304] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:36.517489  9146 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:36.519326  9297 raft_consensus.cc:2804] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.519712  9297 raft_consensus.cc:493] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.520033  9297 raft_consensus.cc:3060] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.520778  9272 consensus_queue.cc:260] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.523073  9272 ts_tablet_manager.cc:1434] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:21:36.522915  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:36.523545  8963 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:36.523967  9221 raft_consensus.cc:3060] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.524502  9272 tablet_bootstrap.cc:492] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.524792  9316 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.524413  8898 catalog_manager.cc:5697] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.526003  9316 raft_consensus.cc:697] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:36.526911  9316 consensus_queue.cc:237] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.529816  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:36.530458  8999 raft_consensus.cc:3060] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.535050  9272 tablet_bootstrap.cc:654] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.537822  8999 raft_consensus.cc:2468] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:36.539067  9039 leader_election.cc:304] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.492920  9270 raft_consensus.cc:359] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.545034  9270 raft_consensus.cc:385] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.546470  9270 raft_consensus.cc:740] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.544126  9364 leader_election.cc:290] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.543954  9341 raft_consensus.cc:515] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.553686  9341 leader_election.cc:290] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.555622  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c992391e9d4940a3b550c8c511b354d1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:36.556434  9222 raft_consensus.cc:3060] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.544247  9322 raft_consensus.cc:2804] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.557469  9322 raft_consensus.cc:697] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:36.550839  9297 raft_consensus.cc:515] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.558750  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c992391e9d4940a3b550c8c511b354d1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:36.561715  9297 leader_election.cc:290] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.562933  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571bcffedf6f4da59792575fd301ed0d" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:36.560614  9322 consensus_queue.cc:237] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.563767  9220 raft_consensus.cc:3060] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.571333  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571bcffedf6f4da59792575fd301ed0d" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:36.572167  8999 raft_consensus.cc:3060] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.588481  8895 catalog_manager.cc:5697] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.588752  8896 catalog_manager.cc:5697] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.609627  8999 raft_consensus.cc:2468] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:36.552662  9270 consensus_queue.cc:260] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.617702  9000 raft_consensus.cc:2468] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:36.618659  9269 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:36.618651  9113 leader_election.cc:304] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.619357  9269 ts_tablet_manager.cc:1403] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.128s	user 0.035s	sys 0.020s
I20260709 12:21:36.619730  9341 raft_consensus.cc:2804] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.620271  9341 raft_consensus.cc:697] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:36.542424  9221 raft_consensus.cc:2468] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:36.621309  9272 tablet_bootstrap.cc:492] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.621114  9341 consensus_queue.cc:237] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.622687  9272 ts_tablet_manager.cc:1403] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.034s
I20260709 12:21:36.615509  9268 tablet_bootstrap.cc:654] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.615408  9316 raft_consensus.cc:493] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.622175  9269 raft_consensus.cc:359] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.614027  9220 raft_consensus.cc:2468] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:36.623276  9316 raft_consensus.cc:515] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.624500  9269 raft_consensus.cc:385] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.624810  9269 raft_consensus.cc:740] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.625649  9316 leader_election.cc:290] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:36.626524  9272 raft_consensus.cc:359] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.627168  9272 raft_consensus.cc:385] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.627434  9272 raft_consensus.cc:740] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.629200  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.630129  9074 raft_consensus.cc:2468] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:36.612977  9222 raft_consensus.cc:2468] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:36.634413  9272 consensus_queue.cc:260] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.631843  9269 consensus_queue.cc:260] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.637225  9272 ts_tablet_manager.cc:1434] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:21:36.637643  9269 ts_tablet_manager.cc:1434] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:21:36.638228  9272 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.631946  8963 leader_election.cc:304] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.640057  9316 raft_consensus.cc:2804] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.640439  9316 raft_consensus.cc:493] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.640764  9316 raft_consensus.cc:3060] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.635673  9113 leader_election.cc:304] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:36.642668  9297 raft_consensus.cc:2804] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.643746  9270 ts_tablet_manager.cc:1434] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.153s	user 0.007s	sys 0.000s
I20260709 12:21:36.644870  9272 tablet_bootstrap.cc:654] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.645004  9270 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.634635  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:36.646934  9220 raft_consensus.cc:2468] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:36.649597  9316 raft_consensus.cc:515] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.653226  9316 leader_election.cc:290] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:36.654808  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:36.655445  9074 raft_consensus.cc:3060] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.655627  9269 tablet_bootstrap.cc:492] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:36.657785  9297 raft_consensus.cc:697] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:36.658746  9297 consensus_queue.cc:237] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.659719  9268 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:36.660394  9268 ts_tablet_manager.cc:1403] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.000s
I20260709 12:21:36.663081  9074 raft_consensus.cc:2468] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:36.664369  8963 leader_election.cc:304] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:36.664467  9268 raft_consensus.cc:359] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.667675  9268 raft_consensus.cc:385] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.667991  9268 raft_consensus.cc:740] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.668154  9316 raft_consensus.cc:2804] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.669147  9316 raft_consensus.cc:697] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:36.670897  9323 raft_consensus.cc:493] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.671402  9323 raft_consensus.cc:515] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.673525  9323 leader_election.cc:290] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.675297  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2290f27d484a4682a825fb8d1bb87924" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.675944  9075 raft_consensus.cc:2468] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:36.670058  9316 consensus_queue.cc:237] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:36.676963  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2290f27d484a4682a825fb8d1bb87924" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:36.678013  9186 leader_election.cc:304] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:36.678174  9146 raft_consensus.cc:2468] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:36.679466  9350 raft_consensus.cc:2804] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.680207  9350 raft_consensus.cc:493] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.680631  9350 raft_consensus.cc:3060] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.681277  9270 tablet_bootstrap.cc:654] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.682037  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:36.685432  9220 raft_consensus.cc:3060] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.690685  9286 consensus_queue.cc:1048] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.702126  9270 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.703814  9270 ts_tablet_manager.cc:1403] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.059s	user 0.007s	sys 0.005s
I20260709 12:21:36.707809  8898 catalog_manager.cc:5697] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.711207  9270 raft_consensus.cc:359] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.711920  9270 raft_consensus.cc:385] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.712240  9270 raft_consensus.cc:740] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.708033  9220 raft_consensus.cc:2468] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:36.713055  8896 catalog_manager.cc:5697] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.714649  9270 consensus_queue.cc:260] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.717260  9270 ts_tablet_manager.cc:1434] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:36.718219  9270 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.722354  9286 consensus_queue.cc:1048] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.726155  9270 tablet_bootstrap.cc:654] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.727270  9316 raft_consensus.cc:493] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.727839  9316 raft_consensus.cc:515] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.730666  9268 consensus_queue.cc:260] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.735006  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8984e03dbda34e0fb5f577335d3870ec" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:36.736126  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8984e03dbda34e0fb5f577335d3870ec" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:36.730154  9316 leader_election.cc:290] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.752110  9322 raft_consensus.cc:493] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.803004  9269 tablet_bootstrap.cc:700] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: Time spent opening tablet: real 0.127s	user 0.001s	sys 0.001s
I20260709 12:21:36.807392  9269 tablet_bootstrap.cc:654] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.808095  9268 ts_tablet_manager.cc:1434] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.147s	user 0.010s	sys 0.000s
I20260709 12:21:36.810633  9322 raft_consensus.cc:515] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.818570  9322 leader_election.cc:290] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
W20260709 12:21:36.803416  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 82078.8us
I20260709 12:21:36.825212  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:36.826285  9149 raft_consensus.cc:2468] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
W20260709 12:21:36.828770  8963 leader_election.cc:343] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.829356  9038 leader_election.cc:304] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
W20260709 12:21:36.829721  8963 leader_election.cc:343] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.830480  9322 raft_consensus.cc:2804] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.830416  8963 leader_election.cc:304] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:36.831064  9322 raft_consensus.cc:493] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.831468  9322 raft_consensus.cc:3060] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.828645  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:36.833861  9316 raft_consensus.cc:2749] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.841166  9272 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:36.841804  9272 ts_tablet_manager.cc:1403] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.204s	user 0.058s	sys 0.092s
I20260709 12:21:36.835848  9220 raft_consensus.cc:2468] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:36.845292  9272 raft_consensus.cc:359] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.846053  9272 raft_consensus.cc:385] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.846383  9272 raft_consensus.cc:740] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.847148  9272 consensus_queue.cc:260] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:36.848254  9270 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:36.848820  9270 ts_tablet_manager.cc:1403] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.131s	user 0.034s	sys 0.050s
I20260709 12:21:36.849138  9272 ts_tablet_manager.cc:1434] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:36.850585  9272 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:36.851801  9270 raft_consensus.cc:359] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.852532  9270 raft_consensus.cc:385] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.852851  9270 raft_consensus.cc:740] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.853741  9270 consensus_queue.cc:260] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.856014  9270 ts_tablet_manager.cc:1434] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:36.857060  9270 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:36.857328  9272 tablet_bootstrap.cc:654] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.863865  9270 tablet_bootstrap.cc:654] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.833360  9350 raft_consensus.cc:515] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.882784  9322 raft_consensus.cc:515] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:36.892809  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2290f27d484a4682a825fb8d1bb87924" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:36.893612  9075 raft_consensus.cc:3060] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.871536  9297 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.894840  9297 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.892917  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2290f27d484a4682a825fb8d1bb87924" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:36.889006  9350 leader_election.cc:290] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:36.898974  9297 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:36.914638  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:36.916512  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
W20260709 12:21:36.916870  9113 leader_election.cc:343] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.917189  9146 raft_consensus.cc:3060] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.904043  9149 raft_consensus.cc:3060] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.920972  9322 leader_election.cc:290] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:36.931876  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:36.932596  9220 raft_consensus.cc:3060] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.935633  9149 raft_consensus.cc:2468] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:36.935655  9366 consensus_queue.cc:1048] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.938933  9186 leader_election.cc:304] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:36.942651  9316 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.943221  9316 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:36.947692  9316 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:36.948256  9268 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:36.949049  9075 raft_consensus.cc:2468] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:37.006778  9269 tablet_bootstrap.cc:492] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:37.007400  9269 ts_tablet_manager.cc:1403] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.352s	user 0.020s	sys 0.011s
I20260709 12:21:36.960312  9350 raft_consensus.cc:2804] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.009814  9272 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.010537  9272 ts_tablet_manager.cc:1403] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.160s	user 0.027s	sys 0.038s
I20260709 12:21:37.008567  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:37.012965  9350 raft_consensus.cc:697] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:37.012894  9075 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.013206  9340 raft_consensus.cc:493] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.019728  9340 raft_consensus.cc:515] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:37.020291  9388 log.cc:927] Time spent T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:21:37.024564  9268 tablet_bootstrap.cc:654] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.016384  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.021791  9149 raft_consensus.cc:2468] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:37.031275  8963 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:37.032408  9348 raft_consensus.cc:2804] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.032852  9348 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.032684  9269 raft_consensus.cc:359] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.033341  9348 raft_consensus.cc:3060] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.034549  9350 consensus_queue.cc:237] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.035866  9272 raft_consensus.cc:359] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.036526  9272 raft_consensus.cc:385] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.036778  9272 raft_consensus.cc:740] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.033488  9269 raft_consensus.cc:385] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.037700  9269 raft_consensus.cc:740] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.037451  9272 consensus_queue.cc:260] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:37.038339  8964 leader_election.cc:343] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.040258  9146 raft_consensus.cc:2468] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.047695  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.048077  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.048489  9341 raft_consensus.cc:493] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.049021  9341 raft_consensus.cc:515] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.054826  9272 ts_tablet_manager.cc:1434] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 12:21:37.055728  9272 tablet_bootstrap.cc:492] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:37.059283  9341 leader_election.cc:290] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:37.060318  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.040632  9340 leader_election.cc:290] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.066547  9220 raft_consensus.cc:2468] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.072679  9269 consensus_queue.cc:260] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.075100  9222 raft_consensus.cc:2468] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:37.078815  9348 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.080106  8998 raft_consensus.cc:2393] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in current term 1: Already voted for candidate e47ee72c4e114aef9818c7c324ed1938 in this term.
I20260709 12:21:37.080153  9269 ts_tablet_manager.cc:1434] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.002s
I20260709 12:21:37.081414  9269 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:37.087742  9270 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:37.088335  9270 ts_tablet_manager.cc:1403] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.231s	user 0.041s	sys 0.031s
I20260709 12:21:37.092239  9367 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.092819  9367 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.113446  9270 raft_consensus.cc:359] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.114063  9270 raft_consensus.cc:385] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.114292  9270 raft_consensus.cc:740] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.133846  9269 tablet_bootstrap.cc:654] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.134577  9316 consensus_queue.cc:1048] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.118093  9270 consensus_queue.cc:260] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.140818  9270 ts_tablet_manager.cc:1434] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.003s
I20260709 12:21:37.147351  9348 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.148485  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
W20260709 12:21:37.162936  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 69167.1us
I20260709 12:21:37.172555  9039 leader_election.cc:304] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:37.174093  9397 consensus_queue.cc:1048] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:37.180469  9399 consensus_queue.cc:1048] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.182675  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:37.183955  9398 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.188823  9401 raft_consensus.cc:2804] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.189963  9398 consensus_queue.cc:1048] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.193127  9401 raft_consensus.cc:697] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:37.193953  9401 consensus_queue.cc:237] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.234745  9113 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:37.236003  9394 raft_consensus.cc:3060] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.245150  9394 raft_consensus.cc:2749] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.245954  9394 consensus_queue.cc:1048] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:37.253381  8964 leader_election.cc:343] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.321094  9268 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:37.321619  9268 ts_tablet_manager.cc:1403] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.374s	user 0.020s	sys 0.014s
I20260709 12:21:37.324915  9268 raft_consensus.cc:359] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.325605  9268 raft_consensus.cc:385] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.325874  9268 raft_consensus.cc:740] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.326647  9268 consensus_queue.cc:260] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.328321  9268 ts_tablet_manager.cc:1434] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.008s
I20260709 12:21:37.329279  9268 tablet_bootstrap.cc:492] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:37.335270  9268 tablet_bootstrap.cc:654] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.340309  9268 tablet_bootstrap.cc:492] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:37.340755  9268 ts_tablet_manager.cc:1403] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
W20260709 12:21:37.342486  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:37.344717  9268 raft_consensus.cc:359] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.345479  9268 raft_consensus.cc:385] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.345721  9268 raft_consensus.cc:740] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.346374  9268 consensus_queue.cc:260] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.347975  9268 ts_tablet_manager.cc:1434] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:37.348912  9268 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:37.352340  9269 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:37.352923  9269 ts_tablet_manager.cc:1403] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.272s	user 0.017s	sys 0.000s
I20260709 12:21:37.354430  9268 tablet_bootstrap.cc:654] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.162300  9272 tablet_bootstrap.cc:654] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.355863  9269 raft_consensus.cc:359] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.356666  9269 raft_consensus.cc:385] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.357000  9269 raft_consensus.cc:740] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.357904  9269 consensus_queue.cc:260] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.358912  9268 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:37.359490  9268 ts_tablet_manager.cc:1403] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:21:37.362409  9268 raft_consensus.cc:359] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.363137  9268 raft_consensus.cc:385] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.363463  9268 raft_consensus.cc:740] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.364326  9268 consensus_queue.cc:260] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.366389  9397 raft_consensus.cc:493] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.366909  9397 raft_consensus.cc:515] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.369036  9397 leader_election.cc:290] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.369263  9270 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:37.371134  9268 ts_tablet_manager.cc:1434] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:21:37.372172  9272 tablet_bootstrap.cc:492] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.372638  9268 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:37.374975  9269 ts_tablet_manager.cc:1434] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.022s	user 0.015s	sys 0.003s
I20260709 12:21:37.372700  9272 ts_tablet_manager.cc:1403] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.317s	user 0.020s	sys 0.000s
I20260709 12:21:37.376050  9269 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:37.385093  9268 tablet_bootstrap.cc:654] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.396931  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:37.376672  9404 log.cc:927] Time spent T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.169s	user 0.004s	sys 0.000s
I20260709 12:21:37.403326  9367 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:37.384670  9272 raft_consensus.cc:359] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.395977  9146 raft_consensus.cc:3060] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.404414  9272 raft_consensus.cc:385] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.404825  9272 raft_consensus.cc:740] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.405576  9272 consensus_queue.cc:260] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.407511  9272 ts_tablet_manager.cc:1434] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.009s
I20260709 12:21:37.408871  9272 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
W20260709 12:21:37.396180  9406 log.cc:927] Time spent T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.174s	user 0.000s	sys 0.003s
W20260709 12:21:37.394017  9112 leader_election.cc:343] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.412822  9112 leader_election.cc:304] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:37.413430  9000 raft_consensus.cc:3060] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.415176  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.419157  9272 tablet_bootstrap.cc:654] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.419246  9113 leader_election.cc:304] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:37.384585  9270 tablet_bootstrap.cc:654] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.422670  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.423449  9221 raft_consensus.cc:2468] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:37.394266  9340 raft_consensus.cc:493] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.427430  9340 raft_consensus.cc:515] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.386261  9269 tablet_bootstrap.cc:654] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.437116  9039 leader_election.cc:304] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:37.438510  9367 raft_consensus.cc:2804] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.438917  9367 raft_consensus.cc:493] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.439317  9367 raft_consensus.cc:3060] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.429946  9340 leader_election.cc:290] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.444649  9297 raft_consensus.cc:2804] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.449446  9297 raft_consensus.cc:493] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.449486  9394 consensus_queue.cc:1048] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.446955  9341 raft_consensus.cc:2749] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.450909  9341 consensus_queue.cc:1048] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.450011  9297 raft_consensus.cc:3060] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.478701  9416 consensus_queue.cc:1048] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:21:37.506260  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:37.507277  9147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:37.507762  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:37.508051  9147 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.510409  9322 consensus_queue.cc:1048] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.512794  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.516716  9148 raft_consensus.cc:2468] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:37.521979  8963 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:37.523739  9348 raft_consensus.cc:2804] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.524232  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.524930  9398 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.526777  9038 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:37.527115  9367 raft_consensus.cc:515] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.528362  9366 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:37.528867  9366 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:37.529340  9366 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.531843  9272 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.532367  9272 ts_tablet_manager.cc:1403] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.124s	user 0.022s	sys 0.024s
I20260709 12:21:37.525509  9297 raft_consensus.cc:515] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.516606  9316 raft_consensus.cc:493] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.535619  9316 raft_consensus.cc:515] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.535738  9272 raft_consensus.cc:359] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.536516  9272 raft_consensus.cc:385] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.536857  9272 raft_consensus.cc:740] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.537863  9316 leader_election.cc:290] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:37.537976  9272 consensus_queue.cc:260] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.538619  9146 raft_consensus.cc:2393] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
W20260709 12:21:37.539072  9112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 89810.3us
I20260709 12:21:37.540092  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.541103  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:37.541795  9219 raft_consensus.cc:3060] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:37.542131  8964 leader_election.cc:343] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.542066  9272 ts_tablet_manager.cc:1434] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:37.545521  9272 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:37.544523  9268 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:37.546720  9268 ts_tablet_manager.cc:1403] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.175s	user 0.031s	sys 0.049s
I20260709 12:21:37.547276  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:37.550637  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:37.551474  9145 raft_consensus.cc:2393] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
W20260709 12:21:37.551923  8963 leader_election.cc:343] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.552553  8963 leader_election.cc:304] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:37.553565  9316 raft_consensus.cc:2749] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.554904  9415 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.555447  9415 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.557624  9415 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.555835  9268 raft_consensus.cc:359] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.558389  9348 raft_consensus.cc:697] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:37.559333  9272 tablet_bootstrap.cc:654] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.559484  9348 consensus_queue.cc:237] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.558545  9268 raft_consensus.cc:385] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.560520  9268 raft_consensus.cc:740] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.560956  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:37.561323  9268 consensus_queue.cc:260] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.562041  9145 raft_consensus.cc:2393] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate e47ee72c4e114aef9818c7c324ed1938 in this term.
I20260709 12:21:37.563474  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.565696  9187 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:37.563791  9268 ts_tablet_manager.cc:1434] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:21:37.571048  9350 raft_consensus.cc:3060] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.573457  9268 tablet_bootstrap.cc:492] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:37.583338  8895 catalog_manager.cc:5697] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.594715  9270 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:37.595254  9270 ts_tablet_manager.cc:1403] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.226s	user 0.032s	sys 0.037s
I20260709 12:21:37.538688  9367 leader_election.cc:290] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.592363  9269 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:37.596563  9269 ts_tablet_manager.cc:1403] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.221s	user 0.008s	sys 0.019s
I20260709 12:21:37.598794  9268 tablet_bootstrap.cc:654] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:37.568195  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 119439us
W20260709 12:21:37.601255  9113 leader_election.cc:343] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.601370  9297 leader_election.cc:290] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.602047  9297 consensus_queue.cc:1048] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.602737  9269 raft_consensus.cc:359] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
W20260709 12:21:37.603154  9113 leader_election.cc:343] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.603457  9269 raft_consensus.cc:385] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.603765  9269 raft_consensus.cc:740] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.604525  9269 consensus_queue.cc:260] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.605172  9366 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.603765  9113 leader_election.cc:304] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:37.622140  9341 raft_consensus.cc:2749] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.623970  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:37.624882  9072 raft_consensus.cc:2393] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:37.629294  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efbde85ba184e82a1ceb3e8ded5d1f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:37.634929  9219 raft_consensus.cc:2468] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.626246  9270 raft_consensus.cc:359] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.636687  9269 ts_tablet_manager.cc:1434] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.001s
I20260709 12:21:37.636929  9270 raft_consensus.cc:385] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.637526  9270 raft_consensus.cc:740] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.635902  9074 raft_consensus.cc:1240] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Rejecting Update request from peer e47ee72c4e114aef9818c7c324ed1938 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:37.640018  9269 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:37.648120  9113 leader_election.cc:304] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:37.650262  9270 consensus_queue.cc:260] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.649509  9341 raft_consensus.cc:2749] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.656437  9366 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:37.656879  9039 leader_election.cc:304] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:37.657054  9366 consensus_queue.cc:1048] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.657121  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:37.658876  9316 consensus_queue.cc:1059] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.660205  9316 raft_consensus.cc:3055] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.660681  9316 raft_consensus.cc:740] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:37.661213  9397 raft_consensus.cc:2804] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.661789  9397 raft_consensus.cc:697] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:37.662853  9397 consensus_queue.cc:237] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.661872  9316 consensus_queue.cc:260] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.676215  9270 ts_tablet_manager.cc:1434] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.007s
I20260709 12:21:37.678872  9269 tablet_bootstrap.cc:654] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.684162  9270 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:37.678032  9316 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.693181  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:37.694731  9366 consensus_queue.cc:1048] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.694911  8896 catalog_manager.cc:5697] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.698535  9039 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:37.700862  9367 raft_consensus.cc:2749] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:37.701602  9350 raft_consensus.cc:2749] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.717172  9272 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.717945  9272 ts_tablet_manager.cc:1403] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.173s	user 0.016s	sys 0.003s
I20260709 12:21:37.722630  9272 raft_consensus.cc:359] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.723548  9272 raft_consensus.cc:385] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.723884  9272 raft_consensus.cc:740] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.724812  9272 consensus_queue.cc:260] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.727273  9397 raft_consensus.cc:493] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.727998  9272 ts_tablet_manager.cc:1434] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:37.727794  9397 raft_consensus.cc:515] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.729928  9397 leader_election.cc:290] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.732225  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.732968  9222 raft_consensus.cc:2468] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:37.734272  9268 tablet_bootstrap.cc:492] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:37.735037  9268 ts_tablet_manager.cc:1403] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.164s	user 0.023s	sys 0.000s
I20260709 12:21:37.735466  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.736150  9000 raft_consensus.cc:2468] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:37.737702  9039 leader_election.cc:304] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:37.737854  9268 raft_consensus.cc:359] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.738637  9268 raft_consensus.cc:385] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.738916  9268 raft_consensus.cc:740] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.739779  9268 consensus_queue.cc:260] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.740463  9397 raft_consensus.cc:2804] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.740906  9397 raft_consensus.cc:493] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.741894  9350 consensus_queue.cc:1048] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.741250  9397 raft_consensus.cc:3060] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.752285  9268 ts_tablet_manager.cc:1434] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:21:37.755837  9367 raft_consensus.cc:493] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.756520  9367 raft_consensus.cc:515] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.758724  9367 leader_election.cc:290] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.760663  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:37.761417  9270 tablet_bootstrap.cc:654] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:37.762751  9038 leader_election.cc:343] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.766516  9397 raft_consensus.cc:515] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.768653  9397 leader_election.cc:290] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.770390  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.772631  9272 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:37.773619  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:37.774289  9222 raft_consensus.cc:3060] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.776722  9341 raft_consensus.cc:493] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.777379  9341 raft_consensus.cc:515] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.778872  9269 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:37.779381  9269 ts_tablet_manager.cc:1403] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.141s	user 0.020s	sys 0.006s
I20260709 12:21:37.780668  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.781396  9272 tablet_bootstrap.cc:654] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.781352  9000 raft_consensus.cc:2468] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:37.782559  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:37.782871  9039 leader_election.cc:343] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:37.784682  9112 leader_election.cc:343] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.792833  9113 leader_election.cc:304] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:37.793852  9297 raft_consensus.cc:2804] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.794274  9297 raft_consensus.cc:493] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.794651  9297 raft_consensus.cc:3060] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.797348  9272 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.797845  9272 ts_tablet_manager.cc:1403] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.002s
I20260709 12:21:37.785310  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:37.798995  9000 raft_consensus.cc:3060] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.800758  9272 raft_consensus.cc:359] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.801525  9272 raft_consensus.cc:385] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.801788  9272 raft_consensus.cc:740] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.802587  9272 consensus_queue.cc:260] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.804770  9272 ts_tablet_manager.cc:1434] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:21:37.804522  8898 catalog_manager.cc:5697] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.805722  9272 tablet_bootstrap.cc:492] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:37.807154  9000 raft_consensus.cc:2468] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.812008  9272 tablet_bootstrap.cc:654] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.816731  9039 leader_election.cc:304] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:37.822041  9423 raft_consensus.cc:493] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.822661  9423 raft_consensus.cc:515] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:37.826781  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 43962.2us
I20260709 12:21:37.827658  9394 raft_consensus.cc:493] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.828318  9394 raft_consensus.cc:515] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.830534  9394 leader_election.cc:290] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.789304  9269 raft_consensus.cc:359] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.832360  9269 raft_consensus.cc:385] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.832777  9269 raft_consensus.cc:740] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.832927  9322 raft_consensus.cc:2749] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.833151  9270 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:37.833637  9270 ts_tablet_manager.cc:1403] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.150s	user 0.009s	sys 0.009s
I20260709 12:21:37.834779  9269 consensus_queue.cc:260] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.836401  9270 raft_consensus.cc:359] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.837085  9270 raft_consensus.cc:385] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.837347  9270 raft_consensus.cc:740] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.839063  9269 ts_tablet_manager.cc:1434] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 12:21:37.839887  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed9ca59931ad482da231fc05861e3b93" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.840137  9269 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:37.817160  8894 catalog_manager.cc:5697] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.817508  9397 raft_consensus.cc:493] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.842883  9397 raft_consensus.cc:515] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:37.844708  9113 leader_election.cc:343] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.845168  9397 leader_election.cc:290] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:37.847076  9367 raft_consensus.cc:493] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.847759  9367 raft_consensus.cc:515] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.846995  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8984e03dbda34e0fb5f577335d3870ec" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:37.848639  9149 raft_consensus.cc:2468] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:37.848959  9272 tablet_bootstrap.cc:492] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.849092  9269 tablet_bootstrap.cc:654] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.849470  9272 ts_tablet_manager.cc:1403] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.015s
I20260709 12:21:37.851172  9367 leader_election.cc:290] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.851781  9038 leader_election.cc:304] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:37.852667  9397 raft_consensus.cc:2804] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.852427  9272 raft_consensus.cc:359] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.853176  9397 raft_consensus.cc:493] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.853418  9272 raft_consensus.cc:385] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.853590  9397 raft_consensus.cc:3060] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.853876  9272 raft_consensus.cc:740] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.854960  9272 consensus_queue.cc:260] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.856405  8898 catalog_manager.cc:5697] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.858016  9272 ts_tablet_manager.cc:1434] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:37.859303  9272 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:37.860325  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4507e0b0b1a74cf98c0b22397b2d56f1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:37.863106  9112 leader_election.cc:343] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.864259  9423 leader_election.cc:290] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:37.839100  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed9ca59931ad482da231fc05861e3b93" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.816824  9341 leader_election.cc:290] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:37.872408  9272 tablet_bootstrap.cc:654] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.838024  9270 consensus_queue.cc:260] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.874745  9316 raft_consensus.cc:493] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.875276  9316 raft_consensus.cc:515] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.876165  9270 ts_tablet_manager.cc:1434] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:21:37.876428  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.820621  9268 tablet_bootstrap.cc:492] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:37.876309  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8984e03dbda34e0fb5f577335d3870ec" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.879360  8998 raft_consensus.cc:2468] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:37.880013  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:37.880666  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:37.881296  8999 raft_consensus.cc:2468] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:37.881425  9039 leader_election.cc:304] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
W20260709 12:21:37.881433  8964 leader_election.cc:343] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.882431  9322 raft_consensus.cc:2804] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.882974  9270 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:37.882939  9322 raft_consensus.cc:697] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:37.883962  9322 consensus_queue.cc:237] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.884815  9316 leader_election.cc:290] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:37.890288  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4507e0b0b1a74cf98c0b22397b2d56f1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:37.892105  9113 leader_election.cc:343] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.892567  9113 leader_election.cc:304] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
W20260709 12:21:37.893863  9113 leader_election.cc:343] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.894287  9113 leader_election.cc:304] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:37.895272  9423 raft_consensus.cc:2749] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.896039  9423 raft_consensus.cc:2749] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.813346  9297 raft_consensus.cc:515] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.902642  9272 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:37.903323  9272 ts_tablet_manager.cc:1403] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.004s
I20260709 12:21:37.905627  9270 tablet_bootstrap.cc:654] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.906898  9268 tablet_bootstrap.cc:654] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.907027  9272 raft_consensus.cc:359] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.908010  9272 raft_consensus.cc:385] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.908372  9272 raft_consensus.cc:740] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.909559  9272 consensus_queue.cc:260] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.912950  9272 ts_tablet_manager.cc:1434] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:21:37.913293  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:37.914095  9272 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
W20260709 12:21:37.914922  9112 leader_election.cc:343] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.870177  9397 raft_consensus.cc:515] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.922613  9272 tablet_bootstrap.cc:654] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.922950  9297 leader_election.cc:290] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:37.920444  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:37.921486  9039 leader_election.cc:304] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:37.925486  9000 raft_consensus.cc:3060] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.925906  9222 raft_consensus.cc:2468] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
W20260709 12:21:37.926965  9039 leader_election.cc:343] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.926671  8894 catalog_manager.cc:5697] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.928171  9322 raft_consensus.cc:2804] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.928218  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8984e03dbda34e0fb5f577335d3870ec" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:37.928653  9322 raft_consensus.cc:493] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.928922  8998 raft_consensus.cc:3060] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.929056  9322 raft_consensus.cc:3060] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.929875  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:37.924729  9397 leader_election.cc:290] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:37.920143  9269 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:37.934142  9269 ts_tablet_manager.cc:1403] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.094s	user 0.036s	sys 0.012s
I20260709 12:21:37.936975  9269 raft_consensus.cc:359] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.938879  9269 raft_consensus.cc:385] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.939337  9269 raft_consensus.cc:740] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.940635  9269 consensus_queue.cc:260] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:37.941128  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8984e03dbda34e0fb5f577335d3870ec" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:37.941272  9322 raft_consensus.cc:515] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:37.941829  9148 raft_consensus.cc:3060] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.942531  9074 raft_consensus.cc:2393] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:37.943303  9269 ts_tablet_manager.cc:1434] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:21:37.945380  8998 raft_consensus.cc:2468] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.943465  9322 leader_election.cc:290] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:37.952759  9269 tablet_bootstrap.cc:492] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:37.948633  9268 tablet_bootstrap.cc:492] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:37.958833  9268 ts_tablet_manager.cc:1403] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.206s	user 0.013s	sys 0.001s
I20260709 12:21:37.961588  9268 raft_consensus.cc:359] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.962991  9269 tablet_bootstrap.cc:654] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.963969  9000 raft_consensus.cc:2468] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:37.965293  9113 leader_election.cc:304] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:37.966590  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:37.967234  8998 raft_consensus.cc:3060] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.967279  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:37.970506  8963 leader_election.cc:304] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:37.972985  9039 leader_election.cc:304] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:37.973798  9322 raft_consensus.cc:2804] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.974274  9322 raft_consensus.cc:697] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:37.975136  9322 consensus_queue.cc:237] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.976519  9297 raft_consensus.cc:2804] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.977195  9297 raft_consensus.cc:697] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:37.962302  9268 raft_consensus.cc:385] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.979280  9268 raft_consensus.cc:740] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.988288  9148 raft_consensus.cc:2468] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:37.978184  9297 consensus_queue.cc:237] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:37.994863  9268 consensus_queue.cc:260] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:37.998616  9268 ts_tablet_manager.cc:1434] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
W20260709 12:21:37.994884  9039 leader_election.cc:343] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.000641  9268 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:38.002884  9272 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:38.003454  9272 ts_tablet_manager.cc:1403] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.090s	user 0.033s	sys 0.017s
I20260709 12:21:38.003739  8898 catalog_manager.cc:5697] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.005400  9270 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:38.006064  9270 ts_tablet_manager.cc:1403] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.123s	user 0.010s	sys 0.007s
I20260709 12:21:38.006704  9269 tablet_bootstrap.cc:492] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:38.007189  9269 ts_tablet_manager.cc:1403] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.005s
I20260709 12:21:38.008627  9268 tablet_bootstrap.cc:654] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.009346  8998 raft_consensus.cc:2468] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:38.009876  9269 raft_consensus.cc:359] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.011245  9269 raft_consensus.cc:385] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.010047  9270 raft_consensus.cc:359] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.011741  9270 raft_consensus.cc:385] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.012053  9270 raft_consensus.cc:740] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.012682  9316 raft_consensus.cc:2749] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.012776  9270 consensus_queue.cc:260] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.013149  9272 raft_consensus.cc:359] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.014158  9272 raft_consensus.cc:385] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.014529  9272 raft_consensus.cc:740] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.015363  9039 leader_election.cc:304] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:38.015215  9272 consensus_queue.cc:260] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.017518  9322 raft_consensus.cc:2804] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.017619  9272 ts_tablet_manager.cc:1434] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:21:38.018051  9322 raft_consensus.cc:697] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:38.018879  9272 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.019057  9269 raft_consensus.cc:740] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.018934  9322 consensus_queue.cc:237] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:38.020082  9269 consensus_queue.cc:260] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.023425  9269 ts_tablet_manager.cc:1434] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:21:38.025375  9272 tablet_bootstrap.cc:654] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.025764  9269 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:38.032105  9269 tablet_bootstrap.cc:654] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.018965  9270 ts_tablet_manager.cc:1434] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:21:38.045269  9270 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:38.057070  9272 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:38.057612  9272 ts_tablet_manager.cc:1403] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.027s
I20260709 12:21:38.058490  9269 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:38.059043  9269 ts_tablet_manager.cc:1403] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.003s
I20260709 12:21:38.060504  9272 raft_consensus.cc:359] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.060817  9423 raft_consensus.cc:493] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.061251  9272 raft_consensus.cc:385] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.061582  9272 raft_consensus.cc:740] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.061436  9423 raft_consensus.cc:515] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.061777  9269 raft_consensus.cc:359] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:38.062652  9269 raft_consensus.cc:385] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.062389  9272 consensus_queue.cc:260] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.062970  9269 raft_consensus.cc:740] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.063786  9423 leader_election.cc:290] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.063856  9269 consensus_queue.cc:260] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:38.064891  9272 ts_tablet_manager.cc:1434] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:38.065052  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.065747  9074 raft_consensus.cc:2468] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:38.065980  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:38.067101  9112 leader_election.cc:304] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
W20260709 12:21:38.067680  9113 leader_election.cc:343] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.067936  9269 ts_tablet_manager.cc:1434] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:21:38.069239  9269 tablet_bootstrap.cc:492] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:38.067999  9341 raft_consensus.cc:2804] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.070300  9341 raft_consensus.cc:493] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.070729  9341 raft_consensus.cc:3060] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.074955  9269 tablet_bootstrap.cc:654] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.076169  9272 tablet_bootstrap.cc:492] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.077833  9268 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:38.078341  9268 ts_tablet_manager.cc:1403] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.078s	user 0.006s	sys 0.005s
I20260709 12:21:38.081065  9268 raft_consensus.cc:359] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.081728  9268 raft_consensus.cc:385] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.082003  9268 raft_consensus.cc:740] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.082672  9268 consensus_queue.cc:260] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.086043  9272 tablet_bootstrap.cc:654] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.088415  9270 tablet_bootstrap.cc:654] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.088629  8896 catalog_manager.cc:5697] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.089761  9350 raft_consensus.cc:493] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.090269  9350 raft_consensus.cc:515] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.091703  8895 catalog_manager.cc:5697] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.092525  9350 leader_election.cc:290] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:38.092826  9268 ts_tablet_manager.cc:1434] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:21:38.094058  9268 tablet_bootstrap.cc:492] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:38.095738  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed9ca59931ad482da231fc05861e3b93" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.096439  9148 raft_consensus.cc:2468] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:38.101670  9186 leader_election.cc:304] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:38.094847  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed9ca59931ad482da231fc05861e3b93" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.103168  8998 raft_consensus.cc:2468] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:38.107030  9268 tablet_bootstrap.cc:654] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.108795  9350 raft_consensus.cc:2804] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.109233  9350 raft_consensus.cc:493] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.111189  9350 raft_consensus.cc:3060] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.121181  9341 raft_consensus.cc:515] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.124496  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:38.125106  9074 raft_consensus.cc:3060] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.139206  9350 raft_consensus.cc:515] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.138172  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
W20260709 12:21:38.142560  9113 leader_election.cc:343] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.136085  9341 leader_election.cc:290] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.141016  9074 raft_consensus.cc:2468] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:38.143829  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed9ca59931ad482da231fc05861e3b93" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:38.144169  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed9ca59931ad482da231fc05861e3b93" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:38.144557  9148 raft_consensus.cc:3060] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.144778  8998 raft_consensus.cc:3060] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.145236  9112 leader_election.cc:304] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:38.143025  9316 consensus_queue.cc:1048] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.146128  9341 raft_consensus.cc:2804] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.146700  9341 raft_consensus.cc:697] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:38.147518  9341 consensus_queue.cc:237] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.151868  8998 raft_consensus.cc:2468] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:38.153184  9187 leader_election.cc:304] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:38.154093  9416 raft_consensus.cc:2804] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.154614  9416 raft_consensus.cc:697] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:38.155432  9416 consensus_queue.cc:237] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.143200  9350 leader_election.cc:290] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:38.160094  9322 consensus_queue.cc:1048] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.161473  9148 raft_consensus.cc:2468] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:38.165186  9268 tablet_bootstrap.cc:492] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:38.165730  9268 ts_tablet_manager.cc:1403] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.007s
I20260709 12:21:38.167630  9270 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:38.168289  9270 ts_tablet_manager.cc:1403] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.123s	user 0.026s	sys 0.014s
I20260709 12:21:38.168835  9268 raft_consensus.cc:359] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.169587  9268 raft_consensus.cc:385] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.169864  9268 raft_consensus.cc:740] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
W20260709 12:21:38.171657  9021 debug-util.cc:398] Leaking SignalData structure 0x7b0800236100 after lost signal to thread 30823
I20260709 12:21:38.172760  9272 tablet_bootstrap.cc:492] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
W20260709 12:21:38.172870  9021 debug-util.cc:398] Leaking SignalData structure 0x7b0800272b60 after lost signal to thread 8932
I20260709 12:21:38.173485  9272 ts_tablet_manager.cc:1403] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.016s
W20260709 12:21:38.174523  9021 debug-util.cc:398] Leaking SignalData structure 0x7b08003785a0 after lost signal to thread 9024
W20260709 12:21:38.175848  9021 debug-util.cc:398] Leaking SignalData structure 0x7b08002fcfc0 after lost signal to thread 9099
I20260709 12:21:38.178114  9269 tablet_bootstrap.cc:492] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:38.178646  9269 ts_tablet_manager.cc:1403] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.015s
I20260709 12:21:38.180892  9270 raft_consensus.cc:359] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.180981  9322 consensus_queue.cc:1048] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.181645  9270 raft_consensus.cc:385] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.182021  9270 raft_consensus.cc:740] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.181645  9269 raft_consensus.cc:359] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.188097  9269 raft_consensus.cc:385] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.188428  9269 raft_consensus.cc:740] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.186450  9268 consensus_queue.cc:260] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.190055  9272 raft_consensus.cc:359] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.191237  9272 raft_consensus.cc:385] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.191527  9272 raft_consensus.cc:740] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.192174  9269 consensus_queue.cc:260] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.192524  9272 consensus_queue.cc:260] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.199605  9269 ts_tablet_manager.cc:1434] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 12:21:38.190625  9270 consensus_queue.cc:260] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.204289  9272 ts_tablet_manager.cc:1434] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.004s
I20260709 12:21:38.210079  9272 tablet_bootstrap.cc:492] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.201396  9269 tablet_bootstrap.cc:492] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:38.211320  9270 ts_tablet_manager.cc:1434] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 12:21:38.206924  9416 raft_consensus.cc:493] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.219321  9416 raft_consensus.cc:515] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.221186  9399 raft_consensus.cc:493] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.228434  8894 catalog_manager.cc:5697] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.229676  9316 consensus_queue.cc:1048] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.247893  9399 raft_consensus.cc:515] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.250031  9399 leader_election.cc:290] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.252651  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf8624d5330403dbdebaeaabf396c07" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:38.253156  9272 tablet_bootstrap.cc:654] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:38.201740  9021 debug-util.cc:398] Leaking SignalData structure 0x7b0800403240 after lost signal to thread 9173
I20260709 12:21:38.230860  9270 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:38.254917  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4507e0b0b1a74cf98c0b22397b2d56f1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.255580  9074 raft_consensus.cc:2468] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
W20260709 12:21:38.256220  9021 debug-util.cc:398] Leaking SignalData structure 0x7b08001cd1c0 after lost signal to thread 9247
I20260709 12:21:38.257736  9186 leader_election.cc:304] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
W20260709 12:21:38.258460  9021 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0ea0 after lost signal to thread 9344
I20260709 12:21:38.258564  9286 raft_consensus.cc:2804] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.259009  9286 raft_consensus.cc:493] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:38.259184  9021 debug-util.cc:398] Leaking SignalData structure 0x7b080005c3a0 after lost signal to thread 9379
I20260709 12:21:38.259369  9286 raft_consensus.cc:3060] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.262048  9270 tablet_bootstrap.cc:654] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.231329  9268 ts_tablet_manager.cc:1434] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 12:21:38.264680  9268 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:38.256734  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf8624d5330403dbdebaeaabf396c07" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.266700  9148 raft_consensus.cc:2468] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
W20260709 12:21:38.257050  9039 leader_election.cc:343] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.268306  9038 leader_election.cc:304] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:38.269431  9399 raft_consensus.cc:2804] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.269799  9399 raft_consensus.cc:493] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.270130  9399 raft_consensus.cc:3060] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.246357  8896 catalog_manager.cc:5697] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.276211  9269 tablet_bootstrap.cc:654] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.276369  9366 consensus_queue.cc:1048] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.283516  9416 leader_election.cc:290] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:38.284952  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4507e0b0b1a74cf98c0b22397b2d56f1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.286814  9148 raft_consensus.cc:2468] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:38.293200  9268 tablet_bootstrap.cc:654] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.280390  9286 raft_consensus.cc:515] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.301767  9416 raft_consensus.cc:493] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.316461  9272 tablet_bootstrap.cc:492] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:38.316735  9286 leader_election.cc:290] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:38.317024  9272 ts_tablet_manager.cc:1403] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.107s	user 0.002s	sys 0.010s
I20260709 12:21:38.319481  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4507e0b0b1a74cf98c0b22397b2d56f1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:38.320127  9074 raft_consensus.cc:3060] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.319927  9272 raft_consensus.cc:359] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.320645  9272 raft_consensus.cc:385] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.320945  9272 raft_consensus.cc:740] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.321883  9272 consensus_queue.cc:260] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.312661  9416 raft_consensus.cc:515] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.323966  9272 ts_tablet_manager.cc:1434] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:38.324048  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4507e0b0b1a74cf98c0b22397b2d56f1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:38.324713  9145 raft_consensus.cc:3060] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.324983  9272 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.325886  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.327364  9416 leader_election.cc:290] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:38.328503  9187 leader_election.cc:343] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.328534  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.330291  9149 raft_consensus.cc:2468] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:38.331390  9272 tablet_bootstrap.cc:654] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.330631  9341 raft_consensus.cc:493] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.335189  9341 raft_consensus.cc:515] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.337411  9341 leader_election.cc:290] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:38.339085  9186 leader_election.cc:304] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.339274  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25135f5b10d44185b9d6be0f2fb5a02c" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:38.342461  9113 leader_election.cc:343] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.346738  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25135f5b10d44185b9d6be0f2fb5a02c" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:38.348383  9113 leader_election.cc:343] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.348871  9113 leader_election.cc:304] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.350505  9416 raft_consensus.cc:2804] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.350867  9297 raft_consensus.cc:2749] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.351357  9416 raft_consensus.cc:493] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.351693  9297 raft_consensus.cc:493] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.352310  9297 raft_consensus.cc:515] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.353430  9074 raft_consensus.cc:2468] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:38.355274  9186 leader_election.cc:304] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:38.356966  9286 raft_consensus.cc:2804] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.357658  9286 raft_consensus.cc:697] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:38.358549  9286 consensus_queue.cc:237] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.358901  9366 consensus_queue.cc:1048] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.351769  9416 raft_consensus.cc:3060] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.362344  9297 leader_election.cc:290] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:38.354619  9145 raft_consensus.cc:2468] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:38.367463  9366 raft_consensus.cc:493] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.377151  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:38.368074  9366 raft_consensus.cc:515] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.400856  9297 raft_consensus.cc:493] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:38.402657  9113 leader_election.cc:343] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.403869  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.417044  9269 tablet_bootstrap.cc:492] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:38.417606  9269 ts_tablet_manager.cc:1403] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.216s	user 0.017s	sys 0.046s
I20260709 12:21:38.417742  9366 leader_election.cc:290] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
W20260709 12:21:38.424671  9113 leader_election.cc:343] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.425258  9113 leader_election.cc:304] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.426855  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.427167  9341 raft_consensus.cc:2749] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.411586  9297 raft_consensus.cc:515] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.427789  9000 raft_consensus.cc:2393] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:38.429569  9297 leader_election.cc:290] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:38.430225  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.432458  9272 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:38.432964  9272 ts_tablet_manager.cc:1403] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.108s	user 0.011s	sys 0.012s
I20260709 12:21:38.432832  9269 raft_consensus.cc:359] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.433642  9269 raft_consensus.cc:385] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.433936  9269 raft_consensus.cc:740] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.434707  9269 consensus_queue.cc:260] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.414986  9399 raft_consensus.cc:515] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.435248  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45f96b48cfe14f749d63e8cb99981b8b" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.438486  9272 raft_consensus.cc:359] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.439195  9272 raft_consensus.cc:385] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.439465  9272 raft_consensus.cc:740] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.440207  9272 consensus_queue.cc:260] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.443365  9399 leader_election.cc:290] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.443271  9038 leader_election.cc:304] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.444825  9270 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:38.446810  9270 ts_tablet_manager.cc:1403] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.217s	user 0.020s	sys 0.002s
W20260709 12:21:38.444864  9112 leader_election.cc:343] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.448570  9272 ts_tablet_manager.cc:1434] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 12:21:38.448734  9269 ts_tablet_manager.cc:1434] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.004s
I20260709 12:21:38.448486  9350 raft_consensus.cc:493] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.450912  9272 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.451275  9269 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:38.451931  9399 raft_consensus.cc:3060] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.452119  9350 raft_consensus.cc:515] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:38.458063  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf8624d5330403dbdebaeaabf396c07" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:38.458791  9145 raft_consensus.cc:3060] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.460180  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.461092  8999 raft_consensus.cc:2393] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:38.462404  9272 tablet_bootstrap.cc:654] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.462536  9394 raft_consensus.cc:493] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.462795  9350 leader_election.cc:290] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:38.462988  9394 raft_consensus.cc:515] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.464960  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39d47714f6214edbaabd7255530669f6" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.465319  9394 leader_election.cc:290] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.463368  9350 raft_consensus.cc:493] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.467325  9350 raft_consensus.cc:515] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.471349  9186 leader_election.cc:304] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.474375  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.475268  9075 raft_consensus.cc:2393] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:38.476742  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:38.480356  9145 raft_consensus.cc:2468] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:38.470188  9350 leader_election.cc:290] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:38.482599  9038 leader_election.cc:304] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:38.483415  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:38.485219  9112 leader_election.cc:343] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.486758  9268 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:38.486142  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45f96b48cfe14f749d63e8cb99981b8b" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.487284  9268 ts_tablet_manager.cc:1403] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.223s	user 0.008s	sys 0.014s
I20260709 12:21:38.497725  9269 tablet_bootstrap.cc:654] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.491745  9270 raft_consensus.cc:359] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.513688  9270 raft_consensus.cc:385] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.490257  9268 raft_consensus.cc:359] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.515313  9268 raft_consensus.cc:385] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.495610  8896 catalog_manager.cc:5697] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.515672  9268 raft_consensus.cc:740] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.514153  9270 raft_consensus.cc:740] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.517606  9270 consensus_queue.cc:260] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.520962  9268 consensus_queue.cc:260] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.494263  9322 raft_consensus.cc:2804] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.498546  9286 raft_consensus.cc:3060] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.492527  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
W20260709 12:21:38.499128  9113 leader_election.cc:343] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.526494  9270 ts_tablet_manager.cc:1434] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.002s
I20260709 12:21:38.495213  9367 consensus_queue.cc:1048] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:21:38.523485  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf8624d5330403dbdebaeaabf396c07" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:38.536257  9268 ts_tablet_manager.cc:1434] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.002s
I20260709 12:21:38.537719  9268 tablet_bootstrap.cc:492] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:38.545243  9272 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:38.534716  9223 raft_consensus.cc:3060] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.545719  9272 ts_tablet_manager.cc:1403] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.095s	user 0.025s	sys 0.027s
I20260709 12:21:38.548401  9186 leader_election.cc:304] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:38.549686  9350 raft_consensus.cc:3060] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.554709  9399 raft_consensus.cc:2749] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.540518  9397 consensus_queue.cc:1048] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:21:38.558099  9350 raft_consensus.cc:2749] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.559937  9268 tablet_bootstrap.cc:654] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.532477  9113 leader_election.cc:304] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:38.570034  9270 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:38.568005  9341 raft_consensus.cc:2749] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.561231  9322 raft_consensus.cc:697] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
W20260709 12:21:38.578691  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 79611.4us
W20260709 12:21:38.580534  9113 leader_election.cc:343] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.580997  9113 leader_election.cc:304] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.581189  9297 consensus_queue.cc:1048] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.581789  9341 raft_consensus.cc:2749] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.582924  9270 tablet_bootstrap.cc:654] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.590798  9272 raft_consensus.cc:359] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.593331  9272 raft_consensus.cc:385] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.593724  9272 raft_consensus.cc:740] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.594555  9272 consensus_queue.cc:260] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.596468  9272 ts_tablet_manager.cc:1434] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 12:21:38.599136  9272 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.605494  9272 tablet_bootstrap.cc:654] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.592963  9269 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:38.619810  9269 ts_tablet_manager.cc:1403] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.169s	user 0.013s	sys 0.007s
I20260709 12:21:38.590950  9322 consensus_queue.cc:237] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.588804  9416 raft_consensus.cc:515] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.592291  9297 consensus_queue.cc:1048] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.632421  9269 raft_consensus.cc:359] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:38.633742  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:38.636413  9269 raft_consensus.cc:385] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.638329  9269 raft_consensus.cc:740] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.639071  9269 consensus_queue.cc:260] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.648747  9394 consensus_queue.cc:1048] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:38.654062  9416 leader_election.cc:290] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:38.644263  9316 raft_consensus.cc:493] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.655570  9316 raft_consensus.cc:515] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.661999  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:38.662546  9268 tablet_bootstrap.cc:492] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:38.663086  9268 ts_tablet_manager.cc:1403] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.126s	user 0.036s	sys 0.031s
I20260709 12:21:38.663980  9316 leader_election.cc:290] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.668541  8996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:38.669225  8996 raft_consensus.cc:3060] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.669348  9394 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.670118  9394 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.665781  9268 raft_consensus.cc:359] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.589241  9223 raft_consensus.cc:2468] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:38.671550  9268 raft_consensus.cc:385] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.662711  9149 raft_consensus.cc:3060] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.677134  9268 raft_consensus.cc:740] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.679381  9268 consensus_queue.cc:260] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.675884  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.670789  9416 consensus_queue.cc:1048] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:21:38.689355  9268 ts_tablet_manager.cc:1434] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:21:38.692023  9268 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:38.699013  9268 tablet_bootstrap.cc:654] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.719909  9269 ts_tablet_manager.cc:1434] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.100s	user 0.016s	sys 0.016s
I20260709 12:21:38.726804  9394 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
W20260709 12:21:38.710623  8963 leader_election.cc:343] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.729426  9490 consensus_queue.cc:1048] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.731657  9350 raft_consensus.cc:493] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.756027  9269 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:38.723960  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:38.754626  9286 raft_consensus.cc:2749] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.761399  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.770262  9272 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:38.770845  9272 ts_tablet_manager.cc:1403] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.172s	user 0.016s	sys 0.002s
I20260709 12:21:38.751595  9482 raft_consensus.cc:493] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.781268  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.778825  9272 raft_consensus.cc:359] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.788326  9272 raft_consensus.cc:385] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.788576  9272 raft_consensus.cc:740] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.794394  9270 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:38.794996  9270 ts_tablet_manager.cc:1403] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.225s	user 0.027s	sys 0.019s
W20260709 12:21:38.789423  9113 leader_election.cc:343] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.797791  9272 consensus_queue.cc:260] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:38.792418  8964 leader_election.cc:343] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.800746  8964 leader_election.cc:304] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:38.800868  9272 ts_tablet_manager.cc:1434] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:21:38.802325  9272 tablet_bootstrap.cc:492] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:38.769709  9350 raft_consensus.cc:515] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.807334  9424 raft_consensus.cc:2749] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.808513  9350 leader_election.cc:290] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:38.808712  9149 raft_consensus.cc:2468] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:38.790395  8996 raft_consensus.cc:2468] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:38.840853  9270 raft_consensus.cc:359] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.842031  9270 raft_consensus.cc:385] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:38.842439  9270 raft_consensus.cc:740] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:38.789944  9482 raft_consensus.cc:515] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:38.843995  9270 consensus_queue.cc:260] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.855439  9270 ts_tablet_manager.cc:1434] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.060s	user 0.010s	sys 0.002s
I20260709 12:21:38.850589  9423 consensus_queue.cc:1048] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:21:38.859812  9269 tablet_bootstrap.cc:654] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:38.853196  9112 leader_election.cc:343] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.868551  9481 consensus_queue.cc:1048] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.912693  9112 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.913645  9297 raft_consensus.cc:2749] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:38.870198  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25135f5b10d44185b9d6be0f2fb5a02c" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.871127  9187 leader_election.cc:304] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:38.880404  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25135f5b10d44185b9d6be0f2fb5a02c" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.914513  9000 raft_consensus.cc:2468] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:38.914747  9146 raft_consensus.cc:2468] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:38.888393  9322 raft_consensus.cc:493] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.892112  9482 leader_election.cc:290] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:38.917027  9322 raft_consensus.cc:515] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.893723  9316 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.919224  9322 leader_election.cc:290] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:38.919642  9316 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.921636  9316 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:38.895009  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81bd1db68c34b918042fd1f8a0d980c" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.929800  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45f96b48cfe14f749d63e8cb99981b8b" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.900023  8898 catalog_manager.cc:5697] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.911073  9270 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:38.911953  9494 log.cc:927] Time spent T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:21:38.913764  9112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 61137.7us
I20260709 12:21:38.938344  9490 raft_consensus.cc:2804] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:38.938902  9490 raft_consensus.cc:697] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
W20260709 12:21:38.939435  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 60413.8us
I20260709 12:21:38.940125  9272 tablet_bootstrap.cc:654] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:38.939743  9490 consensus_queue.cc:237] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:38.942430  9039 leader_election.cc:343] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.942684  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.943742  9341 raft_consensus.cc:493] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.944314  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:38.944281  9341 raft_consensus.cc:515] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:38.945019  9074 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:21:38.945922  9495 consensus_queue.cc:1048] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.946914  9341 leader_election.cc:290] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:38.962028  8963 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
W20260709 12:21:38.962452  9038 leader_election.cc:343] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.962080  8996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:38.963436  9497 consensus_queue.cc:1048] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:38.936496  9187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 64931.8us
W20260709 12:21:38.970670  9113 leader_election.cc:343] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.969357  9316 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:38.977632  9316 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.977978  9316 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.979395  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45f96b48cfe14f749d63e8cb99981b8b" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:38.980366  9146 raft_consensus.cc:2468] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:38.984778  9499 raft_consensus.cc:493] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.985268  9499 raft_consensus.cc:515] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:38.988849  9038 leader_election.cc:304] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:38.990060  9322 raft_consensus.cc:2804] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:38.990494  9322 raft_consensus.cc:493] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.990833  9322 raft_consensus.cc:3060] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:38.993158  9075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:38.998184  9112 leader_election.cc:343] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:38.975517  9186 leader_election.cc:304] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:38.998572  9268 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:39.001430  9497 raft_consensus.cc:2804] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.006585  9268 ts_tablet_manager.cc:1403] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.315s	user 0.025s	sys 0.010s
I20260709 12:21:38.998674  9112 leader_election.cc:304] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:39.008801  9341 raft_consensus.cc:2749] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.010227  9270 tablet_bootstrap.cc:654] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.010018  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:39.010352  9497 raft_consensus.cc:493] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.010813  9497 raft_consensus.cc:3060] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:39.011490  9113 leader_election.cc:343] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.024231  9268 raft_consensus.cc:359] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.025967  9268 raft_consensus.cc:385] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.026378  9268 raft_consensus.cc:740] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.032982  9268 consensus_queue.cc:260] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.040642  9269 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:39.041378  9269 ts_tablet_manager.cc:1403] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.291s	user 0.027s	sys 0.031s
I20260709 12:21:39.049122  9394 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:39.049943  9394 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.050537  9268 ts_tablet_manager.cc:1434] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:21:39.054664  9272 tablet_bootstrap.cc:492] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:39.055389  9272 ts_tablet_manager.cc:1403] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.253s	user 0.017s	sys 0.000s
I20260709 12:21:39.005136  9499 leader_election.cc:290] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:39.058440  9269 raft_consensus.cc:359] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.059563  9269 raft_consensus.cc:385] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.059924  9269 raft_consensus.cc:740] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.122598  9268 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:39.123593  9483 consensus_queue.cc:1048] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:39.131247  9322 raft_consensus.cc:515] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.094231  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:39.137885  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 45550us
I20260709 12:21:39.148216  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81bd1db68c34b918042fd1f8a0d980c" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:39.148450  9269 consensus_queue.cc:260] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.152118  9272 raft_consensus.cc:359] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.152889  9272 raft_consensus.cc:385] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.153154  9272 raft_consensus.cc:740] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.157281  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45f96b48cfe14f749d63e8cb99981b8b" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:39.157920  9149 raft_consensus.cc:3060] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.153836  9272 consensus_queue.cc:260] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.170872  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45f96b48cfe14f749d63e8cb99981b8b" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:39.171339  9149 raft_consensus.cc:2468] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
W20260709 12:21:39.172155  9112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 76680us
I20260709 12:21:39.134092  9322 leader_election.cc:290] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:39.178817  9269 ts_tablet_manager.cc:1434] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.137s	user 0.010s	sys 0.001s
I20260709 12:21:39.179663  9038 leader_election.cc:304] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:39.197592  9272 ts_tablet_manager.cc:1434] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.142s	user 0.007s	sys 0.003s
W20260709 12:21:39.200145  9464 log.cc:927] Time spent T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:21:39.202452  9269 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:39.204928  9272 tablet_bootstrap.cc:492] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
W20260709 12:21:39.206595  9039 leader_election.cc:343] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.210740  9350 raft_consensus.cc:493] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.211236  9350 raft_consensus.cc:515] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
W20260709 12:21:39.218429  9113 leader_election.cc:343] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.219103  9113 leader_election.cc:304] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:39.226426  9341 raft_consensus.cc:2749] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.215895  9322 raft_consensus.cc:2804] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.215492  9350 leader_election.cc:290] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:39.233853  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:39.235337  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 43799.4us
I20260709 12:21:39.236124  9316 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.237144  9366 consensus_queue.cc:1048] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.239157  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e11b03c5914827946641c953501ec8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:39.239916  9039 leader_election.cc:343] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.241230  8998 raft_consensus.cc:2368] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 for earlier term 2. Current term is 3.
I20260709 12:21:39.245952  9113 leader_election.cc:400] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461) with higher term. Message: Invalid argument: T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 for earlier term 2. Current term is 3.
I20260709 12:21:39.246305  9113 leader_election.cc:403] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:21:39.248497  9508 log.cc:927] Time spent T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:21:39.250519  9499 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.251812  9270 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:39.252323  9270 ts_tablet_manager.cc:1403] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.341s	user 0.056s	sys 0.054s
I20260709 12:21:39.255283  9270 raft_consensus.cc:359] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.256031  9270 raft_consensus.cc:385] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.256352  9270 raft_consensus.cc:740] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.257086  9270 consensus_queue.cc:260] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.260259  9270 ts_tablet_manager.cc:1434] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:39.261300  9270 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:39.259006  8894 catalog_manager.cc:5697] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.264484  9316 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:39.259107  9322 raft_consensus.cc:697] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:39.265868  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:39.264822  9394 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:39.264568  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:39.267338  9072 raft_consensus.cc:2393] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in current term 2: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:39.269029  9499 raft_consensus.cc:2749] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461) with higher term. Message: Invalid argument: T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 for earlier term 2. Current term is 3.
W20260709 12:21:39.269570  9186 leader_election.cc:343] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.271237  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e11b03c5914827946641c953501ec8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:39.271916  8998 raft_consensus.cc:2468] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:39.273650  9322 consensus_queue.cc:237] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.260610  9039 leader_election.cc:304] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:39.258621  9497 raft_consensus.cc:515] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.276458  9366 raft_consensus.cc:2749] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.276954  9145 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:21:39.277676  9268 tablet_bootstrap.cc:654] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.278882  8963 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:39.281329  9424 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:39.280546  9187 leader_election.cc:304] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:39.289777  9366 consensus_queue.cc:1048] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.290884  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25135f5b10d44185b9d6be0f2fb5a02c" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:39.291503  8998 raft_consensus.cc:3060] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.283134  9497 leader_election.cc:290] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:39.287343  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:39.288494  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25135f5b10d44185b9d6be0f2fb5a02c" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:39.297102  9270 tablet_bootstrap.cc:654] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.298264  9149 raft_consensus.cc:3060] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.301241  9269 tablet_bootstrap.cc:654] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.303419  9272 tablet_bootstrap.cc:654] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.310382  9072 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.296856  9490 raft_consensus.cc:2804] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.313400  9490 raft_consensus.cc:493] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.314020  9490 raft_consensus.cc:3060] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.297151  9424 raft_consensus.cc:697] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:39.315191  9424 consensus_queue.cc:237] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.317188  8998 raft_consensus.cc:2468] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:39.340397  9187 leader_election.cc:304] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:39.342344  9350 raft_consensus.cc:2804] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.346232  9350 raft_consensus.cc:697] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:39.347229  9350 consensus_queue.cc:237] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.345149  8894 catalog_manager.cc:5697] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.357287  9268 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:39.357802  9268 ts_tablet_manager.cc:1403] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.236s	user 0.014s	sys 0.000s
I20260709 12:21:39.360594  9268 raft_consensus.cc:359] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.367249  9268 raft_consensus.cc:385] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.367736  9268 raft_consensus.cc:740] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.372854  9268 consensus_queue.cc:260] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.371084  9149 raft_consensus.cc:2468] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:39.396646  9072 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:21:39.400658  9268 ts_tablet_manager.cc:1434] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.003s
I20260709 12:21:39.419713  9268 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:39.426055  9268 tablet_bootstrap.cc:654] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.416307  9272 tablet_bootstrap.cc:492] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:39.431882  9272 ts_tablet_manager.cc:1403] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.233s	user 0.011s	sys 0.009s
I20260709 12:21:39.434535  9272 raft_consensus.cc:359] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.403642  9490 raft_consensus.cc:515] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.430794  9350 consensus_queue.cc:1048] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.435179  9272 raft_consensus.cc:385] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.426908  9269 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:39.445243  9272 raft_consensus.cc:740] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.445569  9269 ts_tablet_manager.cc:1403] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.249s	user 0.027s	sys 0.042s
I20260709 12:21:39.446105  9272 consensus_queue.cc:260] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.441303  9341 raft_consensus.cc:493] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.448459  9269 raft_consensus.cc:359] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.449086  9341 raft_consensus.cc:515] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.454272  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81bd1db68c34b918042fd1f8a0d980c" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:39.454958  9074 raft_consensus.cc:2468] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:39.456632  9112 leader_election.cc:304] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:39.458300  9499 raft_consensus.cc:2804] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.458823  9499 raft_consensus.cc:493] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.459226  9499 raft_consensus.cc:3060] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.462764  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81bd1db68c34b918042fd1f8a0d980c" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:39.464599  9113 leader_election.cc:343] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.448475  9272 ts_tablet_manager.cc:1434] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:21:39.465960  9497 consensus_queue.cc:1048] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.466445  9341 leader_election.cc:290] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:39.467748  9272 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:39.449215  9269 raft_consensus.cc:385] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.470340  9269 raft_consensus.cc:740] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.471230  9269 consensus_queue.cc:260] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.472790  9269 ts_tablet_manager.cc:1434] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.002s
I20260709 12:21:39.473649  9269 tablet_bootstrap.cc:492] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:39.476691  9497 raft_consensus.cc:493] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.477217  9497 raft_consensus.cc:515] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.479766  9497 leader_election.cc:290] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:39.481163  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:39.481086  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:39.482635  8999 raft_consensus.cc:2468] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
W20260709 12:21:39.483701  9186 leader_election.cc:343] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.484272  9187 leader_election.cc:304] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:39.486442  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e11b03c5914827946641c953501ec8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:39.486893  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e11b03c5914827946641c953501ec8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:39.487555  8999 raft_consensus.cc:3060] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.487653  9350 raft_consensus.cc:2804] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:39.488046  9186 leader_election.cc:343] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.488085  9350 raft_consensus.cc:493] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.489090  9350 raft_consensus.cc:3060] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.494426  9269 tablet_bootstrap.cc:654] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.497041  9350 raft_consensus.cc:515] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.500468  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:39.500756  9350 leader_election.cc:290] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:39.501940  9186 leader_election.cc:343] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.504997  9270 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:39.505373  9490 leader_election.cc:290] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:39.505520  9270 ts_tablet_manager.cc:1403] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.244s	user 0.017s	sys 0.018s
I20260709 12:21:39.501935  8996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:39.507738  8996 raft_consensus.cc:3060] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.508045  8999 raft_consensus.cc:2468] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:39.510049  9187 leader_election.cc:304] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:39.510232  8898 catalog_manager.cc:5697] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.512184  9499 raft_consensus.cc:515] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.513801  9490 raft_consensus.cc:2804] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.514389  9490 raft_consensus.cc:697] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:39.515529  9483 raft_consensus.cc:493] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.516072  9483 raft_consensus.cc:515] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.518745  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81bd1db68c34b918042fd1f8a0d980c" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:39.519445  9074 raft_consensus.cc:3060] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.519316  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:39.519732  9483 leader_election.cc:290] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:39.520048  9145 raft_consensus.cc:2468] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:39.520515  9490 consensus_queue.cc:237] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.521476  9038 leader_election.cc:304] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:39.522684  9482 raft_consensus.cc:2804] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.523062  9482 raft_consensus.cc:493] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.523404  9482 raft_consensus.cc:3060] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.518668  8996 raft_consensus.cc:2468] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:39.532488  9187 leader_election.cc:304] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:39.533614  9490 raft_consensus.cc:2804] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.534380  9490 raft_consensus.cc:697] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:39.535243  9490 consensus_queue.cc:237] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.537632  9270 raft_consensus.cc:359] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.540632  9270 raft_consensus.cc:385] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.541299  9270 raft_consensus.cc:740] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.542379  9270 consensus_queue.cc:260] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.544893  9366 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.544687  8895 catalog_manager.cc:5697] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.545457  9366 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.547858  9366 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:39.547991  9270 ts_tablet_manager.cc:1434] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:21:39.549867  8996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:39.550515  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:39.551326  9039 leader_election.cc:343] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.551805  9269 tablet_bootstrap.cc:492] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
W20260709 12:21:39.552063  9039 leader_election.cc:343] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.552274  9269 ts_tablet_manager.cc:1403] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.079s	user 0.008s	sys 0.004s
I20260709 12:21:39.552558  9039 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:39.553825  9366 raft_consensus.cc:2749] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.555047  9269 raft_consensus.cc:359] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.555594  9270 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:39.445226  8896 catalog_manager.cc:5697] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.560799  9482 raft_consensus.cc:515] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.567018  9074 raft_consensus.cc:2468] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:39.569614  9482 leader_election.cc:290] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:39.569738  9112 leader_election.cc:304] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:39.571617  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:39.572227  9145 raft_consensus.cc:3060] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.555737  9269 raft_consensus.cc:385] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.573925  9269 raft_consensus.cc:740] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.575034  8996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:39.574862  9269 consensus_queue.cc:260] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:39.576604  9039 leader_election.cc:343] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.571616  9341 raft_consensus.cc:2804] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.582882  9272 tablet_bootstrap.cc:654] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.585280  9268 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:39.602706  9268 ts_tablet_manager.cc:1403] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.183s	user 0.010s	sys 0.001s
I20260709 12:21:39.603709  9341 raft_consensus.cc:697] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:39.585434  8898 catalog_manager.cc:5697] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.596217  9269 ts_tablet_manager.cc:1434] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:21:39.597302  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81bd1db68c34b918042fd1f8a0d980c" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:39.609203  9219 raft_consensus.cc:3060] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.613744  9341 consensus_queue.cc:237] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.623066  9145 raft_consensus.cc:2468] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:39.635421  9038 leader_election.cc:304] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:39.636637  9366 raft_consensus.cc:2804] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:39.637141  9366 raft_consensus.cc:697] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:39.637998  9366 consensus_queue.cc:237] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.599365  9499 leader_election.cc:290] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:39.651536  9269 tablet_bootstrap.cc:492] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:39.605643  9268 raft_consensus.cc:359] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.658526  9268 raft_consensus.cc:385] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.658938  9268 raft_consensus.cc:740] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.585835  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:39.660569  9039 leader_election.cc:343] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.660332  9268 consensus_queue.cc:260] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.663692  9219 raft_consensus.cc:2468] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:39.681382  9268 ts_tablet_manager.cc:1434] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.078s	user 0.004s	sys 0.002s
I20260709 12:21:39.682758  9268 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:39.693670  9272 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:39.692910  8896 catalog_manager.cc:5697] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.694149  9270 tablet_bootstrap.cc:654] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.694234  9272 ts_tablet_manager.cc:1403] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.228s	user 0.007s	sys 0.004s
I20260709 12:21:39.697321  9272 raft_consensus.cc:359] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.699724  9272 raft_consensus.cc:385] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.700117  9272 raft_consensus.cc:740] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.699326  9269 tablet_bootstrap.cc:654] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.707330  8895 catalog_manager.cc:5697] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:39.710143  9270 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:39.714650  9270 ts_tablet_manager.cc:1403] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.165s	user 0.021s	sys 0.000s
I20260709 12:21:39.717533  9270 raft_consensus.cc:359] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.717641  9272 consensus_queue.cc:260] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.718267  9270 raft_consensus.cc:385] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.720424  9270 raft_consensus.cc:740] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.724728  9270 consensus_queue.cc:260] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.721730  9272 ts_tablet_manager.cc:1434] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:21:39.730276  8999 raft_consensus.cc:3060] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.720031  9268 tablet_bootstrap.cc:654] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.734149  9272 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:39.743274  9366 consensus_queue.cc:1048] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.744966  9270 ts_tablet_manager.cc:1434] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 12:21:39.749064  9269 tablet_bootstrap.cc:492] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:39.750567  9497 consensus_queue.cc:1048] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:21:39.760957  9269 ts_tablet_manager.cc:1403] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.002s
I20260709 12:21:39.764573  9269 raft_consensus.cc:359] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.765357  9269 raft_consensus.cc:385] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.753283  9341 raft_consensus.cc:493] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.766911  9341 raft_consensus.cc:515] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.769029  9341 leader_election.cc:290] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:39.769537  9272 tablet_bootstrap.cc:654] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.769757  8996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:39.770641  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:39.770882  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:21:39.771461  9113 leader_election.cc:343] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:39.772043  9112 leader_election.cc:343] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.772488  9112 leader_election.cc:304] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:39.773537  9341 raft_consensus.cc:2749] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.752476  9270 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:39.777326  9366 consensus_queue.cc:1048] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:39.766359  9269 raft_consensus.cc:740] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.783495  9269 consensus_queue.cc:260] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.788336  9270 tablet_bootstrap.cc:654] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.756124  9268 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:39.790798  9268 ts_tablet_manager.cc:1403] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.108s	user 0.007s	sys 0.003s
I20260709 12:21:39.793392  9269 ts_tablet_manager.cc:1434] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:21:39.794428  9269 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:39.795352  9268 raft_consensus.cc:359] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.805099  9146 raft_consensus.cc:1275] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:39.809564  9316 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:39.796041  9268 raft_consensus.cc:385] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.811800  9268 raft_consensus.cc:740] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.812503  9268 consensus_queue.cc:260] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.828742  9268 ts_tablet_manager.cc:1434] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.002s
I20260709 12:21:39.831154  9268 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:39.844904  9490 raft_consensus.cc:493] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.845530  9490 raft_consensus.cc:515] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.861562  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:39.864490  9186 leader_election.cc:343] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.864761  9270 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:39.865468  9270 ts_tablet_manager.cc:1403] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.006s
I20260709 12:21:39.868266  9268 tablet_bootstrap.cc:654] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.848026  9490 leader_election.cc:290] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:39.871325  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:39.880440  9149 raft_consensus.cc:2468] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:39.888821  9269 tablet_bootstrap.cc:654] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:39.888892  9270 raft_consensus.cc:359] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.889842  9270 raft_consensus.cc:385] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.890170  9270 raft_consensus.cc:740] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.887653  9481 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.891463  9270 consensus_queue.cc:260] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:39.891844  9481 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.891968  9186 leader_election.cc:304] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:39.895889  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:39.898427  9350 raft_consensus.cc:2804] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:39.918529  9350 raft_consensus.cc:493] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:39.899120  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:39.919094  9350 raft_consensus.cc:3060] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:39.907699  9272 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:39.910499  9074 raft_consensus.cc:1275] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:39.923977  9272 ts_tablet_manager.cc:1403] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.190s	user 0.053s	sys 0.050s
I20260709 12:21:39.894037  9481 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:39.923883  9270 ts_tablet_manager.cc:1434] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.002s
I20260709 12:21:39.927443  9270 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:39.931246  9341 raft_consensus.cc:493] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.931806  9341 raft_consensus.cc:515] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:39.932289  9039 leader_election.cc:343] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.934001  9341 leader_election.cc:290] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
W20260709 12:21:39.936168  9039 leader_election.cc:343] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.936905  9039 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:39.937104  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:39.938750  9481 raft_consensus.cc:2749] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.936121  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:39.936085  9424 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:39.941918  9112 leader_election.cc:343] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:39.944053  9113 leader_election.cc:343] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:39.944512  9113 leader_election.cc:304] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:39.945995  9499 raft_consensus.cc:2749] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:39.946115  9495 consensus_queue.cc:1048] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:39.947527  9490 consensus_queue.cc:1048] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:39.958285  9272 raft_consensus.cc:359] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.961213  9272 raft_consensus.cc:385] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:39.961946  9272 raft_consensus.cc:740] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:39.962682  9481 raft_consensus.cc:493] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.963284  9481 raft_consensus.cc:515] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:39.963477  9272 consensus_queue.cc:260] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:39.966042  9481 leader_election.cc:290] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:39.968578  9272 ts_tablet_manager.cc:1434] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.001s
I20260709 12:21:39.972167  9272 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:39.957633  9350 raft_consensus.cc:515] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:39.990772  9490 consensus_queue.cc:1048] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:39.994230  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e11b03c5914827946641c953501ec8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:39.995441  9272 tablet_bootstrap.cc:654] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.001443  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44e11b03c5914827946641c953501ec8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.003192  9269 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:40.003748  9269 ts_tablet_manager.cc:1403] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.210s	user 0.020s	sys 0.002s
I20260709 12:21:40.008922  9039 leader_election.cc:304] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.008782  9269 raft_consensus.cc:359] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.009600  9269 raft_consensus.cc:385] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.009806  9366 raft_consensus.cc:2749] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.009938  9269 raft_consensus.cc:740] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.010684  9269 consensus_queue.cc:260] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.009622  9424 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.012661  9424 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.013492  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:40.014478  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:40.014837  9424 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.015134  9145 raft_consensus.cc:3060] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.015548  9269 ts_tablet_manager.cc:1434] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:40.016855  9269 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
W20260709 12:21:40.017590  9186 leader_election.cc:343] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.019057  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:40.019716  9146 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:40.022634  9269 tablet_bootstrap.cc:654] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.029050  8963 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:40.030000  9424 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.030539  9424 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.030757  9270 tablet_bootstrap.cc:654] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.031081  9424 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.030833  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:40.023190  9350 leader_election.cc:290] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
W20260709 12:21:40.036134  8963 leader_election.cc:343] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.040469  9268 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:40.041200  9268 ts_tablet_manager.cc:1403] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.210s	user 0.032s	sys 0.048s
I20260709 12:21:40.043639  9145 raft_consensus.cc:2468] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:40.045049  9186 leader_election.cc:304] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7
I20260709 12:21:40.045982  9268 raft_consensus.cc:359] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.049225  9268 raft_consensus.cc:385] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.049584  9268 raft_consensus.cc:740] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.051927  9268 consensus_queue.cc:260] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.066246  9268 ts_tablet_manager.cc:1434] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 12:21:40.066751  9316 raft_consensus.cc:493] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.067317  9268 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:40.067458  9316 raft_consensus.cc:515] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.069922  9316 leader_election.cc:290] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.050491  9350 raft_consensus.cc:2804] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.081875  9350 raft_consensus.cc:697] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:40.072257  9145 raft_consensus.cc:3060] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.075860  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:40.090618  9149 raft_consensus.cc:2393] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:40.078534  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7ab6e4c736485fbc461ccb19ea580a" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:40.080215  9268 tablet_bootstrap.cc:654] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.083122  9350 consensus_queue.cc:237] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.100342  8963 leader_election.cc:304] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:40.125448  9341 raft_consensus.cc:493] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.125583  9270 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:40.126296  9270 ts_tablet_manager.cc:1403] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.199s	user 0.017s	sys 0.052s
I20260709 12:21:40.126062  9341 raft_consensus.cc:515] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:40.129065  9270 raft_consensus.cc:359] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.129756  9270 raft_consensus.cc:385] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.130015  9270 raft_consensus.cc:740] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.130107  9341 leader_election.cc:290] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:40.130738  9270 consensus_queue.cc:260] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.131639  9268 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:40.132262  9268 ts_tablet_manager.cc:1403] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.004s
I20260709 12:21:40.135902  9268 raft_consensus.cc:359] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.138180  9072 raft_consensus.cc:3060] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.144538  9394 consensus_queue.cc:1048] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.139905  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:40.148278  9497 consensus_queue.cc:1048] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.161824  9416 consensus_queue.cc:1048] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.166747  9272 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:40.167337  9272 ts_tablet_manager.cc:1403] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.195s	user 0.021s	sys 0.011s
I20260709 12:21:40.170674  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.173425  9316 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.173684  9483 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.174160  9483 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.176252  9483 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.177927  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:40.178558  9146 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:40.178833  9341 consensus_queue.cc:1048] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.180032  9038 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:40.173971  9316 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.181449  9483 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.181877  9483 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.182292  9483 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.183133  9316 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:40.177649  9424 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.187335  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:40.188031  9073 raft_consensus.cc:2468] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:40.189692  9424 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.189584  8963 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:40.190332  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:40.191254  9113 leader_election.cc:304] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.191625  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:40.192220  9146 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.199584  9272 raft_consensus.cc:359] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.200344  9272 raft_consensus.cc:385] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.201740  9146 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:40.204874  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:40.204874  8963 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
W20260709 12:21:40.208124  8964 leader_election.cc:343] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.213721  9268 raft_consensus.cc:385] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.214136  9268 raft_consensus.cc:740] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.215080  9316 raft_consensus.cc:2804] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.215507  9316 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.215854  9316 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.215901  9424 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.216566  9424 raft_consensus.cc:697] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:40.217463  9424 consensus_queue.cc:237] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.123214  9549 raft_consensus.cc:3060] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.230027  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.200628  9272 raft_consensus.cc:740] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.232499  9272 consensus_queue.cc:260] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.235628  9272 ts_tablet_manager.cc:1434] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 12:21:40.236773  9272 tablet_bootstrap.cc:492] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:40.214867  9268 consensus_queue.cc:260] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.259037  9268 ts_tablet_manager.cc:1434] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.126s	user 0.006s	sys 0.000s
I20260709 12:21:40.274590  8896 catalog_manager.cc:5697] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:40.279814  9268 tablet_bootstrap.cc:492] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:40.319496  9270 ts_tablet_manager.cc:1434] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.193s	user 0.035s	sys 0.098s
I20260709 12:21:40.320969  9270 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:40.329391  9270 tablet_bootstrap.cc:654] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.329635  9549 raft_consensus.cc:2749] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:40.266264  8964 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 44317.7us
I20260709 12:21:40.317480  9350 raft_consensus.cc:493] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.356596  9394 raft_consensus.cc:493] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.350193  9350 raft_consensus.cc:515] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.380883  9394 raft_consensus.cc:515] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:40.394459  9394 leader_election.cc:290] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:40.396648  9554 log.cc:927] Time spent T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:21:40.405164  9272 tablet_bootstrap.cc:654] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.418294  9561 raft_consensus.cc:493] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.418900  9561 raft_consensus.cc:515] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.429661  9483 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:40.391896  9551 log.cc:927] Time spent T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:21:40.431792  9561 leader_election.cc:290] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.462607  9269 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:40.463203  9269 ts_tablet_manager.cc:1403] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.447s	user 0.020s	sys 0.026s
I20260709 12:21:40.447744  9341 raft_consensus.cc:2749] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:40.461808  9073 raft_consensus.cc:2393] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:40.439867  9316 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.475618  9000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.474206  9268 tablet_bootstrap.cc:654] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.489239  9270 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:40.489873  9270 ts_tablet_manager.cc:1403] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.169s	user 0.018s	sys 0.010s
I20260709 12:21:40.490695  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:40.491393  9146 raft_consensus.cc:2468] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:40.493441  9270 raft_consensus.cc:359] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.494107  9350 leader_election.cc:290] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:40.497094  9316 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:40.499452  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:40.501835  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:40.547503  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.503824  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:40.505795  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.511183  9569 consensus_queue.cc:1048] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.551095  9270 raft_consensus.cc:385] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:40.522737  9113 leader_election.cc:343] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.523615  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:40.529945  9269 raft_consensus.cc:359] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:40.554054  9269 raft_consensus.cc:385] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.554595  9269 raft_consensus.cc:740] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
W20260709 12:21:40.556910  9039 leader_election.cc:343] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.552510  9270 raft_consensus.cc:740] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.558625  9270 consensus_queue.cc:260] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
W20260709 12:21:40.558986  9186 leader_election.cc:343] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:40.559813  8964 leader_election.cc:343] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.561192  9270 ts_tablet_manager.cc:1434] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.000s
I20260709 12:21:40.555694  9269 consensus_queue.cc:260] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:40.563252  9223 raft_consensus.cc:2468] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:40.565557  9269 ts_tablet_manager.cc:1434] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
I20260709 12:21:40.545749  9268 tablet_bootstrap.cc:492] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:40.519490  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:40.567255  9268 ts_tablet_manager.cc:1403] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.288s	user 0.011s	sys 0.011s
I20260709 12:21:40.569219  9350 consensus_queue.cc:1048] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.531132  9483 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.575433  9269 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:40.576696  9270 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:40.574568  9072 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.580335  9268 raft_consensus.cc:359] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.582711  9268 raft_consensus.cc:385] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.583120  9268 raft_consensus.cc:740] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.584018  9268 consensus_queue.cc:260] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:40.578207  9187 leader_election.cc:343] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.605036  9187 leader_election.cc:304] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.616986  9350 raft_consensus.cc:2749] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:40.602029  9557 log.cc:927] Time spent T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: Append to log took a long time: real 0.161s	user 0.000s	sys 0.004s
I20260709 12:21:40.627697  9113 leader_election.cc:304] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.629423  9566 raft_consensus.cc:2804] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.629873  9566 raft_consensus.cc:493] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:40.581408  9038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 107682us
I20260709 12:21:40.630252  9566 raft_consensus.cc:3060] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.659600  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:40.660760  9149 raft_consensus.cc:2393] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 1: Already voted for candidate e47ee72c4e114aef9818c7c324ed1938 in this term.
I20260709 12:21:40.664397  9268 ts_tablet_manager.cc:1434] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.097s	user 0.009s	sys 0.002s
I20260709 12:21:40.672112  9268 tablet_bootstrap.cc:492] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:40.666841  9341 raft_consensus.cc:493] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.672947  9341 raft_consensus.cc:515] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.676977  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:40.677716  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:40.679281  9113 leader_election.cc:343] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:40.680053  9112 leader_election.cc:343] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.680644  9112 leader_election.cc:304] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.681731  9499 raft_consensus.cc:2749] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.682132  9341 leader_election.cc:290] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:40.683112  9566 raft_consensus.cc:515] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:40.663205  9038 leader_election.cc:304] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.666150  9269 tablet_bootstrap.cc:654] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.689792  9573 raft_consensus.cc:2804] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.690204  9573 raft_consensus.cc:493] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.690804  9573 raft_consensus.cc:3060] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.691315  9483 consensus_queue.cc:1048] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.693408  9038 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.694187  9483 raft_consensus.cc:2749] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.698374  9573 raft_consensus.cc:515] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.701434  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
W20260709 12:21:40.702787  9039 leader_election.cc:343] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.705652  9573 leader_election.cc:290] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:40.705768  9270 tablet_bootstrap.cc:654] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.674031  9072 raft_consensus.cc:2468] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:40.726985  9268 tablet_bootstrap.cc:654] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.730651  8963 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:40.731875  9316 raft_consensus.cc:2804] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.732339  9316 raft_consensus.cc:697] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:40.733168  9316 consensus_queue.cc:237] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.736567  9566 leader_election.cc:290] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:40.739511  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:40.730752  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:40.743566  9146 raft_consensus.cc:3060] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.740933  9272 tablet_bootstrap.cc:492] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:40.744391  9272 ts_tablet_manager.cc:1403] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.508s	user 0.021s	sys 0.031s
I20260709 12:21:40.747189  9272 raft_consensus.cc:359] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.748123  9272 raft_consensus.cc:385] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.748443  9272 raft_consensus.cc:740] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.760250  9268 tablet_bootstrap.cc:492] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:40.761045  9268 ts_tablet_manager.cc:1403] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.089s	user 0.013s	sys 0.007s
I20260709 12:21:40.763254  9272 consensus_queue.cc:260] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.766150  9269 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:40.766888  9269 ts_tablet_manager.cc:1403] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.200s	user 0.026s	sys 0.000s
I20260709 12:21:40.770555  9272 ts_tablet_manager.cc:1434] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:21:40.771545  9272 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:40.773545  9146 raft_consensus.cc:2468] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:40.774646  9074 raft_consensus.cc:3060] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:40.775566  9113 leader_election.cc:343] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.782909  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:40.783680  9223 raft_consensus.cc:3060] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.786721  9272 tablet_bootstrap.cc:654] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.788131  9424 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.788839  9424 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.795522  9558 consensus_queue.cc:1048] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:40.797920  9424 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:40.803077  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:40.804219  9073 raft_consensus.cc:2468] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:40.812724  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:40.813851  9269 raft_consensus.cc:359] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.814777  8963 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:40.816955  9424 raft_consensus.cc:2804] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.817407  9424 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.817786  9424 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:40.822713  8964 leader_election.cc:343] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.824333  9038 leader_election.cc:304] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.826658  9561 raft_consensus.cc:2804] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.814826  9269 raft_consensus.cc:385] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.829840  9269 raft_consensus.cc:740] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.822103  9268 raft_consensus.cc:359] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.836954  9268 raft_consensus.cc:385] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.837225  9268 raft_consensus.cc:740] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.831081  9269 consensus_queue.cc:260] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.841082  9316 consensus_queue.cc:1048] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:40.846459  9268 consensus_queue.cc:260] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.838606  9561 raft_consensus.cc:697] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:40.851027  9561 consensus_queue.cc:237] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.857766  9269 ts_tablet_manager.cc:1434] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.000s
I20260709 12:21:40.862896  9269 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:40.868043  9270 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:40.868796  9270 ts_tablet_manager.cc:1403] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.292s	user 0.043s	sys 0.051s
I20260709 12:21:40.885612  9268 ts_tablet_manager.cc:1434] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.124s	user 0.008s	sys 0.000s
I20260709 12:21:40.875207  8896 catalog_manager.cc:5697] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.889151  8896 catalog_manager.cc:5697] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.844997  9424 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.890695  9499 raft_consensus.cc:493] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.901376  9499 raft_consensus.cc:515] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:40.900934  9268 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:40.905088  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:40.907066  9499 leader_election.cc:290] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:40.908257  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:40.910274  9113 leader_election.cc:343] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.912303  9566 raft_consensus.cc:493] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.912999  9566 raft_consensus.cc:515] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:40.915984  9566 leader_election.cc:290] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:40.916335  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:40.875510  9223 raft_consensus.cc:2468] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:40.918237  9268 tablet_bootstrap.cc:654] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:40.929481  9112 leader_election.cc:343] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.930459  9424 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:40.921711  9270 raft_consensus.cc:359] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.933516  9270 raft_consensus.cc:385] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.933804  9270 raft_consensus.cc:740] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.934756  9270 consensus_queue.cc:260] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:40.937604  9270 ts_tablet_manager.cc:1434] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.000s
I20260709 12:21:40.938756  9270 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:40.945384  9272 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:40.946040  9272 ts_tablet_manager.cc:1403] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.175s	user 0.010s	sys 0.008s
I20260709 12:21:40.923271  9490 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:40.922290  9113 leader_election.cc:343] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.959864  9269 tablet_bootstrap.cc:654] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.952358  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:40.957207  9490 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:40.976830  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:40.966260  9072 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.958470  9113 leader_election.cc:304] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:40.970716  9416 consensus_queue.cc:1048] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
W20260709 12:21:40.984342  9585 log.cc:927] Time spent T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:21:40.986656  9566 raft_consensus.cc:2749] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.016963  9270 tablet_bootstrap.cc:654] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.020990  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:41.028921  9268 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:41.029510  9268 ts_tablet_manager.cc:1403] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.129s	user 0.019s	sys 0.004s
I20260709 12:21:41.030373  9490 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:41.034904  9561 raft_consensus.cc:493] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.035176  9272 raft_consensus.cc:359] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.035457  9561 raft_consensus.cc:515] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.036132  9272 raft_consensus.cc:385] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.036280  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:41.018098  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 72279.9us
I20260709 12:21:41.038199  9268 raft_consensus.cc:359] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.038455  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:41.039072  9113 leader_election.cc:304] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:41.039448  9561 leader_election.cc:290] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
W20260709 12:21:41.040333  9113 leader_election.cc:343] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.040810  9113 leader_election.cc:304] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:41.041263  9341 raft_consensus.cc:2804] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.041813  9341 raft_consensus.cc:697] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:41.042091  9564 raft_consensus.cc:2749] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.036496  9272 raft_consensus.cc:740] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.043336  9272 consensus_queue.cc:260] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.043876  9341 consensus_queue.cc:237] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.047564  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86996e32eee445b381d0391def34e7b1" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.050582  9039 leader_election.cc:304] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:41.051509  9366 raft_consensus.cc:2749] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:41.056108  9584 log.cc:927] Time spent T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 12:21:41.058914  8898 catalog_manager.cc:5697] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:41.059747  8964 leader_election.cc:343] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.062072  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:41.062198  9072 raft_consensus.cc:2468] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:41.067225  9073 raft_consensus.cc:2393] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate e47ee72c4e114aef9818c7c324ed1938 in this term.
I20260709 12:21:41.089927  9268 raft_consensus.cc:385] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.068785  9186 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:41.090445  9268 raft_consensus.cc:740] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.115478  9497 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:41.111270  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:41.124548  9272 ts_tablet_manager.cc:1434] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.178s	user 0.006s	sys 0.002s
I20260709 12:21:41.127506  9272 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:41.130715  9268 consensus_queue.cc:260] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:41.117970  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 49278.6us
I20260709 12:21:41.150633  9268 ts_tablet_manager.cc:1434] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.121s	user 0.008s	sys 0.000s
I20260709 12:21:41.151837  9268 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:41.157874  9268 tablet_bootstrap.cc:654] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:41.164528  8963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 169405us
I20260709 12:21:41.167732  9272 tablet_bootstrap.cc:654] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.171541  9270 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:41.172062  9270 ts_tablet_manager.cc:1403] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.234s	user 0.021s	sys 0.003s
I20260709 12:21:41.203464  9497 raft_consensus.cc:2749] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.203858  8963 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:41.219452  9558 raft_consensus.cc:2804] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.220031  9558 raft_consensus.cc:697] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:41.220897  9558 consensus_queue.cc:237] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.229007  9558 consensus_queue.cc:1048] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.208674  9270 raft_consensus.cc:359] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.234285  9564 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.234899  9564 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.235395  8894 catalog_manager.cc:5697] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.236467  9272 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:41.237020  9272 ts_tablet_manager.cc:1403] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.110s	user 0.024s	sys 0.027s
I20260709 12:21:41.237638  9564 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:41.238793  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:41.240083  9112 leader_election.cc:343] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.241063  8896 catalog_manager.cc:5697] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.243432  9270 raft_consensus.cc:385] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.243726  9270 raft_consensus.cc:740] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.244441  9270 consensus_queue.cc:260] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.246840  9270 ts_tablet_manager.cc:1434] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.002s
I20260709 12:21:41.247808  9270 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:41.249706  9272 raft_consensus.cc:359] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.250525  9272 raft_consensus.cc:385] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.250882  9272 raft_consensus.cc:740] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.252012  9272 consensus_queue.cc:260] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.254496  9272 ts_tablet_manager.cc:1434] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:21:41.255345  9269 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:41.255565  9272 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:41.255872  9269 ts_tablet_manager.cc:1403] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.393s	user 0.011s	sys 0.020s
I20260709 12:21:41.263002  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:41.266605  9113 leader_election.cc:343] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.267071  9113 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:41.268277  9564 raft_consensus.cc:2749] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.270064  9272 tablet_bootstrap.cc:654] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.272025  9316 raft_consensus.cc:493] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.272567  9316 raft_consensus.cc:515] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.275565  9316 leader_election.cc:290] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:41.279222  9268 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:41.279735  9268 ts_tablet_manager.cc:1403] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.128s	user 0.014s	sys 0.012s
I20260709 12:21:41.282405  9268 raft_consensus.cc:359] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.282938  9268 raft_consensus.cc:385] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.283171  9268 raft_consensus.cc:740] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.283800  9268 consensus_queue.cc:260] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.285722  9268 ts_tablet_manager.cc:1434] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:21:41.285916  9270 tablet_bootstrap.cc:654] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.290274  9564 raft_consensus.cc:493] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.290815  9564 raft_consensus.cc:515] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.292948  9564 leader_election.cc:290] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.294368  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.296133  9269 raft_consensus.cc:359] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.296860  9269 raft_consensus.cc:385] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:41.297119  9113 leader_election.cc:343] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.297120  9269 raft_consensus.cc:740] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.301549  9269 consensus_queue.cc:260] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.304214  9366 raft_consensus.cc:493] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.304685  9366 raft_consensus.cc:515] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.307305  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.308004  9223 raft_consensus.cc:2468] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:41.308173  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:41.309329  9039 leader_election.cc:304] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
W20260709 12:21:41.310271  9039 leader_election.cc:343] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.310678  9366 leader_election.cc:290] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.310891  9481 raft_consensus.cc:2804] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.311384  9481 raft_consensus.cc:493] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.311720  9481 raft_consensus.cc:3060] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.313557  9270 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:41.314051  9270 ts_tablet_manager.cc:1403] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.066s	user 0.007s	sys 0.002s
I20260709 12:21:41.316457  9269 ts_tablet_manager.cc:1434] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.003s
I20260709 12:21:41.316708  9270 raft_consensus.cc:359] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.317466  9270 raft_consensus.cc:385] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.317724  9270 raft_consensus.cc:740] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.318440  9270 consensus_queue.cc:260] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.320574  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:41.322036  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
W20260709 12:21:41.322184  9113 leader_election.cc:343] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.322705  9113 leader_election.cc:304] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:41.322870  9145 raft_consensus.cc:2393] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:41.325343  9564 raft_consensus.cc:2749] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.326237  9272 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:41.326838  9272 ts_tablet_manager.cc:1403] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.005s
I20260709 12:21:41.326741  9596 raft_consensus.cc:493] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.327345  9596 raft_consensus.cc:515] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.328089  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:41.329306  9596 leader_election.cc:290] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:41.321862  9481 raft_consensus.cc:515] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.330600  9366 raft_consensus.cc:493] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.333142  9366 raft_consensus.cc:515] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.333544  8963 leader_election.cc:304] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:41.334192  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:41.334903  9072 raft_consensus.cc:2468] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:41.335139  9316 raft_consensus.cc:3060] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.332022  9268 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:41.338740  8963 leader_election.cc:304] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:41.339304  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.340154  9558 consensus_queue.cc:1048] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:41.340796  9039 leader_election.cc:343] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.341464  9366 leader_election.cc:290] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:41.341658  9558 raft_consensus.cc:2804] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.342175  9558 raft_consensus.cc:493] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.342602  9558 raft_consensus.cc:3060] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.344233  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:41.344873  9223 raft_consensus.cc:3060] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.349277  9316 raft_consensus.cc:2749] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.322981  9269 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:41.351732  9272 raft_consensus.cc:359] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.352316  9558 raft_consensus.cc:515] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.353294  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:41.363468  9272 raft_consensus.cc:385] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.364148  9272 raft_consensus.cc:740] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.354069  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:41.364785  9149 raft_consensus.cc:2468] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:41.363965  9146 raft_consensus.cc:2468] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:41.332141  9270 ts_tablet_manager.cc:1434] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:21:41.367139  9270 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:41.367312  9481 leader_election.cc:290] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.368427  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:41.369074  8999 raft_consensus.cc:3060] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.369741  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:41.369768  9038 leader_election.cc:304] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:41.371024  9074 raft_consensus.cc:3060] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.375492  9366 raft_consensus.cc:2804] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.377942  9366 raft_consensus.cc:493] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.378409  9366 raft_consensus.cc:3060] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.377938  9074 raft_consensus.cc:2468] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:41.381296  8963 leader_election.cc:304] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:41.379388  9272 consensus_queue.cc:260] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.363444  9223 raft_consensus.cc:2468] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:41.382982  9316 raft_consensus.cc:2804] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.383111  9039 leader_election.cc:304] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:41.383509  9316 raft_consensus.cc:697] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:41.383738  9272 ts_tablet_manager.cc:1434] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 12:21:41.384023  9572 raft_consensus.cc:2804] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.384763  9272 tablet_bootstrap.cc:492] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:41.384647  9572 raft_consensus.cc:697] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:41.384408  9316 consensus_queue.cc:237] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.389488  9270 tablet_bootstrap.cc:654] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.390268  9561 consensus_queue.cc:1048] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.389990  9572 consensus_queue.cc:237] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.391713  9558 leader_election.cc:290] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:41.377362  8999 raft_consensus.cc:2468] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:41.342195  9268 tablet_bootstrap.cc:654] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.419521  9269 tablet_bootstrap.cc:654] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.399631  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:41.431143  8894 catalog_manager.cc:5697] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.426105  9146 raft_consensus.cc:3060] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.449437  9563 consensus_queue.cc:1048] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:41.463388  9268 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:41.464021  9268 ts_tablet_manager.cc:1403] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.003s
I20260709 12:21:41.467202  9272 tablet_bootstrap.cc:654] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.471875  9268 raft_consensus.cc:359] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.464041  9146 raft_consensus.cc:2468] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:41.472702  9268 raft_consensus.cc:385] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.478873  9268 raft_consensus.cc:740] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.483475  9366 raft_consensus.cc:515] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.480499  9268 consensus_queue.cc:260] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.487902  9268 ts_tablet_manager.cc:1434] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:21:41.493105  9268 tablet_bootstrap.cc:492] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:41.511773  8894 catalog_manager.cc:5697] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:41.519183  9269 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:41.521891  9268 tablet_bootstrap.cc:654] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.519753  9269 ts_tablet_manager.cc:1403] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.197s	user 0.014s	sys 0.002s
I20260709 12:21:41.532685  9366 leader_election.cc:290] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:41.538141  9270 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:41.538717  9270 ts_tablet_manager.cc:1403] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.172s	user 0.007s	sys 0.012s
I20260709 12:21:41.539270  9269 raft_consensus.cc:359] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.539703  9272 tablet_bootstrap.cc:492] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:41.540058  9269 raft_consensus.cc:385] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.540319  9272 ts_tablet_manager.cc:1403] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.156s	user 0.027s	sys 0.026s
I20260709 12:21:41.540398  9269 raft_consensus.cc:740] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.541335  9270 raft_consensus.cc:359] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.541607  9269 consensus_queue.cc:260] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.542030  9270 raft_consensus.cc:385] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.542423  9270 raft_consensus.cc:740] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.543156  9270 consensus_queue.cc:260] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.543723  9272 raft_consensus.cc:359] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.544494  9272 raft_consensus.cc:385] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.544840  9272 raft_consensus.cc:740] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.545485  9270 ts_tablet_manager.cc:1434] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:41.545670  9272 consensus_queue.cc:260] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.546489  9270 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:41.546841  9592 raft_consensus.cc:493] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.547309  9592 raft_consensus.cc:515] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.548925  9272 ts_tablet_manager.cc:1434] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:41.549355  9592 leader_election.cc:290] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.550030  9272 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:41.551079  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:41.551733  9149 raft_consensus.cc:2468] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:41.552809  9270 tablet_bootstrap.cc:654] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.553397  9186 leader_election.cc:304] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:41.555262  9416 raft_consensus.cc:2804] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.555646  9416 raft_consensus.cc:493] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.555972  9416 raft_consensus.cc:3060] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.557787  9272 tablet_bootstrap.cc:654] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.569720  9270 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:41.570370  9270 ts_tablet_manager.cc:1403] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.006s
I20260709 12:21:41.571502  9272 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:41.571957  9272 ts_tablet_manager.cc:1403] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 12:21:41.572904  9416 raft_consensus.cc:515] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.573066  9270 raft_consensus.cc:359] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.573799  9270 raft_consensus.cc:385] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.574090  9270 raft_consensus.cc:740] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.574057  9272 raft_consensus.cc:359] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.574911  9272 raft_consensus.cc:385] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.575198  9272 raft_consensus.cc:740] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.575199  9270 consensus_queue.cc:260] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.576576  9416 leader_election.cc:290] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.630861  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:41.674836  9149 raft_consensus.cc:3060] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.631297  9592 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.638985  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:41.677220  9146 raft_consensus.cc:3060] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.642227  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:41.636806  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:41.662415  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9418597cbc1943eda834dfaa445555a8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:41.684090  8998 raft_consensus.cc:2468] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:41.686945  8999 raft_consensus.cc:3060] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.697196  9592 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.700820  9592 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:41.706764  9269 ts_tablet_manager.cc:1434] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.181s	user 0.007s	sys 0.003s
I20260709 12:21:41.709177  9269 tablet_bootstrap.cc:492] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:41.715369  9564 consensus_queue.cc:1048] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:41.576169  9272 consensus_queue.cc:260] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.774957  9146 raft_consensus.cc:2468] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:41.775599  9268 tablet_bootstrap.cc:492] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:41.776576  9268 ts_tablet_manager.cc:1403] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.284s	user 0.018s	sys 0.019s
I20260709 12:21:41.769621  8999 raft_consensus.cc:2468] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:41.780694  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:41.782733  9366 raft_consensus.cc:493] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.783413  9366 raft_consensus.cc:515] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:41.784340  9564 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.785115  9564 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.786393  9366 leader_election.cc:290] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.793012  9272 ts_tablet_manager.cc:1434] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.221s	user 0.005s	sys 0.003s
I20260709 12:21:41.794991  9272 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:41.798101  9270 ts_tablet_manager.cc:1434] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.227s	user 0.006s	sys 0.001s
W20260709 12:21:41.805599  9039 leader_election.cc:343] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:41.806331  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 60516.8us
I20260709 12:21:41.811074  9270 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:41.792949  9112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 108400us
I20260709 12:21:41.816856  9187 leader_election.cc:304] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:41.801285  9268 raft_consensus.cc:359] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.818002  9268 raft_consensus.cc:385] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.818555  9490 raft_consensus.cc:2804] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.821437  9490 raft_consensus.cc:697] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:41.822691  9490 consensus_queue.cc:237] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.820453  9269 tablet_bootstrap.cc:654] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.826719  9268 raft_consensus.cc:740] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.831178  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.833179  9268 consensus_queue.cc:260] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.841715  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:41.846765  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:41.848114  9072 raft_consensus.cc:2393] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate e47ee72c4e114aef9818c7c324ed1938 in this term.
I20260709 12:21:41.822702  9038 leader_election.cc:304] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:41.855566  9563 raft_consensus.cc:2804] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:41.855901  9038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 52717.5us
I20260709 12:21:41.856223  9563 raft_consensus.cc:697] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:41.858366  9223 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.857734  9563 consensus_queue.cc:237] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.870343  9483 raft_consensus.cc:493] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.870946  9483 raft_consensus.cc:515] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.874708  9483 leader_election.cc:290] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:41.877399  9558 raft_consensus.cc:493] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:41.829936  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 60335us
I20260709 12:21:41.879455  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:41.884176  9145 raft_consensus.cc:2468] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:41.874109  9564 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:41.886046  9558 raft_consensus.cc:515] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:41.888825  9558 leader_election.cc:290] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:41.890280  9038 leader_election.cc:304] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:41.891353  9366 raft_consensus.cc:2804] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:41.890504  9039 leader_election.cc:343] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.891952  9366 raft_consensus.cc:493] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.892486  9366 raft_consensus.cc:3060] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.896354  9620 raft_consensus.cc:493] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.897106  9620 raft_consensus.cc:515] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:41.900159  9620 leader_election.cc:290] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:41.901196  9149 raft_consensus.cc:2468] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:41.854874  9186 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
W20260709 12:21:41.902858  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 48281.2us
I20260709 12:21:41.904500  9268 ts_tablet_manager.cc:1434] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.128s	user 0.009s	sys 0.001s
I20260709 12:21:41.904470  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.905714  9218 raft_consensus.cc:2393] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:41.914731  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:41.915979  9558 consensus_queue.cc:1048] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:41.917240  8963 leader_election.cc:304] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:41.919427  9316 raft_consensus.cc:3060] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.925356  9272 tablet_bootstrap.cc:654] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.928367  9620 raft_consensus.cc:493] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.934386  9316 raft_consensus.cc:2749] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.936761  9268 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:41.937901  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:41.940078  9039 leader_election.cc:343] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.928948  9620 raft_consensus.cc:515] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:41.940577  9039 leader_election.cc:304] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:41.949244  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.958792  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.966797  9618 consensus_queue.cc:1048] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:41.967357  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:41.969863  9316 consensus_queue.cc:1048] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.968149  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:41.978551  9596 consensus_queue.cc:1048] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:21:41.979038  9620 leader_election.cc:290] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:41.957767  9483 raft_consensus.cc:2749] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.985391  9268 tablet_bootstrap.cc:654] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:41.987372  9039 leader_election.cc:343] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:41.987579  9563 consensus_queue.cc:1048] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 12:21:41.991354  9112 leader_election.cc:343] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.006552  9366 raft_consensus.cc:515] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.005764  8895 catalog_manager.cc:5697] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.024784  9269 tablet_bootstrap.cc:492] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:42.019389  8894 catalog_manager.cc:5697] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.025398  9592 raft_consensus.cc:2749] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.025662  9270 tablet_bootstrap.cc:654] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.027196  9366 leader_election.cc:290] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:42.029389  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:42.029471  9596 consensus_queue.cc:1048] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:42.032298  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:42.032974  9149 raft_consensus.cc:3060] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.025336  9269 ts_tablet_manager.cc:1403] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.317s	user 0.013s	sys 0.015s
I20260709 12:21:42.052251  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
W20260709 12:21:42.015137  9113 leader_election.cc:343] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:42.058816  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 48004.1us
W20260709 12:21:42.062165  9112 leader_election.cc:343] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.064586  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:42.069443  9269 raft_consensus.cc:359] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.075124  9269 raft_consensus.cc:385] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.075982  9269 raft_consensus.cc:740] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
W20260709 12:21:42.084980  9039 leader_election.cc:343] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.090231  9572 consensus_queue.cc:1048] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:42.100659  9269 consensus_queue.cc:260] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:42.120025  9113 leader_election.cc:343] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.129024  9269 ts_tablet_manager.cc:1434] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.001s
I20260709 12:21:42.123438  9113 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
W20260709 12:21:42.132721  9113 leader_election.cc:343] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.133157  9113 leader_election.cc:304] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
W20260709 12:21:42.135535  9113 leader_election.cc:343] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.136077  9113 leader_election.cc:304] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:42.137542  9269 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:42.139087  9341 raft_consensus.cc:2749] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.148319  9272 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:42.148860  9272 ts_tablet_manager.cc:1403] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.354s	user 0.024s	sys 0.009s
I20260709 12:21:42.151907  9619 raft_consensus.cc:2749] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.168692  9270 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:42.169248  9270 ts_tablet_manager.cc:1403] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.359s	user 0.003s	sys 0.019s
I20260709 12:21:42.176257  9626 raft_consensus.cc:2749] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.185671  9272 raft_consensus.cc:359] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.186739  9272 raft_consensus.cc:385] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.187034  9272 raft_consensus.cc:740] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.190706  9272 consensus_queue.cc:260] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.195924  9149 raft_consensus.cc:2468] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
W20260709 12:21:42.215929  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 80413.8us
I20260709 12:21:42.226217  9268 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:42.227633  9268 ts_tablet_manager.cc:1403] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.322s	user 0.007s	sys 0.024s
I20260709 12:21:42.240528  9272 ts_tablet_manager.cc:1434] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.002s
I20260709 12:21:42.242074  9272 tablet_bootstrap.cc:492] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:42.261850  9038 leader_election.cc:304] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:42.280678  9270 raft_consensus.cc:359] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.291528  9270 raft_consensus.cc:385] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.291885  9270 raft_consensus.cc:740] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.293123  9592 raft_consensus.cc:493] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.298460  9481 raft_consensus.cc:2804] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:42.299142  9481 raft_consensus.cc:697] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:42.298861  9592 raft_consensus.cc:515] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.300114  9481 consensus_queue.cc:237] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:42.294502  9038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 47660.8us
I20260709 12:21:42.302423  9618 raft_consensus.cc:493] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.303036  9618 raft_consensus.cc:515] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.305322  9618 leader_election.cc:290] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:42.305534  9270 consensus_queue.cc:260] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.307320  9270 ts_tablet_manager.cc:1434] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.138s	user 0.010s	sys 0.000s
I20260709 12:21:42.308470  9270 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:42.314687  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:42.318099  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:42.318904  9268 raft_consensus.cc:359] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.319653  9268 raft_consensus.cc:385] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.319916  9268 raft_consensus.cc:740] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.320772  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:42.325418  9112 leader_election.cc:343] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.325743  9272 tablet_bootstrap.cc:654] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.327524  9592 leader_election.cc:290] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:42.329846  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30a7af654bae45a4a4bae112448a8932" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:42.330715  9149 raft_consensus.cc:2393] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 343eaafd515a4dfc9de3df299924f0d4 in current term 1: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:42.333578  9186 leader_election.cc:304] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:42.320605  9268 consensus_queue.cc:260] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.345510  9269 tablet_bootstrap.cc:654] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.340831  9490 raft_consensus.cc:3060] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.359048  9483 consensus_queue.cc:1048] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:42.377804  9268 ts_tablet_manager.cc:1434] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.150s	user 0.005s	sys 0.003s
W20260709 12:21:42.396360  9113 leader_election.cc:343] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.396953  9113 leader_election.cc:304] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:42.399088  9620 raft_consensus.cc:2749] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.415026  9270 tablet_bootstrap.cc:654] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.421777  9268 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
W20260709 12:21:42.461302  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 118222us
I20260709 12:21:42.478056  9366 raft_consensus.cc:493] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.493322  9490 raft_consensus.cc:2749] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.498510  9592 consensus_queue.cc:1048] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:21:42.503647  9366 raft_consensus.cc:515] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.542088  9563 consensus_queue.cc:1048] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:42.555713  9366 leader_election.cc:290] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:42.560539  9268 tablet_bootstrap.cc:654] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.563377  9272 tablet_bootstrap.cc:492] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:42.565466  9658 consensus_queue.cc:1048] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:42.577075  9272 ts_tablet_manager.cc:1403] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.335s	user 0.022s	sys 0.050s
I20260709 12:21:42.580458  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:42.587307  9272 raft_consensus.cc:359] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:42.588801  9039 leader_election.cc:343] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.590932  9272 raft_consensus.cc:385] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.591300  9272 raft_consensus.cc:740] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.592170  9272 consensus_queue.cc:260] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.590708  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:42.596553  9039 leader_election.cc:343] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.596992  9039 leader_election.cc:304] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:42.597695  9366 raft_consensus.cc:2749] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.599146  9272 ts_tablet_manager.cc:1434] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:21:42.600487  9272 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:42.607918  9272 tablet_bootstrap.cc:654] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:42.613363  9652 log.cc:927] Time spent T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:21:42.651850  9269 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:42.652491  9269 ts_tablet_manager.cc:1403] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.515s	user 0.025s	sys 0.012s
W20260709 12:21:42.663793  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 185013us
I20260709 12:21:42.674304  9272 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:42.679580  9272 ts_tablet_manager.cc:1403] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.014s
I20260709 12:21:42.685344  9593 raft_consensus.cc:493] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.685947  9593 raft_consensus.cc:515] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.689869  9593 leader_election.cc:290] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:42.691193  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:42.692009  9073 raft_consensus.cc:2468] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:42.693361  8963 leader_election.cc:304] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:42.694212  9593 raft_consensus.cc:2804] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:42.694705  9593 raft_consensus.cc:493] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:42.695112  9593 raft_consensus.cc:3060] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.696164  9272 raft_consensus.cc:359] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.696931  9272 raft_consensus.cc:385] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.697193  9272 raft_consensus.cc:740] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.701149  9269 raft_consensus.cc:359] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.701853  9269 raft_consensus.cc:385] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.705013  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:42.707576  9270 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:42.682629  8896 catalog_manager.cc:5697] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.708168  9270 ts_tablet_manager.cc:1403] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.400s	user 0.006s	sys 0.020s
I20260709 12:21:42.722586  9270 raft_consensus.cc:359] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.723407  9270 raft_consensus.cc:385] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.723739  9270 raft_consensus.cc:740] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.724561  9270 consensus_queue.cc:260] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:42.726588  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:42.731510  9268 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:42.732038  9268 ts_tablet_manager.cc:1403] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.311s	user 0.009s	sys 0.011s
I20260709 12:21:42.740908  9270 ts_tablet_manager.cc:1434] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 12:21:42.742156  9270 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:42.702113  9269 raft_consensus.cc:740] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.746912  9269 consensus_queue.cc:260] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.747839  9223 raft_consensus.cc:2468] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
W20260709 12:21:42.749243  9039 consensus_peers.cc:597] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 -> Peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Couldn't send request to peer 343eaafd515a4dfc9de3df299924f0d4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:42.750980  9268 raft_consensus.cc:359] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.753886  9268 raft_consensus.cc:385] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.754247  9268 raft_consensus.cc:740] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.755017  9268 consensus_queue.cc:260] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.756667  9268 ts_tablet_manager.cc:1434] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 12:21:42.757751  9268 tablet_bootstrap.cc:492] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:42.753515  9269 ts_tablet_manager.cc:1434] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.001s
I20260709 12:21:42.763262  9269 tablet_bootstrap.cc:492] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:42.697876  9272 consensus_queue.cc:260] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.768769  9272 ts_tablet_manager.cc:1434] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.000s
I20260709 12:21:42.769805  9272 tablet_bootstrap.cc:492] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:42.776875  9270 tablet_bootstrap.cc:654] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.782152  9269 tablet_bootstrap.cc:654] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.783759  9272 tablet_bootstrap.cc:654] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.789533  9270 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:42.791033  9593 raft_consensus.cc:515] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.793503  9269 tablet_bootstrap.cc:492] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:42.794134  9269 ts_tablet_manager.cc:1403] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.013s
I20260709 12:21:42.795636  9268 tablet_bootstrap.cc:654] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.797101  9269 raft_consensus.cc:359] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.797835  9269 raft_consensus.cc:385] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.790186  9270 ts_tablet_manager.cc:1403] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.000s
I20260709 12:21:42.798099  9269 raft_consensus.cc:740] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.798887  9269 consensus_queue.cc:260] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.800910  9270 raft_consensus.cc:359] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.801348  9269 ts_tablet_manager.cc:1434] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:42.801601  9270 raft_consensus.cc:385] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.801921  9270 raft_consensus.cc:740] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.802404  9269 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:42.802366  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:42.803328  9270 consensus_queue.cc:260] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.803647  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:42.804046  9219 raft_consensus.cc:3060] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.804466  9072 raft_consensus.cc:3060] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.805721  9270 ts_tablet_manager.cc:1434] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:42.807008  9270 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:42.812597  9219 raft_consensus.cc:2468] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:42.820727  9072 raft_consensus.cc:2468] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:42.812940  9270 tablet_bootstrap.cc:654] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.828975  8964 leader_election.cc:304] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:42.829821  9269 tablet_bootstrap.cc:654] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.829988  9645 raft_consensus.cc:2804] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:42.830600  9645 raft_consensus.cc:697] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:42.831393  9645 consensus_queue.cc:237] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.833704  9270 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:42.834224  9270 ts_tablet_manager.cc:1403] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 12:21:42.803592  9593 leader_election.cc:290] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:42.825716  9268 tablet_bootstrap.cc:492] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:42.827865  9272 tablet_bootstrap.cc:492] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:42.835024  9269 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:42.835366  9272 ts_tablet_manager.cc:1403] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.000s
I20260709 12:21:42.835552  9269 ts_tablet_manager.cc:1403] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.006s
I20260709 12:21:42.837414  9268 ts_tablet_manager.cc:1403] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.080s	user 0.017s	sys 0.004s
I20260709 12:21:42.838435  9272 raft_consensus.cc:359] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.839104  9272 raft_consensus.cc:385] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.839377  9272 raft_consensus.cc:740] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.840055  9268 raft_consensus.cc:359] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.840166  9272 consensus_queue.cc:260] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.840765  9268 raft_consensus.cc:385] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.841089  9268 raft_consensus.cc:740] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.841849  9268 consensus_queue.cc:260] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.843979  9268 ts_tablet_manager.cc:1434] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:42.846781  9272 ts_tablet_manager.cc:1434] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:42.848629  9269 raft_consensus.cc:359] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.849350  9269 raft_consensus.cc:385] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.855091  9269 raft_consensus.cc:740] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.855840  9269 consensus_queue.cc:260] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.858207  9269 ts_tablet_manager.cc:1434] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.004s
I20260709 12:21:42.852458  9270 raft_consensus.cc:359] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.860000  9270 raft_consensus.cc:385] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.860234  9269 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:42.860321  9270 raft_consensus.cc:740] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.853873  9268 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:42.854789  9272 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:42.866457  9270 consensus_queue.cc:260] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.867457  9272 tablet_bootstrap.cc:654] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.869077  9270 ts_tablet_manager.cc:1434] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:21:42.870276  9270 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:42.871766  9268 tablet_bootstrap.cc:654] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.878417  9269 tablet_bootstrap.cc:654] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.881954  9270 tablet_bootstrap.cc:654] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.886255  9269 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:42.888015  9269 ts_tablet_manager.cc:1403] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 12:21:42.888314  9268 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:42.888796  9268 ts_tablet_manager.cc:1403] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 12:21:42.892530  9341 raft_consensus.cc:493] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.892737  9268 raft_consensus.cc:359] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.893030  9341 raft_consensus.cc:515] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.893534  9268 raft_consensus.cc:385] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.893759  9272 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:42.893906  9268 raft_consensus.cc:740] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.894534  9272 ts_tablet_manager.cc:1403] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.004s
I20260709 12:21:42.894627  9268 consensus_queue.cc:260] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.895056  9341 leader_election.cc:290] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:42.896360  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f4d49acc064d6ea6cf83616117be16" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:42.896840  9268 ts_tablet_manager.cc:1434] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:21:42.897989  9268 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
W20260709 12:21:42.898401  9113 leader_election.cc:343] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.899672  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f4d49acc064d6ea6cf83616117be16" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:42.905766  9366 consensus_queue.cc:1048] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:42.906747  9113 leader_election.cc:343] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.907213  9113 leader_election.cc:304] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:42.908327  9268 tablet_bootstrap.cc:654] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.908203  9341 raft_consensus.cc:2749] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.911701  9272 raft_consensus.cc:359] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.912621  9272 raft_consensus.cc:385] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.912968  9272 raft_consensus.cc:740] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.908128  9558 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.913935  9558 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.916051  9269 raft_consensus.cc:359] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.916879  9269 raft_consensus.cc:385] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.917222  9269 raft_consensus.cc:740] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.917554  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:42.919081  8964 leader_election.cc:343] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.921730  9272 consensus_queue.cc:260] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.922823  9270 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:42.922896  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:42.923388  9270 ts_tablet_manager.cc:1403] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.011s
I20260709 12:21:42.923758  9144 raft_consensus.cc:2468] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:42.923588  9269 consensus_queue.cc:260] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.925240  9272 ts_tablet_manager.cc:1434] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 12:21:42.926380  9272 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:42.926579  9649 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.927147  9649 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.927520  8963 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:42.928400  9593 raft_consensus.cc:2804] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:42.928777  9593 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:42.929088  9593 raft_consensus.cc:3060] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.929340  9649 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:42.930657  9656 raft_consensus.cc:493] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.931140  9656 raft_consensus.cc:515] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.933190  9272 tablet_bootstrap.cc:654] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.933599  9270 raft_consensus.cc:359] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.934340  9270 raft_consensus.cc:385] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.934593  9270 raft_consensus.cc:740] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.934816  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:42.935453  9070 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:42.935281  9270 consensus_queue.cc:260] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.936592  9268 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:42.937090  9268 ts_tablet_manager.cc:1403] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.003s
I20260709 12:21:42.935739  9269 ts_tablet_manager.cc:1434] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 12:21:42.940454  9270 ts_tablet_manager.cc:1434] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.002s
I20260709 12:21:42.940348  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:42.941751  9270 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:42.911460  8894 catalog_manager.cc:5697] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.942632  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:42.943318  9144 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:42.944447  9649 raft_consensus.cc:493] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.945034  9649 raft_consensus.cc:515] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.945817  9269 tablet_bootstrap.cc:492] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:42.945870  8963 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:42.948163  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:42.948827  9219 raft_consensus.cc:2468] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:42.948966  9619 raft_consensus.cc:493] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.949550  9619 raft_consensus.cc:515] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:42.950250  8964 leader_election.cc:304] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:42.950762  9316 raft_consensus.cc:2804] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:42.951148  9316 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:42.951335  9649 leader_election.cc:290] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:42.951495  9316 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.951862  9619 leader_election.cc:290] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:42.955579  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:42.956619  9558 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
W20260709 12:21:42.957567  9113 leader_election.cc:343] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.957753  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:42.959213  9112 leader_election.cc:343] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:42.959771  9269 tablet_bootstrap.cc:654] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:42.959656  9112 leader_election.cc:304] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:42.951902  9649 raft_consensus.cc:2804] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:42.960722  9649 raft_consensus.cc:493] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:42.961059  9649 raft_consensus.cc:3060] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.961277  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:42.962028  9144 raft_consensus.cc:2468] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:42.962481  9341 raft_consensus.cc:2749] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.961810  9316 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.968626  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:42.968580  9268 raft_consensus.cc:359] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.969486  9268 raft_consensus.cc:385] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:42.969750  9268 raft_consensus.cc:740] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:42.970470  9268 consensus_queue.cc:260] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.970860  9186 leader_election.cc:304] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:42.990665  9662 raft_consensus.cc:2749] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.991127  9495 raft_consensus.cc:493] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.991634  9495 raft_consensus.cc:515] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:42.993752  9495 leader_election.cc:290] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:42.971202  9316 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:42.972447  9619 raft_consensus.cc:493] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.994885  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:42.995040  9619 raft_consensus.cc:515] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:42.995661  9070 raft_consensus.cc:2468] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
I20260709 12:21:42.996322  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:42.973093  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:42.997743  9149 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:42.998977  9268 ts_tablet_manager.cc:1434] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 12:21:43.000167  9268 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:43.002946  9619 leader_election.cc:290] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:42.973675  9649 raft_consensus.cc:515] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.004676  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:43.007288  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:43.008860  9113 leader_election.cc:343] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.009341  9649 leader_election.cc:290] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
W20260709 12:21:43.010788  9112 leader_election.cc:343] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.011234  9112 leader_election.cc:304] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:43.012672  9626 raft_consensus.cc:2749] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:42.973845  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:42.977874  9656 leader_election.cc:290] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:42.985049  9593 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:42.997151  9186 leader_election.cc:304] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:43.014302  9072 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.016993  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:43.017580  9219 raft_consensus.cc:3060] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.018807  9656 raft_consensus.cc:2804] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:43.019209  9656 raft_consensus.cc:493] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.019620  9656 raft_consensus.cc:3060] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:43.022562  9187 leader_election.cc:343] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.014137  9149 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.031613  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:43.032390  9146 raft_consensus.cc:3060] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.014153  9270 tablet_bootstrap.cc:654] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.015185  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:43.032519  9593 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:43.039338  9144 raft_consensus.cc:3060] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.051934  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:43.047844  9619 raft_consensus.cc:493] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.060577  8963 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:43.065587  9593 raft_consensus.cc:2804] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:43.074735  8964 leader_election.cc:343] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.076043  9269 tablet_bootstrap.cc:492] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:43.076591  9269 ts_tablet_manager.cc:1403] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.002s
I20260709 12:21:43.079890  9272 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:43.080453  9272 ts_tablet_manager.cc:1403] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.154s	user 0.034s	sys 0.018s
I20260709 12:21:43.080874  9269 raft_consensus.cc:359] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.081789  9269 raft_consensus.cc:385] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.065840  9619 raft_consensus.cc:515] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.082056  9269 raft_consensus.cc:740] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.083379  9272 raft_consensus.cc:359] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.084115  9272 raft_consensus.cc:385] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.084277  9619 leader_election.cc:290] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:43.084384  9272 raft_consensus.cc:740] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.090040  9593 raft_consensus.cc:697] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:43.091204  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:43.091867  9268 tablet_bootstrap.cc:654] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.085031  9269 consensus_queue.cc:260] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.079770  9072 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.099769  9146 raft_consensus.cc:2468] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.100287  9272 consensus_queue.cc:260] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.103089  9272 ts_tablet_manager.cc:1434] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 12:21:43.104524  9272 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2: Bootstrap starting.
I20260709 12:21:43.110922  9272 tablet_bootstrap.cc:654] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.091204  9593 consensus_queue.cc:237] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:43.114888  9219 raft_consensus.cc:2468] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.118732  8964 leader_election.cc:304] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:43.120002  9316 raft_consensus.cc:2804] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:43.120738  9316 raft_consensus.cc:697] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:43.123775  9269 ts_tablet_manager.cc:1434] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 12:21:43.121799  9316 consensus_queue.cc:237] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.120009  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:43.129053  9113 leader_election.cc:343] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:43.132575  9112 leader_election.cc:343] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.134042  9112 leader_election.cc:304] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:43.117957  9656 raft_consensus.cc:515] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.136653  9656 leader_election.cc:290] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:43.138190  9619 raft_consensus.cc:2749] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:43.143746  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:43.146085  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:43.146766  9072 raft_consensus.cc:3060] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.147255  9625 raft_consensus.cc:493] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.147969  9625 raft_consensus.cc:515] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:43.150626  8963 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:43.151768  8999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:43.153360  9187 leader_election.cc:343] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:43.153229  9039 leader_election.cc:343] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.158928  9269 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:43.154878  9625 leader_election.cc:290] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:43.158707  9649 raft_consensus.cc:2804] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:43.180507  9649 raft_consensus.cc:697] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:43.181407  9649 consensus_queue.cc:237] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.152657  9144 raft_consensus.cc:2468] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.185453  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:43.197693  9148 raft_consensus.cc:2468] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:43.200562  8894 catalog_manager.cc:5697] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:43.221133  9483 raft_consensus.cc:493] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.221680  9483 raft_consensus.cc:515] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.233880  9483 leader_election.cc:290] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:43.223598  9038 leader_election.cc:304] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:43.225944  9269 tablet_bootstrap.cc:654] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.237672  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:43.238658  9149 raft_consensus.cc:2468] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:43.239990  9038 leader_election.cc:304] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:43.253962  9316 consensus_queue.cc:1048] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:43.259411  9625 raft_consensus.cc:2804] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:43.260052  9625 raft_consensus.cc:493] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.260488  9625 raft_consensus.cc:3060] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.264941  9072 raft_consensus.cc:2468] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:43.250370  9366 raft_consensus.cc:2804] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:43.272452  9366 raft_consensus.cc:493] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.272897  9366 raft_consensus.cc:3060] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.280611  9272 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2: No bootstrap required, opened a new log
I20260709 12:21:43.281350  9272 ts_tablet_manager.cc:1403] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent bootstrapping tablet: real 0.177s	user 0.022s	sys 0.009s
I20260709 12:21:43.291198  9593 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.321023  9270 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
W20260709 12:21:43.321123  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 87630us
I20260709 12:21:43.321759  9270 ts_tablet_manager.cc:1403] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.380s	user 0.030s	sys 0.011s
I20260709 12:21:43.292155  9593 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.324723  9186 leader_election.cc:304] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:43.329792  9316 consensus_queue.cc:1048] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:43.327651  9592 raft_consensus.cc:2804] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:43.336710  9592 raft_consensus.cc:697] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:43.337615  9592 consensus_queue.cc:237] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.343094  9270 raft_consensus.cc:359] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.343801  9268 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:43.344419  9268 ts_tablet_manager.cc:1403] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.345s	user 0.021s	sys 0.024s
I20260709 12:21:43.347012  9272 raft_consensus.cc:359] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.347745  9272 raft_consensus.cc:385] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.348018  9272 raft_consensus.cc:740] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.348721  9272 consensus_queue.cc:260] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.350083  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:43.350801  9219 raft_consensus.cc:2468] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:43.352779  9593 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:43.354049  9268 raft_consensus.cc:359] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.354782  9268 raft_consensus.cc:385] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.355059  9268 raft_consensus.cc:740] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.343833  9270 raft_consensus.cc:385] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.359424  9366 raft_consensus.cc:515] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:43.355739  9268 consensus_queue.cc:260] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.356146  9270 raft_consensus.cc:740] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.370177  9269 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:43.370970  9269 ts_tablet_manager.cc:1403] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.230s	user 0.008s	sys 0.034s
W20260709 12:21:43.372714  9685 log.cc:927] Time spent T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:21:43.373207  9272 ts_tablet_manager.cc:1434] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.002s
I20260709 12:21:43.374159  9268 ts_tablet_manager.cc:1434] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:21:43.375272  9268 tablet_bootstrap.cc:492] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:43.376116  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:43.377066  9269 raft_consensus.cc:359] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.370577  9270 consensus_queue.cc:260] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.377732  9269 raft_consensus.cc:385] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.379180  9269 raft_consensus.cc:740] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.388010  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:43.388937  9149 raft_consensus.cc:3060] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.392851  9269 consensus_queue.cc:260] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.401899  9269 ts_tablet_manager.cc:1434] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.002s
I20260709 12:21:43.404822  9366 leader_election.cc:290] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:43.396023  9619 raft_consensus.cc:493] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.410607  9619 raft_consensus.cc:515] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.413342  9619 leader_election.cc:290] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:43.394520  8964 leader_election.cc:343] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.408649  9270 ts_tablet_manager.cc:1434] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.002s
I20260709 12:21:43.429409  9149 raft_consensus.cc:2468] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:43.443279  9366 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.444133  9366 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.435190  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:43.451824  9366 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:43.438050  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:43.455341  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:43.438505  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:43.465739  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:43.468597  8995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:43.468783  9625 raft_consensus.cc:515] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.471722  9270 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:43.476027  9221 raft_consensus.cc:3060] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.478852  9625 leader_election.cc:290] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:43.478356  8894 catalog_manager.cc:5697] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:43.470232  9073 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
W20260709 12:21:43.473457  9113 leader_election.cc:343] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.451645  9038 leader_election.cc:304] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:43.485455  9366 raft_consensus.cc:2804] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:43.485926  9366 raft_consensus.cc:697] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:43.486667  9366 consensus_queue.cc:237] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:43.488410  8895 catalog_manager.cc:5697] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:43.490021  8895 catalog_manager.cc:5697] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:43.489899  9039 leader_election.cc:343] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:43.492810  9039 leader_election.cc:343] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.495321  9269 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:43.497380  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:43.498030  9218 raft_consensus.cc:3060] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:43.502348  9039 leader_election.cc:343] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.502861  9268 tablet_bootstrap.cc:654] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.502979  9039 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:43.504315  9625 raft_consensus.cc:2749] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:43.510842  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
W20260709 12:21:43.511296  8963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 75233.2us
I20260709 12:21:43.512068  9149 raft_consensus.cc:3060] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:43.512555  9113 leader_election.cc:343] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.513096  9113 leader_election.cc:304] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:43.514232  9619 raft_consensus.cc:2749] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:43.516944  9270 tablet_bootstrap.cc:654] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.527884  8963 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:43.529675  9316 raft_consensus.cc:2804] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:43.530045  9316 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.530400  9316 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.534827  9149 raft_consensus.cc:2468] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:43.539053  9619 raft_consensus.cc:493] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.540230  9218 raft_consensus.cc:2468] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:43.559180  9039 leader_election.cc:304] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:43.567026  9366 raft_consensus.cc:2804] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:43.570170  9649 consensus_queue.cc:1048] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:43.573087  9366 raft_consensus.cc:697] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:43.574007  9366 consensus_queue.cc:237] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.547482  9619 raft_consensus.cc:515] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.588788  9619 leader_election.cc:290] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:43.593402  9269 tablet_bootstrap.cc:654] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:43.600705  9649 consensus_queue.cc:1048] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:43.607875  9645 consensus_queue.cc:1048] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:43.609745  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:43.612561  9113 leader_election.cc:343] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.578577  9316 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.620687  9268 tablet_bootstrap.cc:492] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:43.621300  9268 ts_tablet_manager.cc:1403] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.246s	user 0.027s	sys 0.010s
I20260709 12:21:43.624594  9223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:43.624514  9268 raft_consensus.cc:359] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.625949  9268 raft_consensus.cc:385] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:43.626266  9113 leader_election.cc:343] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:43.626770  9113 leader_election.cc:304] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:43.626281  9268 raft_consensus.cc:740] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.627861  9619 raft_consensus.cc:2749] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:43.630368  9268 consensus_queue.cc:260] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.637770  9316 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:43.653450  9268 ts_tablet_manager.cc:1434] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 12:21:43.663357  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:43.664151  9219 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.652781  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:43.668008  9072 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:43.690429  9268 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:43.699190  9219 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.705279  9072 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:43.708072  8964 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:43.724215  9316 raft_consensus.cc:2804] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:43.753731  9316 raft_consensus.cc:697] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:43.754699  9316 consensus_queue.cc:237] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.763191  9268 tablet_bootstrap.cc:654] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:43.845141  9685 log.cc:927] Time spent T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 12:21:43.886089  9705 log.cc:927] Time spent T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:21:43.894356  9269 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:43.895081  9269 ts_tablet_manager.cc:1403] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.443s	user 0.013s	sys 0.005s
I20260709 12:21:43.901276  9270 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:43.901902  9270 ts_tablet_manager.cc:1403] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.454s	user 0.006s	sys 0.013s
I20260709 12:21:43.906838  9715 raft_consensus.cc:493] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.907475  9715 raft_consensus.cc:515] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:43.909590  9715 leader_election.cc:290] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:43.936761  9268 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:43.937433  9268 ts_tablet_manager.cc:1403] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.257s	user 0.007s	sys 0.016s
W20260709 12:21:43.948977  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:43.958001  9220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:43.958994  9688 raft_consensus.cc:493] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.959492  9688 raft_consensus.cc:515] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:43.962670  9269 raft_consensus.cc:359] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.963407  9269 raft_consensus.cc:385] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:43.963680  9269 raft_consensus.cc:740] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:43.964390  9269 consensus_queue.cc:260] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:43.970528  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d24a7495d4a42b94f6f25809db7f8" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:43.971447  9069 raft_consensus.cc:2393] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate 343eaafd515a4dfc9de3df299924f0d4 in this term.
I20260709 12:21:43.991282  9269 ts_tablet_manager.cc:1434] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.000s
I20260709 12:21:43.986603  9619 raft_consensus.cc:493] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.024742  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:44.027010  9688 leader_election.cc:290] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:44.027177  9145 raft_consensus.cc:2468] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:44.026068  9000 raft_consensus.cc:3060] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.033589  9038 leader_election.cc:304] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:44.034878  9693 raft_consensus.cc:2804] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:44.035317  9693 raft_consensus.cc:493] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.035658  9693 raft_consensus.cc:3060] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.021375  9619 raft_consensus.cc:515] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.040421  9619 leader_election.cc:290] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:44.044517  9693 raft_consensus.cc:515] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.047060  9693 leader_election.cc:290] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:44.051471  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:44.052398  9149 raft_consensus.cc:3060] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:44.055923  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 50477.6us
I20260709 12:21:44.055482  9658 consensus_queue.cc:1048] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:44.061048  9268 raft_consensus.cc:359] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.061851  9268 raft_consensus.cc:385] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.062093  9268 raft_consensus.cc:740] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.066709  9268 consensus_queue.cc:260] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:44.068845  8964 consensus_peers.cc:597] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 -> Peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Couldn't send request to peer 343eaafd515a4dfc9de3df299924f0d4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:44.069859  9270 raft_consensus.cc:359] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.070758  9270 raft_consensus.cc:385] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.071054  9270 raft_consensus.cc:740] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.071836  9270 consensus_queue.cc:260] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.080458  9268 ts_tablet_manager.cc:1434] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.143s	user 0.003s	sys 0.007s
I20260709 12:21:44.082929  9682 consensus_queue.cc:1048] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:44.083513  9149 raft_consensus.cc:2468] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:44.084764  9721 consensus_queue.cc:1048] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:44.090593  9038 leader_election.cc:304] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:44.091632  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:44.096061  9113 leader_election.cc:343] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:44.072710  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:44.101265  9072 raft_consensus.cc:2468] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:44.107527  9270 ts_tablet_manager.cc:1434] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.205s	user 0.010s	sys 0.000s
I20260709 12:21:44.108817  9270 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:44.110543  9268 tablet_bootstrap.cc:492] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:44.112905  9366 raft_consensus.cc:2804] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:44.113456  9366 raft_consensus.cc:697] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:44.114257  9366 consensus_queue.cc:237] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.088639  9715 consensus_queue.cc:1048] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:21:44.127207  9112 leader_election.cc:304] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:44.136844  9626 raft_consensus.cc:2804] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:44.137353  9626 raft_consensus.cc:493] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.137753  9626 raft_consensus.cc:3060] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.153836  9270 tablet_bootstrap.cc:654] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.160079  9268 tablet_bootstrap.cc:654] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.212843  9658 raft_consensus.cc:493] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.213507  9658 raft_consensus.cc:515] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.208132  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:44.228645  9626 raft_consensus.cc:515] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.232009  9626 leader_election.cc:290] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:44.232757  9270 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:44.233430  9270 ts_tablet_manager.cc:1403] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.125s	user 0.018s	sys 0.004s
I20260709 12:21:44.238492  9268 tablet_bootstrap.cc:492] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:44.238119  9270 raft_consensus.cc:359] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.240108  9270 raft_consensus.cc:385] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.240144  9268 ts_tablet_manager.cc:1403] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.149s	user 0.015s	sys 0.020s
I20260709 12:21:44.240541  9270 raft_consensus.cc:740] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.221727  9658 leader_election.cc:290] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:44.241470  9270 consensus_queue.cc:260] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.247234  9268 raft_consensus.cc:359] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.248297  9268 raft_consensus.cc:385] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.248566  9268 raft_consensus.cc:740] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.249567  9270 ts_tablet_manager.cc:1434] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.001s
I20260709 12:21:44.249321  9268 consensus_queue.cc:260] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.251746  9268 ts_tablet_manager.cc:1434] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:44.252113  9270 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:44.252871  9268 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:44.477171  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:44.477906  9072 raft_consensus.cc:3060] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.483160  9481 consensus_queue.cc:1048] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:44.496240  9270 tablet_bootstrap.cc:654] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.503648  9268 tablet_bootstrap.cc:654] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.507107  8995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f4d49acc064d6ea6cf83616117be16" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:44.508279  8995 raft_consensus.cc:2468] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
W20260709 12:21:44.525683  9039 leader_election.cc:343] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:44.533444  8963 leader_election.cc:304] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:44.536661  9724 raft_consensus.cc:2749] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:44.537968  9187 leader_election.cc:304] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:44.559646  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:44.560196  9727 raft_consensus.cc:2804] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:44.560961  9727 raft_consensus.cc:493] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.560335  8998 raft_consensus.cc:3060] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.561468  9727 raft_consensus.cc:3060] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.564339  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:44.568480  9269 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:44.573513  8998 raft_consensus.cc:2468] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:44.580471  9727 raft_consensus.cc:515] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.582437  9269 tablet_bootstrap.cc:654] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.583487  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f4d49acc064d6ea6cf83616117be16" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:44.584120  9145 raft_consensus.cc:3060] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.587041  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f4d49acc064d6ea6cf83616117be16" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:44.593709  9268 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:44.594352  9268 ts_tablet_manager.cc:1403] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.342s	user 0.014s	sys 0.003s
I20260709 12:21:44.596454  9072 raft_consensus.cc:2468] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:44.597925  8896 catalog_manager.cc:5697] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:44.599982  9270 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:44.600528  9270 ts_tablet_manager.cc:1403] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.349s	user 0.014s	sys 0.001s
I20260709 12:21:44.606463  9727 leader_election.cc:290] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:44.613471  9039 consensus_peers.cc:597] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:44.614445  9112 leader_election.cc:304] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:44.616223  9626 raft_consensus.cc:2804] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:44.616469  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f4d49acc064d6ea6cf83616117be16" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:44.616779  9626 raft_consensus.cc:697] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:44.620529  8998 raft_consensus.cc:3060] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.620893  9626 consensus_queue.cc:237] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.605123  9270 raft_consensus.cc:359] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.630729  9270 raft_consensus.cc:385] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.631057  9270 raft_consensus.cc:740] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.631714  9270 consensus_queue.cc:260] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.635586  9269 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:44.643709  9269 ts_tablet_manager.cc:1403] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.075s	user 0.007s	sys 0.007s
I20260709 12:21:44.629756  9145 raft_consensus.cc:2468] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
W20260709 12:21:44.648476  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 53686.1us
I20260709 12:21:44.649509  8998 raft_consensus.cc:2468] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:44.657063  9270 ts_tablet_manager.cc:1434] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.003s
I20260709 12:21:44.659126  9270 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:44.668686  9186 leader_election.cc:304] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:44.669520  9656 raft_consensus.cc:2804] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:44.670032  9656 raft_consensus.cc:697] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:44.670925  9656 consensus_queue.cc:237] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.675886  9593 consensus_queue.cc:1048] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:44.677404  9268 raft_consensus.cc:359] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.678493  9268 raft_consensus.cc:385] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.678830  9268 raft_consensus.cc:740] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.679636  9268 consensus_queue.cc:260] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.680491  9270 tablet_bootstrap.cc:654] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.702636  9694 consensus_queue.cc:1048] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:44.703007  9269 raft_consensus.cc:359] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.704011  9269 raft_consensus.cc:385] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.704321  9269 raft_consensus.cc:740] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.706194  9269 consensus_queue.cc:260] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.707301  9726 consensus_queue.cc:1048] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:44.711480  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 97366.2us
I20260709 12:21:44.727943  9269 ts_tablet_manager.cc:1434] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.081s	user 0.001s	sys 0.006s
I20260709 12:21:44.729158  9269 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:44.736510  9269 tablet_bootstrap.cc:654] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.739564  9726 raft_consensus.cc:493] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.740108  9726 raft_consensus.cc:515] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.742154  9726 leader_election.cc:290] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:44.750973  9268 ts_tablet_manager.cc:1434] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.156s	user 0.008s	sys 0.000s
I20260709 12:21:44.749917  8898 catalog_manager.cc:5697] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:44.758544  8898 catalog_manager.cc:5697] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } }
I20260709 12:21:44.760617  9268 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7: Bootstrap starting.
I20260709 12:21:44.761451  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:44.775707  9143 raft_consensus.cc:2468] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 0.
I20260709 12:21:44.762053  9270 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:44.777101  9270 ts_tablet_manager.cc:1403] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.005s
I20260709 12:21:44.778127  9038 leader_election.cc:304] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:44.779017  9726 raft_consensus.cc:2804] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:44.779445  9726 raft_consensus.cc:493] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.779830  9726 raft_consensus.cc:3060] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.779946  9270 raft_consensus.cc:359] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.780709  9270 raft_consensus.cc:385] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.780999  9270 raft_consensus.cc:740] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
W20260709 12:21:44.773499  9039 leader_election.cc:343] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:44.792203  9270 consensus_queue.cc:260] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.793480  9694 consensus_queue.cc:1048] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:44.809365  9268 tablet_bootstrap.cc:654] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.817399  8894 catalog_manager.cc:5697] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:44.823796  9270 ts_tablet_manager.cc:1434] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:21:44.827557  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:44.832870  9039 consensus_peers.cc:597] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 -> Peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Couldn't send request to peer 343eaafd515a4dfc9de3df299924f0d4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:44.831959  9593 consensus_queue.cc:1048] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:21:44.833689  9039 leader_election.cc:343] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:44.830772  9656 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.837184  9656 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.839130  9270 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:44.842527  9656 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:44.843977  9726 raft_consensus.cc:515] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.844748  8895 catalog_manager.cc:5697] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:44.845980  9726 leader_election.cc:290] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:44.847584  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:44.850924  9649 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.851157  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:44.851529  9649 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
W20260709 12:21:44.855185  9039 leader_election.cc:343] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:44.858393  9270 tablet_bootstrap.cc:654] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.855333  9269 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:44.850191  8895 catalog_manager.cc:5697] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:44.859510  9269 ts_tablet_manager.cc:1403] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.131s	user 0.018s	sys 0.021s
I20260709 12:21:44.860409  9649 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:44.859432  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:44.862098  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:44.862772  9070 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:44.864073  8963 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:44.864944  9649 raft_consensus.cc:2804] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:44.865332  9649 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.872465  9270 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:44.873016  9270 ts_tablet_manager.cc:1403] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.003s
I20260709 12:21:44.865671  9649 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.875861  9270 raft_consensus.cc:359] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.876518  9270 raft_consensus.cc:385] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.876770  9268 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7: No bootstrap required, opened a new log
I20260709 12:21:44.877264  9268 ts_tablet_manager.cc:1403] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.003s
I20260709 12:21:44.879897  9268 raft_consensus.cc:359] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.880565  9268 raft_consensus.cc:385] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.880821  9268 raft_consensus.cc:740] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.881491  9268 consensus_queue.cc:260] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.876780  9270 raft_consensus.cc:740] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.882845  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:44.883461  9149 raft_consensus.cc:3060] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.883261  9270 consensus_queue.cc:260] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:44.891335  9270 ts_tablet_manager.cc:1434] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:21:44.892771  9270 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:44.891800  9149 raft_consensus.cc:2468] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:44.899741  9649 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.901628  9038 leader_election.cc:304] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:44.902215  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:44.903294  9726 raft_consensus.cc:2804] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:44.903836  9726 raft_consensus.cc:697] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:44.904954  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:44.904728  9726 consensus_queue.cc:237] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.912012  9270 tablet_bootstrap.cc:654] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.914712  9186 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:44.915310  9268 ts_tablet_manager.cc:1434] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:21:44.885087  9269 raft_consensus.cc:359] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.915992  9656 raft_consensus.cc:2749] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:44.916317  9269 raft_consensus.cc:385] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.916649  9269 raft_consensus.cc:740] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.917322  9269 consensus_queue.cc:260] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.917778  9074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:44.918215  9219 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:44.918380  9074 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.918936  9269 ts_tablet_manager.cc:1434] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.001s
I20260709 12:21:44.919936  9269 tablet_bootstrap.cc:492] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:44.925266  9074 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:44.926472  8963 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:44.927311  9593 raft_consensus.cc:2804] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:44.927800  9593 raft_consensus.cc:697] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:44.928582  9593 consensus_queue.cc:237] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:44.930697  9269 tablet_bootstrap.cc:654] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.933292  8997 raft_consensus.cc:3060] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.934352  9619 raft_consensus.cc:493] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.935031  9619 raft_consensus.cc:515] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:44.937956  9619 leader_election.cc:290] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:44.939743  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:44.940485  9070 raft_consensus.cc:2468] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 0.
I20260709 12:21:44.941568  9112 leader_election.cc:304] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:44.943010  9619 raft_consensus.cc:2804] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:44.943452  9619 raft_consensus.cc:493] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.943866  9619 raft_consensus.cc:3060] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:44.944818  9269 tablet_bootstrap.cc:492] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:44.945379  9269 ts_tablet_manager.cc:1403] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 12:21:44.939646  8998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:44.948271  9113 leader_election.cc:343] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:44.953054  9269 raft_consensus.cc:359] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.953847  9269 raft_consensus.cc:385] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:44.954479  9649 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:44.954115  9269 raft_consensus.cc:740] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:44.957585  9269 consensus_queue.cc:260] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:44.961669  9269 ts_tablet_manager.cc:1434] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:21:44.962749  9269 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:44.973956  9269 tablet_bootstrap.cc:654] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:44.992218  9269 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:44.992777  9269 ts_tablet_manager.cc:1403] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 12:21:45.001487  9269 raft_consensus.cc:359] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.002247  9269 raft_consensus.cc:385] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.002552  9269 raft_consensus.cc:740] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.003255  9269 consensus_queue.cc:260] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.005720  9269 ts_tablet_manager.cc:1434] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:21:45.006953  9269 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:45.029217  9269 tablet_bootstrap.cc:654] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:45.053360  9269 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:45.053880  9269 ts_tablet_manager.cc:1403] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.006s
I20260709 12:21:45.083436  9742 consensus_queue.cc:1048] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.083315  9269 raft_consensus.cc:359] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.085506  9269 raft_consensus.cc:385] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.085825  9269 raft_consensus.cc:740] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.086524  9269 consensus_queue.cc:260] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.096973  9269 ts_tablet_manager.cc:1434] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:21:45.098003  9269 tablet_bootstrap.cc:492] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:45.100667  9341 consensus_queue.cc:1048] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.112607  9727 consensus_queue.cc:1048] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.116303  9269 tablet_bootstrap.cc:654] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:45.161248  9495 consensus_queue.cc:1048] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:45.200542  9219 raft_consensus.cc:3060] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.205150  8894 catalog_manager.cc:5697] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:45.231362  9649 raft_consensus.cc:493] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.232127  9649 raft_consensus.cc:515] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.236522  9619 raft_consensus.cc:515] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:45.240319  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:45.240959  9070 raft_consensus.cc:3060] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.242492  9649 leader_election.cc:290] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
W20260709 12:21:45.245718  8946 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:45.250769  9726 raft_consensus.cc:493] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.251303  9726 raft_consensus.cc:515] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:45.253363  9726 leader_election.cc:290] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:45.255793  8995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
W20260709 12:21:45.259552  8929 debug-util.cc:398] Leaking SignalData structure 0x7b080004a700 after lost signal to thread 30823
I20260709 12:21:45.260680  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:45.261226  9039 leader_election.cc:343] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:45.261402  9222 raft_consensus.cc:2468] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
W20260709 12:21:45.261237  8929 debug-util.cc:398] Leaking SignalData structure 0x7b080028c380 after lost signal to thread 8932
W20260709 12:21:45.263268  8929 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1aa0 after lost signal to thread 9024
W20260709 12:21:45.264525  8929 debug-util.cc:398] Leaking SignalData structure 0x7b08002a7300 after lost signal to thread 9099
W20260709 12:21:45.265759  8929 debug-util.cc:398] Leaking SignalData structure 0x7b0800178a60 after lost signal to thread 9173
I20260709 12:21:45.268611  8964 leader_election.cc:304] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:45.259544  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:45.270694  9649 raft_consensus.cc:2804] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:45.271116  9649 raft_consensus.cc:493] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:45.271474  9649 raft_consensus.cc:3060] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:45.272179  9039 leader_election.cc:343] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:45.272637  9039 leader_election.cc:304] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:45.275275  9698 raft_consensus.cc:2749] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:45.278599  9270 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:45.279165  9270 ts_tablet_manager.cc:1403] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.387s	user 0.040s	sys 0.038s
I20260709 12:21:45.242784  9619 leader_election.cc:290] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
W20260709 12:21:45.295157  8929 debug-util.cc:398] Leaking SignalData structure 0x7b08000f63a0 after lost signal to thread 9247
I20260709 12:21:45.300684  9070 raft_consensus.cc:2468] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:45.281865  9270 raft_consensus.cc:359] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:45.303519  9270 raft_consensus.cc:385] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.309952  9270 raft_consensus.cc:740] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.302670  9112 leader_election.cc:304] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:45.301504  8897 catalog_manager.cc:5697] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:45.320889  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:45.321604  9143 raft_consensus.cc:2468] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:45.317131  9619 raft_consensus.cc:2804] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:45.323340  9619 raft_consensus.cc:697] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:45.324390  9619 consensus_queue.cc:237] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:45.327320  9698 consensus_queue.cc:1048] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.316949  9270 consensus_queue.cc:260] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:45.338873  9779 raft_consensus.cc:493] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.362764  8995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
I20260709 12:21:45.365188  9592 raft_consensus.cc:493] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.365761  9592 raft_consensus.cc:515] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:45.367871  9592 leader_election.cc:290] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:45.369725  9649 raft_consensus.cc:515] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.373518  9269 tablet_bootstrap.cc:492] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:45.374207  9269 ts_tablet_manager.cc:1403] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.276s	user 0.028s	sys 0.087s
I20260709 12:21:45.375905  9146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:45.383404  9270 ts_tablet_manager.cc:1434] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.104s	user 0.009s	sys 0.000s
I20260709 12:21:45.385617  9270 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:45.386264  9216 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.387643  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:45.376349  9779 raft_consensus.cc:515] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:45.391696  9222 raft_consensus.cc:3060] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.394488  8995 raft_consensus.cc:3060] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.394856  9186 leader_election.cc:304] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:45.404172  9779 leader_election.cc:290] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:45.410754  9148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:45.411834  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013d7d1179ed4bffba8be127fe4a2d03" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:45.412621  9070 raft_consensus.cc:2393] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
I20260709 12:21:45.414172  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:45.414796  9144 raft_consensus.cc:3060] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.413550  8898 catalog_manager.cc:5697] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:45.422102  9662 raft_consensus.cc:2749] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:45.424042  8963 leader_election.cc:304] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:45.425196  9269 raft_consensus.cc:359] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:45.426007  9269 raft_consensus.cc:385] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.426350  9269 raft_consensus.cc:740] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.426133  9726 consensus_queue.cc:1048] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.427088  9269 consensus_queue.cc:260] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:45.429149  9269 ts_tablet_manager.cc:1434] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.000s
I20260709 12:21:45.430060  9269 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4: Bootstrap starting.
I20260709 12:21:45.438632  9649 leader_election.cc:290] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:45.441596  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:45.442562  9219 raft_consensus.cc:3060] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.449611  9270 tablet_bootstrap.cc:654] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:45.453631  8995 raft_consensus.cc:2468] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:45.454643  9780 raft_consensus.cc:2749] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:45.457593  9219 raft_consensus.cc:2468] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:45.458173  9144 raft_consensus.cc:2468] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:45.462364  8963 leader_election.cc:304] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:45.464957  9780 raft_consensus.cc:2804] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:45.465600  9780 raft_consensus.cc:697] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:45.466764  9780 consensus_queue.cc:237] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.508634  9649 consensus_queue.cc:1048] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:45.561049  9269 tablet_bootstrap.cc:654] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:45.576059  8895 catalog_manager.cc:5697] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:45.581152  9592 raft_consensus.cc:493] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.581907  9592 raft_consensus.cc:515] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.587136  9592 leader_election.cc:290] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:45.588961  8997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938" is_pre_election: true
I20260709 12:21:45.591949  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:45.592705  9143 raft_consensus.cc:2468] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 0.
W20260709 12:21:45.592285  9187 leader_election.cc:343] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:45.598778  9186 leader_election.cc:304] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:45.605238  9592 raft_consensus.cc:2804] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:45.605741  9592 raft_consensus.cc:493] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:45.606180  9592 raft_consensus.cc:3060] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.610891  9269 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4: No bootstrap required, opened a new log
I20260709 12:21:45.611421  9269 ts_tablet_manager.cc:1403] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.000s
I20260709 12:21:45.614619  9780 consensus_queue.cc:1048] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.615490  9592 raft_consensus.cc:515] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.617812  9592 leader_election.cc:290] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:45.614051  9269 raft_consensus.cc:359] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:45.623286  8995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
W20260709 12:21:45.624960  9187 leader_election.cc:343] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:45.623456  9269 raft_consensus.cc:385] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.625828  9269 raft_consensus.cc:740] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.626556  9269 consensus_queue.cc:260] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:45.627691  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:45.628381  9143 raft_consensus.cc:3060] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.631441  9269 ts_tablet_manager.cc:1434] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:21:45.653476  9143 raft_consensus.cc:2468] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:45.681244  9186 leader_election.cc:304] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:45.682487  9719 raft_consensus.cc:2804] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:45.683403  9270 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:45.683974  9270 ts_tablet_manager.cc:1403] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.299s	user 0.026s	sys 0.019s
I20260709 12:21:45.685436  9719 raft_consensus.cc:697] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:45.686477  9719 consensus_queue.cc:237] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.715094  9270 raft_consensus.cc:359] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.715910  9270 raft_consensus.cc:385] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.716212  9270 raft_consensus.cc:740] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.716890  9270 consensus_queue.cc:260] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.745784  9270 ts_tablet_manager.cc:1434] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.062s	user 0.004s	sys 0.003s
I20260709 12:21:45.778435  8895 catalog_manager.cc:5697] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 1 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:45.782666  8963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 57025.3us
I20260709 12:21:45.796392  9270 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:45.843837  9270 tablet_bootstrap.cc:654] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:45.844862  9744 consensus_queue.cc:1048] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:45.849432  9592 raft_consensus.cc:493] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.850054  9592 raft_consensus.cc:515] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:45.852196  9592 leader_election.cc:290] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:45.855507  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:45.856083  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13219c476384456b808b69575b85edcd" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:45.860900  9270 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: No bootstrap required, opened a new log
I20260709 12:21:45.861141  9186 leader_election.cc:304] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:45.861461  9270 ts_tablet_manager.cc:1403] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.002s
I20260709 12:21:45.861591  9222 raft_consensus.cc:3060] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.864878  9270 raft_consensus.cc:359] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:45.873875  9270 raft_consensus.cc:385] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.874241  9270 raft_consensus.cc:740] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.875406  9270 consensus_queue.cc:260] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:45.881551  9270 ts_tablet_manager.cc:1434] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:21:45.896734  9734 consensus_queue.cc:1048] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.905432  9495 raft_consensus.cc:2749] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:45.925496  9316 raft_consensus.cc:493] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.926000  9316 raft_consensus.cc:515] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:45.939965  9793 consensus_queue.cc:1048] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.947198  9316 leader_election.cc:290] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:45.948817  9144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:45.949710  9144 raft_consensus.cc:2393] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e47ee72c4e114aef9818c7c324ed1938 in current term 1: Already voted for candidate 343eaafd515a4dfc9de3df299924f0d4 in this term.
I20260709 12:21:45.950431  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:45.952098  8964 leader_election.cc:304] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:45.953153  9316 raft_consensus.cc:3060] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.956964  9794 consensus_queue.cc:1048] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:45.967314  9780 raft_consensus.cc:493] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.967864  9780 raft_consensus.cc:515] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:45.970213  9780 leader_election.cc:290] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:45.971625  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:45.972334  9070 raft_consensus.cc:2468] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:45.973038  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:45.973752  9222 raft_consensus.cc:2468] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 0.
I20260709 12:21:45.973978  8963 leader_election.cc:304] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:45.975390  9780 raft_consensus.cc:2804] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:45.975893  9780 raft_consensus.cc:493] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:45.976316  9780 raft_consensus.cc:3060] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.992439  9316 raft_consensus.cc:2749] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:46.045300  9780 raft_consensus.cc:515] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:46.054593  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:46.055409  9222 raft_consensus.cc:3060] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:46.057852  9070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:46.058764  9070 raft_consensus.cc:3060] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:46.063835  9222 raft_consensus.cc:2468] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:46.065223  8964 leader_election.cc:304] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:21:46.066089  9316 raft_consensus.cc:2804] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:46.066648  9316 raft_consensus.cc:697] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:21:46.067492  9316 consensus_queue.cc:237] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:46.057983  9780 leader_election.cc:290] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 1 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:46.107400  8897 catalog_manager.cc:5697] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 0 to 1, leader changed from <none> to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:46.119688  9070 raft_consensus.cc:2468] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 1.
I20260709 12:21:46.169354  9662 consensus_queue.cc:1048] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:46.199909  9495 consensus_queue.cc:1048] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:46.544239  9316 consensus_queue.cc:1048] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:46.596156  9316 consensus_queue.cc:1048] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:46.615348 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
W20260709 12:21:46.622670  9113 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 87 similar messages]
W20260709 12:21:46.800773  8964 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 190581us
W20260709 12:21:46.822702  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 150735us
W20260709 12:21:46.909794  9113 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:46.914320  9113 consensus_peers.cc:597] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:46.921458  9113 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.071637  9038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 152107us
W20260709 12:21:47.085084  8964 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 145049us
W20260709 12:21:47.135052  9113 consensus_peers.cc:597] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.153256  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 175256us
W20260709 12:21:47.154879  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 237783us
W20260709 12:21:47.208102  8963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 100113us
W20260709 12:21:47.314576  9113 consensus_peers.cc:597] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.322057  9113 consensus_peers.cc:597] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.337235  9186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 150620us
W20260709 12:21:47.348852  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 148936us
W20260709 12:21:47.348925  9113 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.375598  9039 consensus_peers.cc:597] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.383795  9039 consensus_peers.cc:597] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.400996  9039 consensus_peers.cc:597] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.516196  9187 consensus_peers.cc:597] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.519136  9187 consensus_peers.cc:597] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.519810  9187 consensus_peers.cc:597] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.520416  9187 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.520993  9187 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:47.521371  9835 raft_consensus.cc:493] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa2e7a4cc66248d4be460ea6a1fd48f2)
W20260709 12:21:47.521512  9187 consensus_peers.cc:597] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.523324  9187 consensus_peers.cc:597] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.524015  9187 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.524643  9187 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:47.521989  9835 raft_consensus.cc:515] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:47.547003  9835 leader_election.cc:290] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
W20260709 12:21:47.547308  9039 consensus_peers.cc:597] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.548002  9039 consensus_peers.cc:597] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.548607  9039 consensus_peers.cc:597] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.554893  9187 consensus_peers.cc:597] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.562355  9039 consensus_peers.cc:597] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.563139  9039 consensus_peers.cc:597] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.563716  9039 consensus_peers.cc:597] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.564301  9039 consensus_peers.cc:597] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.564833  9039 consensus_peers.cc:597] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:47.571934  9039 consensus_peers.cc:597] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:47.587951  9835 raft_consensus.cc:493] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ded781e200f40a1b78076731e570fb7)
I20260709 12:21:47.588536  9835 raft_consensus.cc:515] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:47.590703  9835 leader_election.cc:290] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:47.591528  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:47.594681  8963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:43631, user_credentials={real_user=slave}} blocked reactor thread for 73141.1us
I20260709 12:21:47.597499  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:47.598241  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77c1d2e7167495fbe8556f3ba691a55" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.599670  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ee9ad5a10c4a739bf573cebe2cb609" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:47.604681  8963 leader_election.cc:304] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.605511  9806 raft_consensus.cc:2749] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.607579  8963 leader_election.cc:304] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:47.607946 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:47.608287  9806 raft_consensus.cc:2749] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.609352 30820 tablet_replica.cc:333] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.610496 30820 raft_consensus.cc:2243] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.611068 30820 raft_consensus.cc:2272] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.613680 30820 tablet_replica.cc:333] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.614625 30820 raft_consensus.cc:2243] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.615132 30820 raft_consensus.cc:2272] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.617697 30820 tablet_replica.cc:333] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.618837 30820 raft_consensus.cc:2243] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.621387 30820 raft_consensus.cc:2272] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.623896 30820 tablet_replica.cc:333] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.624734 30820 raft_consensus.cc:2243] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.625247 30820 raft_consensus.cc:2272] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.632792 30820 tablet_replica.cc:333] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.633672 30820 raft_consensus.cc:2243] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.634708 30820 raft_consensus.cc:2272] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.638346 30820 tablet_replica.cc:333] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.639266 30820 raft_consensus.cc:2243] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.640270 30820 raft_consensus.cc:2272] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.642850 30820 tablet_replica.cc:333] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.643718 30820 raft_consensus.cc:2243] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.645308 30820 raft_consensus.cc:2272] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.647928 30820 tablet_replica.cc:333] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.648772 30820 raft_consensus.cc:2243] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.649797 30820 raft_consensus.cc:2272] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.653543 30820 tablet_replica.cc:333] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.654412 30820 raft_consensus.cc:2243] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.654914 30820 raft_consensus.cc:2272] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.662906  9806 raft_consensus.cc:493] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa2e7a4cc66248d4be460ea6a1fd48f2)
I20260709 12:21:47.663439  9806 raft_consensus.cc:515] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:47.665629  9806 leader_election.cc:290] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:47.666699  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c992391e9d4940a3b550c8c511b354d1" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.668560  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c992391e9d4940a3b550c8c511b354d1" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:47.669713 30820 tablet_replica.cc:333] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.670161  8964 leader_election.cc:304] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.671459  9806 raft_consensus.cc:2749] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.671542 30820 raft_consensus.cc:2243] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.675688 30820 raft_consensus.cc:2272] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.679986 30820 tablet_replica.cc:333] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.680936 30820 raft_consensus.cc:2243] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.682267 30820 raft_consensus.cc:2272] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.691192 30820 tablet_replica.cc:333] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.694520 30820 raft_consensus.cc:2243] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.696810 30820 raft_consensus.cc:2272] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.699551 30820 tablet_replica.cc:333] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.700507 30820 raft_consensus.cc:2243] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.701169 30820 raft_consensus.cc:2272] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.704070 30820 tablet_replica.cc:333] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.705041 30820 raft_consensus.cc:2243] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.705662 30820 raft_consensus.cc:2272] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.713323  9806 raft_consensus.cc:493] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa2e7a4cc66248d4be460ea6a1fd48f2)
I20260709 12:21:47.713907  9806 raft_consensus.cc:515] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:47.716078  9806 leader_election.cc:290] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:47.718755  9843 raft_consensus.cc:493] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 343eaafd515a4dfc9de3df299924f0d4)
I20260709 12:21:47.719267  9843 raft_consensus.cc:515] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:47.721314  9843 leader_election.cc:290] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:47.721988  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.723012  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f703417858b43a1b0d7affdd86231f6" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:47.723340  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149942a6b05d454496753897bc2ed0b8" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:47.725926  8963 leader_election.cc:304] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.732851 30820 tablet_replica.cc:333] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.733844 30820 raft_consensus.cc:2243] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.733884  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149942a6b05d454496753897bc2ed0b8" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:47.734611 30820 raft_consensus.cc:2272] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.737644 30820 tablet_replica.cc:333] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.737833  8963 leader_election.cc:304] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4
I20260709 12:21:47.738528 30820 raft_consensus.cc:2243] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.739107 30820 raft_consensus.cc:2272] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.740818  9835 raft_consensus.cc:2749] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.741290  9806 raft_consensus.cc:493] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 343eaafd515a4dfc9de3df299924f0d4)
I20260709 12:21:47.741922  9806 raft_consensus.cc:515] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:47.727149  9843 raft_consensus.cc:2749] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.751101  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.753407  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb71a58f7994546bdcf177f0ec7b6b1" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:47.757146  8963 leader_election.cc:304] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.758383  9843 raft_consensus.cc:2749] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.762565  9806 leader_election.cc:290] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:47.771169 30820 tablet_replica.cc:333] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.772161 30820 raft_consensus.cc:2243] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.772763 30820 raft_consensus.cc:2272] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.789564 30820 tablet_replica.cc:333] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.790637 30820 raft_consensus.cc:2243] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.791337 30820 raft_consensus.cc:2272] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.793540  9843 raft_consensus.cc:493] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa2e7a4cc66248d4be460ea6a1fd48f2)
I20260709 12:21:47.794044  9843 raft_consensus.cc:515] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:47.796144  9843 leader_election.cc:290] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:47.797567  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:47.798534  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2c3b1da14244ec82ac5d25e49e0862" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.797520 30820 tablet_replica.cc:333] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.799938  8963 leader_election.cc:304] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.800923  9806 raft_consensus.cc:2749] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.801712 30820 raft_consensus.cc:2243] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.802469 30820 raft_consensus.cc:2272] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.806879 30820 tablet_replica.cc:333] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.812328 30820 raft_consensus.cc:2243] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.812966 30820 raft_consensus.cc:2272] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.830349  9806 raft_consensus.cc:493] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ded781e200f40a1b78076731e570fb7)
I20260709 12:21:47.830956  9806 raft_consensus.cc:515] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:47.833042  9806 leader_election.cc:290] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:47.835194  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.838557  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec4c3600c2a4b22876a710877cf531d" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:47.848965  8963 leader_election.cc:304] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.850055  9843 raft_consensus.cc:2749] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.820760 30820 tablet_replica.cc:333] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.851804 30820 raft_consensus.cc:2243] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.856745 30820 raft_consensus.cc:2272] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.860384 30820 tablet_replica.cc:333] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.861411 30820 raft_consensus.cc:2243] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.862092 30820 raft_consensus.cc:2272] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.865453 30820 tablet_replica.cc:333] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.867239  9843 raft_consensus.cc:493] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ded781e200f40a1b78076731e570fb7)
I20260709 12:21:47.868202 30820 raft_consensus.cc:2243] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.869412 30820 raft_consensus.cc:2272] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.872318 30820 tablet_replica.cc:333] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.867918  9843 raft_consensus.cc:515] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:47.875903 30820 raft_consensus.cc:2243] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.876746  9843 leader_election.cc:290] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:47.877388  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:47.879189 30820 raft_consensus.cc:2272] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.882195 30820 tablet_replica.cc:333] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.883133 30820 raft_consensus.cc:2243] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.884358  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:47.885937  8963 leader_election.cc:304] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e47ee72c4e114aef9818c7c324ed1938; no voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:21:47.886801  9843 raft_consensus.cc:2749] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:47.887167 30820 raft_consensus.cc:2272] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.896795 30820 tablet_replica.cc:333] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.897775 30820 raft_consensus.cc:2243] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.898447 30820 raft_consensus.cc:2272] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.901226 30820 tablet_replica.cc:333] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.902055 30820 raft_consensus.cc:2243] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.902669 30820 raft_consensus.cc:2272] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.905967 30820 tablet_replica.cc:333] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.906838 30820 raft_consensus.cc:2243] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.907490 30820 raft_consensus.cc:2272] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.910375 30820 tablet_replica.cc:333] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.911234 30820 raft_consensus.cc:2243] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.916961 30820 raft_consensus.cc:2272] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.919937 30820 tablet_replica.cc:333] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.920845 30820 raft_consensus.cc:2243] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.921419 30820 raft_consensus.cc:2272] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.924096 30820 tablet_replica.cc:333] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.925028 30820 raft_consensus.cc:2243] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.925637 30820 raft_consensus.cc:2272] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.928866 30820 tablet_replica.cc:333] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.929742 30820 raft_consensus.cc:2243] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.930526 30820 raft_consensus.cc:2272] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.933439 30820 tablet_replica.cc:333] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.934469 30820 raft_consensus.cc:2243] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.935014 30820 raft_consensus.cc:2272] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.938970 30820 tablet_replica.cc:333] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.939900 30820 raft_consensus.cc:2243] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.940486 30820 raft_consensus.cc:2272] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.953351 30820 tablet_replica.cc:333] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.954453 30820 raft_consensus.cc:2243] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.955192 30820 raft_consensus.cc:2272] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.958151 30820 tablet_replica.cc:333] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.958997 30820 raft_consensus.cc:2243] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.959565 30820 raft_consensus.cc:2272] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.962159 30820 tablet_replica.cc:333] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.963023 30820 raft_consensus.cc:2243] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.963549 30820 raft_consensus.cc:2272] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.967402 30820 tablet_replica.cc:333] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.968238 30820 raft_consensus.cc:2243] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.969343 30820 raft_consensus.cc:2272] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.973377 30820 tablet_replica.cc:333] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.974175 30820 raft_consensus.cc:2243] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.975216 30820 raft_consensus.cc:2272] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.979221 30820 tablet_replica.cc:333] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.980018 30820 raft_consensus.cc:2243] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.980988 30820 raft_consensus.cc:2272] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.983431 30820 tablet_replica.cc:333] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.984253 30820 raft_consensus.cc:2243] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.984894 30820 raft_consensus.cc:2272] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.987780 30820 tablet_replica.cc:333] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.988555 30820 raft_consensus.cc:2243] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:47.989115 30820 raft_consensus.cc:2272] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.991818 30820 tablet_replica.cc:333] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.992529 30820 raft_consensus.cc:2243] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.993538 30820 raft_consensus.cc:2272] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.997360 30820 tablet_replica.cc:333] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:47.998132 30820 raft_consensus.cc:2243] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.999212 30820 raft_consensus.cc:2272] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:48.002077 30820 tablet_replica.cc:333] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: stopping tablet replica
I20260709 12:21:48.002871 30820 raft_consensus.cc:2243] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:48.003479 30820 raft_consensus.cc:2272] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:48.825974  9619 raft_consensus.cc:493] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.826584  9619 raft_consensus.cc:515] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.830088  9619 leader_election.cc:290] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.831260  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
W20260709 12:21:48.834172  9113 leader_election.cc:336] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:48.845535  9113 leader_election.cc:304] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:48.846781  9848 raft_consensus.cc:2749] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:48.889451  9814 raft_consensus.cc:493] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.890209  9821 raft_consensus.cc:493] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.890103  9814 raft_consensus.cc:515] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.890899  9821 raft_consensus.cc:515] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.893232  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:48.893695  9495 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.892390  9814 leader_election.cc:290] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.894305  9495 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.896569  9495 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.898958  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:48.899552  9821 leader_election.cc:290] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.901790  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:48.902581  9143 raft_consensus.cc:2468] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:48.904901  9186 leader_election.cc:304] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:48.906021  9821 raft_consensus.cc:2804] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:48.906412  9821 raft_consensus.cc:493] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.906805  9821 raft_consensus.cc:3060] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:48.909786  9187 leader_election.cc:336] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:48.910401  9187 leader_election.cc:304] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:48.911213  9495 raft_consensus.cc:2749] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:48.912194  9187 leader_election.cc:336] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:48.913906  9187 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:48.915340  9495 raft_consensus.cc:2749] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:48.922253  9187 leader_election.cc:336] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:48.926352  9495 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.926995  9495 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.930776  9814 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.931383  9814 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.931409  9495 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:48.934396  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:48.937603  9814 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:48.939020  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:48.945971  9849 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.946653  9849 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.966446  9848 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.967070  9848 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:48.967793  9847 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.968432  9847 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.969265  9848 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:48.947109  9619 raft_consensus.cc:493] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.970033  9619 raft_consensus.cc:515] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.972002  9619 leader_election.cc:290] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.955005  9825 raft_consensus.cc:493] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.974536  9825 raft_consensus.cc:515] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.976826  9825 leader_election.cc:290] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.978047  9825 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.978615  9825 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.981189  9825 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.959194  9821 raft_consensus.cc:515] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.965699  9495 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.984216  9495 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.985586  9821 leader_election.cc:290] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.981972  9822 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.959574  9726 raft_consensus.cc:493] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.986294  9495 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:48.992020  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd869b989df40608f17c7091c100fc7" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:48.987501  9822 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.992844  9143 raft_consensus.cc:3060] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:48.994985  9822 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.989051  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:48.996627  9619 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:48.996753  9073 raft_consensus.cc:2468] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:48.997159  9619 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:48.987726  9726 raft_consensus.cc:515] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:48.999006  9619 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:48.999805  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.000367  9726 leader_election.cc:290] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:48.996170  9113 leader_election.cc:336] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.001181  9113 leader_election.cc:336] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.001259  9112 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.002395  9619 raft_consensus.cc:2749] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.005182  9186 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
W20260709 12:21:49.009893  9113 leader_election.cc:336] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.010950  9814 raft_consensus.cc:2804] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.011384  9814 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.011746  9814 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:49.015595  9039 leader_election.cc:336] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.022589  9187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 100359us
I20260709 12:21:49.024322  9849 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.025027  9847 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.022018  9814 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
W20260709 12:21:49.032071  9039 leader_election.cc:336] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.032717  9039 leader_election.cc:336] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.033187  9039 leader_election.cc:336] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.033635  9039 leader_election.cc:336] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.034950  9113 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.038735  9113 leader_election.cc:336] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.040242  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:49.051101  9072 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.045509  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.054939  9217 raft_consensus.cc:2468] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:49.046051  9495 raft_consensus.cc:493] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.056051  9495 raft_consensus.cc:515] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.047667  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.049352  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.058743  9216 raft_consensus.cc:2468] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:49.021986  9143 raft_consensus.cc:2468] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 2.
I20260709 12:21:49.058246  9495 leader_election.cc:290] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.062873  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.063616  9222 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:49.050417  9814 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.065482  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.066781  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.066919  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.067871  9216 raft_consensus.cc:2393] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 2: Already voted for candidate 343eaafd515a4dfc9de3df299924f0d4 in this term.
I20260709 12:21:49.068480  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.065491  9072 raft_consensus.cc:2468] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 2.
I20260709 12:21:49.076301  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.078817  9186 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.080794  9039 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.082013  9825 raft_consensus.cc:2804] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.082407  9825 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.082813  9825 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.083271  9821 raft_consensus.cc:2804] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.083904  9821 raft_consensus.cc:697] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:49.085662  9186 leader_election.cc:304] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.087355  9814 raft_consensus.cc:2804] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.088083  9814 raft_consensus.cc:697] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:49.089254  9814 consensus_queue.cc:237] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:49.091396  9825 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.084908  9821 consensus_queue.cc:237] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.095105  9113 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.097446  9039 leader_election.cc:304] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.098176  9847 raft_consensus.cc:2749] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.099177  9039 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.099947  9726 raft_consensus.cc:2749] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.101188  9849 raft_consensus.cc:2804] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.101614  9849 raft_consensus.cc:493] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.101969  9849 raft_consensus.cc:3060] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.102242  9847 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.103663  9847 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.106587  9822 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.107287  9822 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.139045  9495 raft_consensus.cc:493] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.139695  9495 raft_consensus.cc:515] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.140240  9822 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
W20260709 12:21:49.143321 30820 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9021)
I20260709 12:21:49.144011  9495 leader_election.cc:290] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.144886  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:49.145637  9149 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:49.146521  9847 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.149962  9038 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.151197  9113 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.151721  9822 raft_consensus.cc:2804] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:21:49.151988  9039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.4:38651, user_credentials={real_user=slave}} blocked reactor thread for 52884.3us
I20260709 12:21:49.152231  9822 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.152710  9822 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.153402  9825 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:49.157980  9187 leader_election.cc:336] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.158526  9187 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.159319  9619 raft_consensus.cc:2749] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.160537  9113 leader_election.cc:304] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.161387  9822 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.162539  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.165475  9072 raft_consensus.cc:2468] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:49.166595  9186 leader_election.cc:304] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.167547  9619 raft_consensus.cc:2804] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.167975  9619 raft_consensus.cc:493] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.168323  9619 raft_consensus.cc:3060] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.164289  9847 raft_consensus.cc:493] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.174234  9847 raft_consensus.cc:515] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.175271  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:49.175999  9149 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.175118  9619 raft_consensus.cc:515] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:49.177964  9619 leader_election.cc:290] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.178613  9495 raft_consensus.cc:2749] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.179716  9851 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.180198  9851 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:49.181576  9187 leader_election.cc:336] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.182116  9187 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.184653  9149 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:21:49.163980  9039 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.185786  9038 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.186571  9039 leader_election.cc:304] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.186766  9853 raft_consensus.cc:2749] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.187628  9853 raft_consensus.cc:2804] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.188683  9726 raft_consensus.cc:2749] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.188061  8895 catalog_manager.cc:5697] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.182199  9851 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.191174  9495 raft_consensus.cc:2749] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.189728  8895 catalog_manager.cc:5697] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:49.192108  9187 leader_election.cc:336] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.192873  9187 leader_election.cc:336] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.195410  9851 raft_consensus.cc:2804] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.195951  9851 raft_consensus.cc:493] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.196381  9851 raft_consensus.cc:3060] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.196421  9822 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.199362  9847 leader_election.cc:290] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.202582  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:49.203301  9143 raft_consensus.cc:2468] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:49.203447  9853 raft_consensus.cc:697] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:49.204392  9853 consensus_queue.cc:237] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:49.206262  9113 leader_election.cc:336] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.215206  9849 raft_consensus.cc:515] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:49.207855  9186 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.225083  9814 raft_consensus.cc:2804] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.225502  9814 raft_consensus.cc:493] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.225874  9814 raft_consensus.cc:3060] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.213649  9851 raft_consensus.cc:515] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.221645  9619 raft_consensus.cc:493] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.226697  9619 raft_consensus.cc:515] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.229481  9619 leader_election.cc:290] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.231503  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.235673  9851 leader_election.cc:290] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.245730  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69899b55ef9441a39d4f4a192d838f17" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:49.246346  9851 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.246987  9851 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:49.246417  9072 raft_consensus.cc:3060] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.249107  9851 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.254562  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:49.258555  9849 leader_election.cc:290] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.261930  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:49.269794  8898 catalog_manager.cc:5697] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.266674  9619 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.271932  9221 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.272300  9619 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:49.272614  9113 consensus_peers.cc:597] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.274017  9853 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.274556  9619 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.267928  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.274662  9853 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.275277  9216 raft_consensus.cc:2468] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:49.269622  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.276707  9217 raft_consensus.cc:2468] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:49.277475  9853 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
W20260709 12:21:49.270782  9039 leader_election.cc:336] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.278052  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
W20260709 12:21:49.278501  9039 leader_election.cc:336] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.278512  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
W20260709 12:21:49.279250  9039 leader_election.cc:336] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.279403  9073 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 3.
I20260709 12:21:49.234434  9814 raft_consensus.cc:515] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:49.280687  9112 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.278729  9143 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 3.
I20260709 12:21:49.266125  9072 raft_consensus.cc:2468] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 2.
I20260709 12:21:49.283013  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fdefe9cd9cb4f9f821cf256b4ec6f6e" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:49.283758  9038 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.284880  9619 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:21:49.269519  9187 leader_election.cc:336] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.286554  9187 leader_election.cc:336] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.285511  9853 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:49.286957  9187 leader_election.cc:304] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.287158  9853 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.287909  9853 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.288741  9186 leader_election.cc:304] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.289050  9619 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.290103  9619 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.297557  9619 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.302847  9821 raft_consensus.cc:2749] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.302031  9495 raft_consensus.cc:493] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.304042  9149 raft_consensus.cc:3060] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.303723  9495 raft_consensus.cc:515] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.304772  9821 raft_consensus.cc:2804] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.305872  9495 leader_election.cc:290] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.302191  9221 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:21:49.311944  9849 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.314610  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.324214  9072 raft_consensus.cc:2468] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:49.314289  9149 raft_consensus.cc:2468] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 2.
I20260709 12:21:49.325878  9821 raft_consensus.cc:697] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:49.281572  9814 leader_election.cc:290] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:49.326804  9113 leader_election.cc:336] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.326336  9186 leader_election.cc:304] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.328816  9186 leader_election.cc:304] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.328125  9113 leader_election.cc:304] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.326870  9821 consensus_queue.cc:237] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.332139  9848 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.327286  9849 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:49.327199  9855 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.332653  9848 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.333211  9619 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 4 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
W20260709 12:21:49.329277  9039 leader_election.cc:336] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.333395  9855 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:49.334185  9039 leader_election.cc:336] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.334828  9848 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.335773  9855 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.335978  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.336685  9073 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
I20260709 12:21:49.337288  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:49.337888  9145 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:49.338200  9112 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.338944  9038 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.329757  9859 raft_consensus.cc:2804] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.339757  9847 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.340133  9847 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.340346  9850 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.340476  9859 raft_consensus.cc:697] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:49.340868  9848 raft_consensus.cc:2749] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.340878  9850 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.341989  9850 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.341653  9859 consensus_queue.cc:237] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:49.342901  9113 leader_election.cc:336] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.332760  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:49.329322  9814 raft_consensus.cc:2804] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.343871  9814 raft_consensus.cc:493] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.344340  9814 raft_consensus.cc:3060] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.344277  9853 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.345649  9072 raft_consensus.cc:2393] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in current term 4: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:49.340546  9847 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.338272  9849 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:49.359596  9847 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.360540  9853 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 4 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.361507  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:49.362247  9145 raft_consensus.cc:2393] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 4: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
W20260709 12:21:49.358739  9039 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.364615  9850 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.367640  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:49.368445  9072 raft_consensus.cc:2393] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in current term 2: Already voted for candidate 0ded781e200f40a1b78076731e570fb7 in this term.
I20260709 12:21:49.370035  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce01c7da004041a58a5a746d2f258ee9" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:49.370668  9221 raft_consensus.cc:3060] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.372848  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:49.373740  9145 raft_consensus.cc:2393] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ded781e200f40a1b78076731e570fb7 in current term 2: Already voted for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in this term.
W20260709 12:21:49.366178  9113 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.372646  9847 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.375443  9850 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.375573  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98c726df727346b694e0509fae44d789" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:49.376219  9217 raft_consensus.cc:3060] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.377130  9222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4" is_pre_election: true
I20260709 12:21:49.377767  9222 raft_consensus.cc:2468] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 1.
W20260709 12:21:49.381321  9187 leader_election.cc:336] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.384531  8898 catalog_manager.cc:5697] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.386430  8898 catalog_manager.cc:5697] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.389273  9221 raft_consensus.cc:2468] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:21:49.390856  9217 raft_consensus.cc:2468] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
W20260709 12:21:49.395015  9187 leader_election.cc:336] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.395565  9113 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.396934  9113 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.397388  9113 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.399118  9850 raft_consensus.cc:2749] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:21:49.399322  9113 leader_election.cc:336] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.404043  9113 leader_election.cc:336] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.405889  9039 leader_election.cc:336] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.372627  9814 raft_consensus.cc:515] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.409216  9814 leader_election.cc:290] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.414819  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72198f72a6de43099cba6530224ef11d" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:49.415597  9072 raft_consensus.cc:3060] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.421648  9113 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.424001  9849 raft_consensus.cc:2804] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.424434  9849 raft_consensus.cc:493] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.424770  9849 raft_consensus.cc:3060] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.426239  9859 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.426908  9859 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.432916  9073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:49.433243  9849 raft_consensus.cc:515] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:49.433650  9073 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 1.
I20260709 12:21:49.434234  9859 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.429623  9072 raft_consensus.cc:2468] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 2.
I20260709 12:21:49.435189  9186 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.437726  9186 leader_election.cc:304] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.438652  9814 raft_consensus.cc:2804] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.439126  9814 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.439579  9814 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.443912  9849 leader_election.cc:290] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:21:49.435804  9039 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.438679  9113 leader_election.cc:304] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.445118  9726 raft_consensus.cc:2804] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.445523  9726 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.445736  9821 raft_consensus.cc:2804] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.445930  9726 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.446488  9821 raft_consensus.cc:697] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:49.447441  9821 consensus_queue.cc:237] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.445534  9113 leader_election.cc:304] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.457317  9039 leader_election.cc:304] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.457701  9848 raft_consensus.cc:2804] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.458279  9853 raft_consensus.cc:2804] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.458333  9848 raft_consensus.cc:697] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:49.458907  9853 raft_consensus.cc:493] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.459380  9853 raft_consensus.cc:3060] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.459681  9848 consensus_queue.cc:237] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:49.455200  9726 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.458494  9850 raft_consensus.cc:2804] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.462000  9850 raft_consensus.cc:697] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:49.465296  9850 consensus_queue.cc:237] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:49.467904  9853 raft_consensus.cc:515] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.469794  9814 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.469183  9039 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.470973  9855 raft_consensus.cc:2804] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.471452  9855 raft_consensus.cc:697] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:49.472232  9855 consensus_queue.cc:237] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.477617  9726 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.477905  8898 catalog_manager.cc:5697] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.480003  9853 leader_election.cc:290] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
I20260709 12:21:49.481482  9814 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:49.482986  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "343eaafd515a4dfc9de3df299924f0d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
W20260709 12:21:49.484678  9113 leader_election.cc:336] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.488701  9113 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.488951  9072 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:49.489698  9113 consensus_peers.cc:597] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.490819  9849 raft_consensus.cc:2749] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:21:49.481104  9039 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.491762  9039 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.492558  9726 raft_consensus.cc:2749] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:21:49.500010  9187 leader_election.cc:336] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.501011  9187 leader_election.cc:336] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.501353  9187 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.509590  9814 raft_consensus.cc:2749] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.509610  8898 catalog_manager.cc:5697] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:49.510671  9187 leader_election.cc:336] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.511467  9187 leader_election.cc:336] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.511404  8898 catalog_manager.cc:5697] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:49.514583  9113 leader_election.cc:336] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.515408  9847 raft_consensus.cc:493] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.515993  9847 raft_consensus.cc:515] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.518059  9847 leader_election.cc:290] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.520138  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:49.520263  9072 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 343eaafd515a4dfc9de3df299924f0d4 in term 2.
I20260709 12:21:49.520750  9149 raft_consensus.cc:2468] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 1.
I20260709 12:21:49.521979  9038 leader_election.cc:304] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.522630  9186 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:21:49.523546  9814 raft_consensus.cc:2804] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.523905  8895 catalog_manager.cc:5697] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:49.525008  9814 raft_consensus.cc:697] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Leader. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:21:49.525979  9814 consensus_queue.cc:237] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.522744  9847 raft_consensus.cc:2804] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:49.534332  9847 raft_consensus.cc:493] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:21:49.534718  9847 raft_consensus.cc:3060] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:49.538937  9039 leader_election.cc:336] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.541114  9039 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7; no voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.541913  9853 raft_consensus.cc:2749] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:49.547014  9217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f9c182103549958915a7a7ed1eab44" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:49.547701  9217 raft_consensus.cc:3060] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.556182  9847 raft_consensus.cc:515] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.557044  9221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:49.557794  9221 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.559314  9149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae55b8962494f7582e7d5304cb39d13" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:49.559386  9218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46df15ab8dec424780c28700160bbdb2" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:21:49.560074  9149 raft_consensus.cc:3060] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.560197  9218 raft_consensus.cc:3060] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:49.560747  9039 leader_election.cc:336] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.561268  9039 leader_election.cc:336] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.563972  9187 leader_election.cc:336] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.565065  9187 leader_election.cc:336] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.567206  9221 raft_consensus.cc:2468] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:21:49.568077  9218 raft_consensus.cc:2468] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
W20260709 12:21:49.570909  9039 leader_election.cc:336] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.571338  9217 raft_consensus.cc:2468] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
W20260709 12:21:49.571738  9039 consensus_peers.cc:597] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.577289  9187 leader_election.cc:336] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:49.583356  9149 raft_consensus.cc:2468] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:21:49.587136  9038 leader_election.cc:304] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:49.588223  9825 raft_consensus.cc:2804] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.588692  9825 raft_consensus.cc:697] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:49.589565  9825 consensus_queue.cc:237] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:49.590057  8895 catalog_manager.cc:5697] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4). New cstate: current_term: 2 leader_uuid: "343eaafd515a4dfc9de3df299924f0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.592366  9039 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.598542  9853 raft_consensus.cc:2804] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.599056  9853 raft_consensus.cc:697] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:49.598948  9039 leader_election.cc:304] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.600019  9853 consensus_queue.cc:237] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:49.602545  9726 raft_consensus.cc:2804] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.603089  9726 raft_consensus.cc:697] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:49.603904  9726 consensus_queue.cc:237] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
W20260709 12:21:49.607447  9039 leader_election.cc:336] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:49.617920  9187 consensus_peers.cc:597] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.620857  9113 leader_election.cc:304] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:49.621739  9850 raft_consensus.cc:2804] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:49.622274  9850 raft_consensus.cc:697] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:49.623360  9850 consensus_queue.cc:237] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
W20260709 12:21:49.631681  9039 consensus_peers.cc:597] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.634208  9145 raft_consensus.cc:1275] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer 343eaafd515a4dfc9de3df299924f0d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.636757  9859 consensus_queue.cc:1048] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:49.637584  9847 leader_election.cc:290] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:49.655851  8895 catalog_manager.cc:5697] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.662503  8898 catalog_manager.cc:5697] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.687613  9072 raft_consensus.cc:1275] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer 343eaafd515a4dfc9de3df299924f0d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.693521  9814 consensus_queue.cc:1048] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:49.700376  8898 catalog_manager.cc:5697] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:49.715155  9039 consensus_peers.cc:597] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.713871  8898 catalog_manager.cc:5697] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 2, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:49.725337  9187 consensus_peers.cc:597] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:49.728096  9113 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.735893  9187 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.748090  9149 raft_consensus.cc:1275] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer 343eaafd515a4dfc9de3df299924f0d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.749487  9821 consensus_queue.cc:1048] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:49.766101  9113 consensus_peers.cc:597] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.768035  9143 raft_consensus.cc:1275] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.769484  9847 consensus_queue.cc:1048] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:49.788079  9039 consensus_peers.cc:597] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.811370  9039 consensus_peers.cc:597] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.821313  9039 consensus_peers.cc:597] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.828063  9187 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.836966  9039 consensus_peers.cc:597] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:49.852931  9039 consensus_peers.cc:597] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:49.853668  9072 raft_consensus.cc:1275] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer 343eaafd515a4dfc9de3df299924f0d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.855297  9821 consensus_queue.cc:1048] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:49.858042  9074 raft_consensus.cc:1275] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer 343eaafd515a4dfc9de3df299924f0d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:49.864537  9859 consensus_queue.cc:1048] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:49.898468  9187 consensus_peers.cc:597] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.898952  9187 consensus_peers.cc:597] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:49.908051  9039 consensus_peers.cc:597] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.902228  9187 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.912530  9187 consensus_peers.cc:597] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:49.945591  9113 consensus_peers.cc:597] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:49.963874  9187 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.968346  9039 consensus_peers.cc:597] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:49.968950  9039 consensus_peers.cc:597] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:49.996608  9039 consensus_peers.cc:597] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:50.009270  9187 consensus_peers.cc:597] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.011066  9221 raft_consensus.cc:1275] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:50.021588  9850 consensus_queue.cc:1048] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:50.021914  9187 consensus_peers.cc:597] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:50.028997  9217 raft_consensus.cc:1275] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:50.030447  9726 consensus_queue.cc:1048] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:50.050033  9113 consensus_peers.cc:597] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:50.058444  9187 consensus_peers.cc:597] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:50.066656  9113 consensus_peers.cc:597] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.077419  9074 raft_consensus.cc:1275] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer 343eaafd515a4dfc9de3df299924f0d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:50.079061  9859 consensus_queue.cc:1048] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:50.091102  9187 consensus_peers.cc:597] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:50.093591  9221 raft_consensus.cc:1275] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:50.104310  9113 consensus_peers.cc:597] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.105126  9850 consensus_queue.cc:1048] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:50.107656  9221 raft_consensus.cc:1275] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:50.113816  9850 consensus_queue.cc:1048] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:50.097570  9039 consensus_peers.cc:597] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:50.120288  9218 raft_consensus.cc:1275] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:50.134955  9847 consensus_queue.cc:1048] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:50.138937  9187 consensus_peers.cc:597] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.142064  9219 raft_consensus.cc:1275] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:50.143043  9039 consensus_peers.cc:597] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:50.143620  9039 consensus_peers.cc:597] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:50.144265 30820 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 9021)
W20260709 12:21:50.149127  9039 consensus_peers.cc:597] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.179569  9847 consensus_queue.cc:1048] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:50.210984  9145 raft_consensus.cc:1275] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:50.212541  9726 consensus_queue.cc:1048] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:50.236359  9187 consensus_peers.cc:597] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:50.236918  9187 consensus_peers.cc:597] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:50.500650  9039 consensus_peers.cc:597] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:51.088594  9848 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.089162  9848 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:51.091878  9848 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:51.092900  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7" is_pre_election: true
I20260709 12:21:51.093608  9069 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 4.
I20260709 12:21:51.094867  9112 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:51.095746  9848 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:51.096140  9848 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.096457  9848 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.104553  9848 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:51.108973  9848 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 5 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:21:51.110539  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:21:51.111491  9069 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:51.111846  9113 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:21:51.115219  9113 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:51.132272  9069 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 5.
I20260709 12:21:51.133984  9112 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:51.134707  9848 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:51.136559  9848 raft_consensus.cc:697] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:21:51.138044  9848 consensus_queue.cc:237] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
W20260709 12:21:51.144726 30820 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 9021)
I20260709 12:21:51.162233  8898 catalog_manager.cc:5697] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 3 to 5, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 5 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.283967  9726 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.284430  9726 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:51.287458  9726 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:21:51.288234  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" is_pre_election: true
I20260709 12:21:51.288909  9145 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:21:51.290653  9038 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:21:51.291988  9825 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.292343  9825 raft_consensus.cc:493] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.292646  9825 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:51.322264  9039 leader_election.cc:336] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:51.308806  9825 raft_consensus.cc:515] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:51.328680  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f97ca50b64e4e5eb86f9392e8a6d6cd" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:21:51.329289  9145 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.329995  9825 leader_election.cc:290] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 3 election: Requested vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
W20260709 12:21:51.332788  9039 leader_election.cc:336] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:21:51.337181  9145 raft_consensus.cc:2468] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 3.
I20260709 12:21:51.338454  9038 leader_election.cc:304] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: e47ee72c4e114aef9818c7c324ed1938
I20260709 12:21:51.339578  9853 raft_consensus.cc:2804] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:51.354113  9853 raft_consensus.cc:697] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 3 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:21:51.355043  9853 consensus_queue.cc:237] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:51.368855  8898 catalog_manager.cc:5697] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 3, leader changed from e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.544703  9074 raft_consensus.cc:1275] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 5 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:21:51.546207  9848 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:21:51.601352  9113 consensus_peers.cc:597] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:51.641794  9039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111) [suppressed 366 similar messages]
W20260709 12:21:51.739964  9187 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 50961.6us
W20260709 12:21:51.776930  9113 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:51.782647  9113 consensus_peers.cc:597] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:51.784732  9145 raft_consensus.cc:1275] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:51.793336  9853 consensus_queue.cc:1048] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:51.863025  9039 consensus_peers.cc:597] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:51.932883  9039 consensus_peers.cc:597] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:51.947753  9113 consensus_peers.cc:597] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:51.986135  9039 consensus_peers.cc:597] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.053727  9187 consensus_peers.cc:597] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.074213  9187 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.133822  9187 consensus_peers.cc:597] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.134287  9187 consensus_peers.cc:597] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.139403  9039 consensus_peers.cc:597] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.145455 30820 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 9021)
W20260709 12:21:52.163448  9187 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.187793  9113 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.200623  9113 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.202932  9113 consensus_peers.cc:597] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.259999  9187 consensus_peers.cc:597] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.264478  9187 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.278301  9039 consensus_peers.cc:597] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.286371  9039 consensus_peers.cc:597] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.290951  9039 consensus_peers.cc:597] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.336210  9039 consensus_peers.cc:597] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.356909  9187 consensus_peers.cc:597] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.380780  9187 consensus_peers.cc:597] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.388741  9039 consensus_peers.cc:597] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.440826  9187 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.493902  9187 consensus_peers.cc:597] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.526880  9187 consensus_peers.cc:597] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.529265  9039 consensus_peers.cc:597] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.538848  9113 consensus_peers.cc:597] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.551189  9113 consensus_peers.cc:597] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.572563  9039 consensus_peers.cc:597] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.587087  9187 consensus_peers.cc:597] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.620497  9113 consensus_peers.cc:597] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.696326  9039 consensus_peers.cc:597] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.702171  9187 consensus_peers.cc:597] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.707558  9187 consensus_peers.cc:597] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.733230  9039 consensus_peers.cc:597] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.744962  9113 consensus_peers.cc:597] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.789389  9187 consensus_peers.cc:597] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.799675  9039 consensus_peers.cc:597] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:52.805835  9039 consensus_peers.cc:597] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.811125  9039 consensus_peers.cc:597] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:52.823138  9039 consensus_peers.cc:597] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:53.051110  9039 consensus_peers.cc:597] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:53.146051 30820 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 9021)
W20260709 12:21:54.140645  9113 consensus_peers.cc:597] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.146554 30820 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 9021)
W20260709 12:21:54.214452  9113 consensus_peers.cc:597] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:54.306931  9039 consensus_peers.cc:597] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.356839  9187 consensus_peers.cc:597] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:54.364349  9113 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.379906  9113 consensus_peers.cc:597] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.384298  9039 consensus_peers.cc:597] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.394269  9039 consensus_peers.cc:597] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.473028  9187 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.498023  9187 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.518889  9187 consensus_peers.cc:597] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:54.558528  9039 consensus_peers.cc:597] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.664028  9113 consensus_peers.cc:597] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.677634  9039 consensus_peers.cc:597] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:54.688238  9187 consensus_peers.cc:597] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:54.686379  9113 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.702833  9039 consensus_peers.cc:597] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.743103  9113 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.754418  9039 consensus_peers.cc:597] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.805337  9187 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.829252  9187 consensus_peers.cc:597] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:54.831002  9039 consensus_peers.cc:597] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.926121  9113 consensus_peers.cc:597] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.930270  9187 consensus_peers.cc:597] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:54.970945  9187 consensus_peers.cc:597] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:54.971436  9187 consensus_peers.cc:597] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:54.982354  9039 consensus_peers.cc:597] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.001716  9187 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.007413  9187 consensus_peers.cc:597] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.027637  9187 consensus_peers.cc:597] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.042047  9039 consensus_peers.cc:597] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.055330  9113 consensus_peers.cc:597] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.070487  9113 consensus_peers.cc:597] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.089146  9039 consensus_peers.cc:597] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.105057  9187 consensus_peers.cc:597] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.143628  9039 consensus_peers.cc:597] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.147095 30820 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 9021)
W20260709 12:21:55.177518  9039 consensus_peers.cc:597] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.182070  9039 consensus_peers.cc:597] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.188669  9187 consensus_peers.cc:597] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.235436  9113 consensus_peers.cc:597] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.328670  9039 consensus_peers.cc:597] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:55.402887  9187 consensus_peers.cc:597] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.435366  9039 consensus_peers.cc:597] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.528383  9039 consensus_peers.cc:597] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:55.555917  9039 consensus_peers.cc:597] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:56.204830 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
I20260709 12:21:56.354676 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:56.389433  9919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:56.400691  9921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:56.404271  9918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:56.404747 30820 server_base.cc:1061] running on GCE node
I20260709 12:21:56.408854 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:56.409157 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:56.409394 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599716409368 us; error 0 us; skew 500 ppm
I20260709 12:21:56.410161 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:56.457755 30820 webserver.cc:533] Webserver started at http://127.30.25.1:42335/ using document root <none> and password file <none>
I20260709 12:21:56.458619 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:56.458954 30820 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:21:56.470356 30820 fs_manager.cc:714] Time spent opening directory manager: real 0.010s	user 0.002s	sys 0.008s
I20260709 12:21:56.488003  9927 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:56.489130 30820 fs_manager.cc:730] Time spent opening block manager: real 0.009s	user 0.001s	sys 0.007s
I20260709 12:21:56.489589 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "e47ee72c4e114aef9818c7c324ed1938"
format_stamp: "Formatted at 2026-07-09 12:21:33 on dist-test-slave-rldg"
I20260709 12:21:56.490180 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:56.515280 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:56.517113 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:56.525326 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:56.540570  9934 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:21:56.642066  9039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111) [suppressed 456 similar messages]
W20260709 12:21:56.701596  9039 consensus_peers.cc:597] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:56.759686  9113 consensus_peers.cc:597] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:56.797065  9113 consensus_peers.cc:597] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:56.801167  9039 consensus_peers.cc:597] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:56.809258  9187 consensus_peers.cc:597] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:56.819324  9113 consensus_peers.cc:597] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:56.837821  9039 consensus_peers.cc:597] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:56.843204 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:21:56.843598 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.306s	user 0.007s	sys 0.006s
I20260709 12:21:56.843909 30820 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:21:56.851255  9934 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:56.881181  9934 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:56.883293  9934 tablet_bootstrap.cc:492] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:56.884073  9934 ts_tablet_manager.cc:1403] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.033s	user 0.002s	sys 0.024s
I20260709 12:21:56.886981  9934 raft_consensus.cc:359] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:56.887784  9934 raft_consensus.cc:740] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.888734  9934 consensus_queue.cc:260] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:56.896116  9934 ts_tablet_manager.cc:1434] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.008s
I20260709 12:21:56.897073  9934 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:56.923756  9113 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:56.968818  9187 consensus_peers.cc:597] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:56.984140  9187 consensus_peers.cc:597] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:56.988408  9039 consensus_peers.cc:597] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.007992  9934 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.009152  9934 tablet_bootstrap.cc:492] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.009948  9934 ts_tablet_manager.cc:1403] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.113s	user 0.015s	sys 0.007s
I20260709 12:21:57.012524  9934 raft_consensus.cc:359] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.013235  9934 raft_consensus.cc:740] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.014338  9934 consensus_queue.cc:260] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.016750  9934 ts_tablet_manager.cc:1434] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:57.017550  9934 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.046079  9113 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.048034  9934 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.049059  9934 tablet_bootstrap.cc:492] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.049997  9934 ts_tablet_manager.cc:1403] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.005s
I20260709 12:21:57.052596  9934 raft_consensus.cc:359] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.054076  9934 raft_consensus.cc:740] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.059290  9934 consensus_queue.cc:260] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
W20260709 12:21:57.061362  9039 consensus_peers.cc:597] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.061630  9934 ts_tablet_manager.cc:1434] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:21:57.062703  9934 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.080106  9934 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.081128  9934 tablet_bootstrap.cc:492] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.081820  9934 ts_tablet_manager.cc:1403] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:21:57.084369  9934 raft_consensus.cc:359] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.085074  9934 raft_consensus.cc:740] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.085716  9934 consensus_queue.cc:260] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.087581  9934 ts_tablet_manager.cc:1434] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:57.088402  9934 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.094705  9187 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.102700  9934 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.103605  9934 tablet_bootstrap.cc:492] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.104166  9934 ts_tablet_manager.cc:1403] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:21:57.106184  9934 raft_consensus.cc:359] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.106921  9934 raft_consensus.cc:740] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.107594  9934 consensus_queue.cc:260] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.109591  9934 ts_tablet_manager.cc:1434] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:21:57.110404  9934 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.112239  9187 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.117193 30820 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:21:57.117551 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.274s	user 0.250s	sys 0.004s
W20260709 12:21:57.134460  9113 consensus_peers.cc:597] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.164582  9934 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.165688  9934 tablet_bootstrap.cc:492] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.166458  9934 ts_tablet_manager.cc:1403] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.056s	user 0.028s	sys 0.027s
I20260709 12:21:57.169379  9934 raft_consensus.cc:359] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.170116  9934 raft_consensus.cc:740] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.171133  9934 consensus_queue.cc:260] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.173280  9934 ts_tablet_manager.cc:1434] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:57.174198  9934 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.178654  9187 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:57.171862  9039 consensus_peers.cc:597] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:57.188683  9113 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.189079  9934 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.190119  9934 tablet_bootstrap.cc:492] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.190864  9934 ts_tablet_manager.cc:1403] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:21:57.193706  9934 raft_consensus.cc:359] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.194471  9934 raft_consensus.cc:740] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.195322  9934 consensus_queue.cc:260] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.197489  9934 ts_tablet_manager.cc:1434] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:21:57.198247  9934 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.216033  9934 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.217073  9934 tablet_bootstrap.cc:492] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.217895  9934 ts_tablet_manager.cc:1403] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
W20260709 12:21:57.218516  9039 consensus_peers.cc:597] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:57.221895  9934 raft_consensus.cc:359] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.223388  9934 raft_consensus.cc:740] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.224077  9934 consensus_queue.cc:260] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.226369  9934 ts_tablet_manager.cc:1434] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:21:57.227181  9934 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.244243  9934 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.245434  9934 tablet_bootstrap.cc:492] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.246198  9934 ts_tablet_manager.cc:1403] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.001s
I20260709 12:21:57.249320  9934 raft_consensus.cc:359] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.250217  9934 raft_consensus.cc:740] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.251209  9934 consensus_queue.cc:260] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.253808  9934 ts_tablet_manager.cc:1434] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:57.254828  9934 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.263516  9039 consensus_peers.cc:597] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:57.277308  9934 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.278439  9934 tablet_bootstrap.cc:492] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.279181  9934 ts_tablet_manager.cc:1403] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 12:21:57.281929  9934 raft_consensus.cc:359] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:57.282675  9934 raft_consensus.cc:740] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.283504  9934 consensus_queue.cc:260] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:21:57.285619  9934 ts_tablet_manager.cc:1434] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:57.286558  9934 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.304785  9934 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.305523  9934 tablet_bootstrap.cc:492] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.306066  9934 ts_tablet_manager.cc:1403] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
W20260709 12:21:57.306836  9039 consensus_peers.cc:597] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.308558  9934 raft_consensus.cc:359] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.309243  9934 raft_consensus.cc:740] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.310029  9934 consensus_queue.cc:260] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.312047  9934 ts_tablet_manager.cc:1434] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:21:57.312834  9934 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.322890 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:34461
I20260709 12:21:57.323124 10005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:34461 every 8 connection(s)
I20260709 12:21:57.352661  9934 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:21:57.357437  9039 consensus_peers.cc:597] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.358881  9934 tablet_bootstrap.cc:492] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.359854  9934 ts_tablet_manager.cc:1403] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.011s
I20260709 12:21:57.363749  9934 raft_consensus.cc:359] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.364518  9934 raft_consensus.cc:740] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.365710  9934 consensus_queue.cc:260] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.367401  9934 ts_tablet_manager.cc:1434] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:57.368327  9934 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.375994  9979 raft_consensus.cc:3060] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.384768  9934 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.386834 10007 heartbeater.cc:344] Connected to a master server at 127.30.25.62:37527
I20260709 12:21:57.386996  9934 tablet_bootstrap.cc:492] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.387431 10007 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:57.387826  9934 ts_tablet_manager.cc:1403] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 12:21:57.388386 10007 heartbeater.cc:507] Master 127.30.25.62:37527 requested a full tablet report, sending...
I20260709 12:21:57.390959  9934 raft_consensus.cc:359] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.391726  9934 raft_consensus.cc:740] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.392558  9934 consensus_queue.cc:260] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.395378  9934 ts_tablet_manager.cc:1434] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 12:21:57.395643  9113 consensus_peers.cc:597] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.396253  9934 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.409391  8898 ts_manager.cc:194] Re-registered known tserver with Master: e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461)
W20260709 12:21:57.412982  9187 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.416188  9934 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.418212  9934 tablet_bootstrap.cc:492] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.418915  9934 ts_tablet_manager.cc:1403] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 12:21:57.421554  9934 raft_consensus.cc:359] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.422268  9934 raft_consensus.cc:740] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.423009  9934 consensus_queue.cc:260] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.425186  9934 ts_tablet_manager.cc:1434] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:57.425967  9934 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.443241  9934 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.443113  9979 raft_consensus.cc:3060] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.444466  9934 tablet_bootstrap.cc:492] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.445168  9934 ts_tablet_manager.cc:1403] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:21:57.448735  9934 raft_consensus.cc:359] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.449527  9934 raft_consensus.cc:740] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.450387  9934 consensus_queue.cc:260] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.456372  9934 ts_tablet_manager.cc:1434] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:21:57.457193  9934 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.458194  8898 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38470
I20260709 12:21:57.460246  9978 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.465766 10007 heartbeater.cc:499] Master 127.30.25.62:37527 was elected leader, sending a full tablet report...
I20260709 12:21:57.477391  9934 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.478578  9934 tablet_bootstrap.cc:492] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.479221  9934 ts_tablet_manager.cc:1403] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.008s
I20260709 12:21:57.481714  9934 raft_consensus.cc:359] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.482450  9934 raft_consensus.cc:740] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.483179  9934 consensus_queue.cc:260] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.485528  9934 ts_tablet_manager.cc:1434] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:57.486443  9934 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.495334  9187 consensus_peers.cc:597] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:57.502750  9934 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.503759  9934 tablet_bootstrap.cc:492] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.504433  9934 ts_tablet_manager.cc:1403] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 12:21:57.507148  9934 raft_consensus.cc:359] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.507882  9934 raft_consensus.cc:740] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.508599  9934 consensus_queue.cc:260] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.510766  9934 ts_tablet_manager.cc:1434] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:21:57.511618  9934 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.526511  9934 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.527246  9934 tablet_bootstrap.cc:492] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.527767  9934 ts_tablet_manager.cc:1403] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.008s
I20260709 12:21:57.529666  9934 raft_consensus.cc:359] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.530201  9934 raft_consensus.cc:740] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.530846  9934 consensus_queue.cc:260] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.532683  9934 ts_tablet_manager.cc:1434] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:57.533480  9934 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.548069  9934 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.549106  9934 tablet_bootstrap.cc:492] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.549957  9934 ts_tablet_manager.cc:1403] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:21:57.552486  9934 raft_consensus.cc:359] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.553208  9934 raft_consensus.cc:740] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.553903  9934 consensus_queue.cc:260] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.557124  9934 ts_tablet_manager.cc:1434] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:57.557906  9934 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.563716  9187 consensus_peers.cc:597] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.584817  9934 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.585732  9934 tablet_bootstrap.cc:492] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.586393  9934 ts_tablet_manager.cc:1403] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.005s
I20260709 12:21:57.588201  9934 raft_consensus.cc:359] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.588730  9934 raft_consensus.cc:740] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.589329  9934 consensus_queue.cc:260] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.591842  9934 ts_tablet_manager.cc:1434] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:21:57.593060  9934 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.608722  9934 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.609642  9934 tablet_bootstrap.cc:492] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.610409  9934 ts_tablet_manager.cc:1403] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:21:57.613209  9934 raft_consensus.cc:359] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.613917  9934 raft_consensus.cc:740] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.615208  9979 raft_consensus.cc:3060] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.616562  9934 consensus_queue.cc:260] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.619345  9934 ts_tablet_manager.cc:1434] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:57.620206  9934 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.641582  9934 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.642437  9934 tablet_bootstrap.cc:492] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.643110  9934 ts_tablet_manager.cc:1403] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 12:21:57.645589  9934 raft_consensus.cc:359] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.646230  9934 raft_consensus.cc:740] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.646939  9934 consensus_queue.cc:260] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.648908  9934 ts_tablet_manager.cc:1434] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:57.649713  9934 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
W20260709 12:21:57.655674  9039 consensus_peers.cc:597] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:57.664391  9934 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.665325  9934 tablet_bootstrap.cc:492] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.665889  9934 ts_tablet_manager.cc:1403] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:21:57.667743  9934 raft_consensus.cc:359] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.668236  9934 raft_consensus.cc:740] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.668892  9934 consensus_queue.cc:260] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.672555  9979 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.673641  9934 ts_tablet_manager.cc:1434] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:57.674872  9934 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.691839  9934 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.692961  9934 tablet_bootstrap.cc:492] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.693739  9934 ts_tablet_manager.cc:1403] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.001s
I20260709 12:21:57.695693  9934 raft_consensus.cc:359] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.696395  9934 raft_consensus.cc:740] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.697113  9934 consensus_queue.cc:260] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.699241  9934 ts_tablet_manager.cc:1434] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:21:57.700047  9934 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.711884  9979 raft_consensus.cc:3060] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.714180  9934 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.715042  9934 tablet_bootstrap.cc:492] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.715632  9934 ts_tablet_manager.cc:1403] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.007s
I20260709 12:21:57.728521  9934 raft_consensus.cc:359] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.729283  9934 raft_consensus.cc:740] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.730468  9934 consensus_queue.cc:260] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.732580  9934 ts_tablet_manager.cc:1434] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:21:57.733417  9934 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.734414  9976 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:57.748383  9039 consensus_peers.cc:597] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:57.749428  9039 consensus_peers.cc:597] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:57.756208  9934 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.758354  9934 tablet_bootstrap.cc:492] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.759071  9934 ts_tablet_manager.cc:1403] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.000s
I20260709 12:21:57.761617  9934 raft_consensus.cc:359] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.762293  9934 raft_consensus.cc:740] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.763026  9934 consensus_queue.cc:260] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.765164  9934 ts_tablet_manager.cc:1434] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:21:57.766016  9934 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.779742  9934 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.780673  9934 tablet_bootstrap.cc:492] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.781378  9934 ts_tablet_manager.cc:1403] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:21:57.783790  9934 raft_consensus.cc:359] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.784479  9934 raft_consensus.cc:740] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.785142  9934 consensus_queue.cc:260] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.787286  9934 ts_tablet_manager.cc:1434] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:21:57.788070  9934 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.801517  9934 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.802448  9934 tablet_bootstrap.cc:492] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.803098  9934 ts_tablet_manager.cc:1403] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:21:57.804993  9934 raft_consensus.cc:359] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.805433  9934 raft_consensus.cc:740] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.805862  9934 consensus_queue.cc:260] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.807452  9934 ts_tablet_manager.cc:1434] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:57.808144  9934 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.823017  9934 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.824461  9934 tablet_bootstrap.cc:492] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
W20260709 12:21:57.825029  9113 consensus_peers.cc:597] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:57.825129  9934 ts_tablet_manager.cc:1403] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:21:57.827471  9934 raft_consensus.cc:359] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.828091  9934 raft_consensus.cc:740] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.828588  9934 consensus_queue.cc:260] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.830495  9934 ts_tablet_manager.cc:1434] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:21:57.831501  9934 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.833065  9979 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.860567  9934 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.861588  9934 tablet_bootstrap.cc:492] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.862299  9934 ts_tablet_manager.cc:1403] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.000s
I20260709 12:21:57.864315  9934 raft_consensus.cc:359] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.864926  9934 raft_consensus.cc:740] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.865628  9934 consensus_queue.cc:260] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.867843  9934 ts_tablet_manager.cc:1434] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:57.868680  9934 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.892899  9934 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.893936  9934 tablet_bootstrap.cc:492] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.895627  9934 ts_tablet_manager.cc:1403] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.000s
I20260709 12:21:57.902614  9934 raft_consensus.cc:359] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.903343  9934 raft_consensus.cc:740] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.904088  9934 consensus_queue.cc:260] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.905689  9934 ts_tablet_manager.cc:1434] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:21:57.906569  9934 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.932232  9934 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.933347  9934 tablet_bootstrap.cc:492] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.934033  9934 ts_tablet_manager.cc:1403] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 12:21:57.937719  9934 raft_consensus.cc:359] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.938599  9934 raft_consensus.cc:740] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.939397  9934 consensus_queue.cc:260] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:57.941583  9934 ts_tablet_manager.cc:1434] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:57.942461  9934 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.957630  9934 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.958493  9934 tablet_bootstrap.cc:492] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.959079  9934 ts_tablet_manager.cc:1403] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:21:57.960865  9934 raft_consensus.cc:359] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.961387  9934 raft_consensus.cc:740] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.961974  9934 consensus_queue.cc:260] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:57.963753  9934 ts_tablet_manager.cc:1434] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:57.964557  9934 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:57.990024  9934 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:57.990829  9934 tablet_bootstrap.cc:492] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:57.991400  9934 ts_tablet_manager.cc:1403] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.017s
I20260709 12:21:57.993429  9934 raft_consensus.cc:359] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.994050  9934 raft_consensus.cc:740] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.994738  9934 consensus_queue.cc:260] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:57.996492  9934 ts_tablet_manager.cc:1434] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:57.997653  9934 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.012795  9934 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.014930  9934 tablet_bootstrap.cc:492] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.015645  9934 ts_tablet_manager.cc:1403] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:21:58.018026  9934 raft_consensus.cc:359] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.018726  9934 raft_consensus.cc:740] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.019430  9934 consensus_queue.cc:260] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.021430  9934 ts_tablet_manager.cc:1434] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:58.022219  9934 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.043287  9934 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.044225  9934 tablet_bootstrap.cc:492] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.044812  9934 ts_tablet_manager.cc:1403] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.007s
I20260709 12:21:58.047567  9934 raft_consensus.cc:359] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.048280  9934 raft_consensus.cc:740] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.048866  9934 consensus_queue.cc:260] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.050886  9934 ts_tablet_manager.cc:1434] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:58.051782  9934 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.066128  9934 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.067037  9934 tablet_bootstrap.cc:492] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.067570  9934 ts_tablet_manager.cc:1403] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:21:58.069869  9934 raft_consensus.cc:359] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.070621  9934 raft_consensus.cc:740] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.071382  9934 consensus_queue.cc:260] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.071830  9979 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:58.073541  9934 ts_tablet_manager.cc:1434] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:58.074667  9934 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.090987  9934 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.092003  9934 tablet_bootstrap.cc:492] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.092742  9934 ts_tablet_manager.cc:1403] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:21:58.096094  9934 raft_consensus.cc:359] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:58.096892  9934 raft_consensus.cc:740] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.097694  9934 consensus_queue.cc:260] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:58.100113  9934 ts_tablet_manager.cc:1434] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:58.101025  9934 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.122720  9934 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.123769  9934 tablet_bootstrap.cc:492] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.124666  9934 ts_tablet_manager.cc:1403] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 12:21:58.127107  9934 raft_consensus.cc:359] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.127676  9934 raft_consensus.cc:740] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.128274  9934 consensus_queue.cc:260] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.130340  9934 ts_tablet_manager.cc:1434] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:21:58.131505  9934 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.135682  9979 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:21:58.147825  9934 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.148928  9934 tablet_bootstrap.cc:492] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.149802  9934 ts_tablet_manager.cc:1403] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:21:58.152613  9934 raft_consensus.cc:359] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.153165  9934 raft_consensus.cc:740] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.153798  9934 consensus_queue.cc:260] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.155594  9934 ts_tablet_manager.cc:1434] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:58.156500  9934 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.178972  9934 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.184194  9934 tablet_bootstrap.cc:492] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.186045  9934 ts_tablet_manager.cc:1403] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.004s
I20260709 12:21:58.190014  9934 raft_consensus.cc:359] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:58.190876  9934 raft_consensus.cc:740] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.191605  9934 consensus_queue.cc:260] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:58.193826  9934 ts_tablet_manager.cc:1434] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:58.197364  9934 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.218485  9934 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.219481  9934 tablet_bootstrap.cc:492] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.220096  9934 ts_tablet_manager.cc:1403] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:21:58.222623  9934 raft_consensus.cc:359] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.223148  9934 raft_consensus.cc:740] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.223804  9934 consensus_queue.cc:260] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:21:58.225911  9934 ts_tablet_manager.cc:1434] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:58.227046  9934 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.240658  9934 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.241616  9934 tablet_bootstrap.cc:492] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.242403  9979 raft_consensus.cc:3060] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:58.242750  9934 ts_tablet_manager.cc:1403] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:21:58.245853  9934 raft_consensus.cc:359] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.246567  9934 raft_consensus.cc:740] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.247243  9934 consensus_queue.cc:260] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.251183  9934 ts_tablet_manager.cc:1434] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:58.252043  9934 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.266809  9934 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.267694  9934 tablet_bootstrap.cc:492] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.268239  9934 ts_tablet_manager.cc:1403] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 12:21:58.271240  9934 raft_consensus.cc:359] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.272018  9934 raft_consensus.cc:740] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.272833  9934 consensus_queue.cc:260] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:21:58.275152  9934 ts_tablet_manager.cc:1434] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:21:58.278725  9934 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap starting.
I20260709 12:21:58.278991  9976 raft_consensus.cc:3060] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:21:58.367863  9038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:35913, user_credentials={real_user=slave}} blocked reactor thread for 79119.2us
I20260709 12:21:58.394037  9976 raft_consensus.cc:3060] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:58.397858  9113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:34461, user_credentials={real_user=slave}} blocked reactor thread for 109628us
I20260709 12:21:58.404611  9979 raft_consensus.cc:3060] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:58.463306  9977 raft_consensus.cc:3060] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:58.453806  9934 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:58.475359  9934 tablet_bootstrap.cc:492] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Bootstrap complete.
I20260709 12:21:58.499291  9934 ts_tablet_manager.cc:1403] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Time spent bootstrapping tablet: real 0.221s	user 0.017s	sys 0.000s
I20260709 12:21:58.566022  9934 raft_consensus.cc:359] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:58.567795  9934 raft_consensus.cc:740] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.568560  9934 consensus_queue.cc:260] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:21:58.570894  9934 ts_tablet_manager.cc:1434] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938: Time spent starting tablet: real 0.068s	user 0.011s	sys 0.003s
I20260709 12:21:58.759883  9977 raft_consensus.cc:3060] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.351828 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:07.403836  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:07.404594  9069 raft_consensus.cc:606] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:22:07.407222  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:07.407752  9069 raft_consensus.cc:606] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:07.418960  9216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b"
dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:46316
I20260709 12:22:07.419500  9216 raft_consensus.cc:606] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:07.422487  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:07.422960  9069 raft_consensus.cc:606] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:07.429885  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:07.430964  9069 raft_consensus.cc:606] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:07.436165  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:07.436796  9069 raft_consensus.cc:606] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:07.437978 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:07.439637 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:07.440522 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:07.659456 10091 raft_consensus.cc:993] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.660125 10080 raft_consensus.cc:1081] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.661437  9977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:07.661844  9977 raft_consensus.cc:493] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.662264  9977 raft_consensus.cc:3060] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.669518  9977 raft_consensus.cc:515] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.671566  9977 leader_election.cc:290] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:07.683209  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:07.684057  9219 raft_consensus.cc:3060] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.685154  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed17e2a536b342e3b9222a83b27741c8" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:07.685693  9072 raft_consensus.cc:3055] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.685887  9072 raft_consensus.cc:740] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:07.686757  9072 consensus_queue.cc:260] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.687899  9072 raft_consensus.cc:3060] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.690603  9219 raft_consensus.cc:2468] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.691910  9929 leader_election.cc:304] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:07.693699 10098 raft_consensus.cc:2804] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.696133  9072 raft_consensus.cc:2468] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.696728 10098 raft_consensus.cc:697] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:07.697881 10098 consensus_queue.cc:237] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.711963 10080 raft_consensus.cc:993] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.712567 10080 raft_consensus.cc:1081] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.719758  8894 catalog_manager.cc:5697] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:07.720875  9978 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:07.721433  9978 raft_consensus.cc:493] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.721911  9978 raft_consensus.cc:3060] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.731410  9978 raft_consensus.cc:515] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.733654  9978 leader_election.cc:290] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:07.733994  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:07.734637  9219 raft_consensus.cc:3060] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.735595  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cc9d94cd3304974abe8834d2f696077" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:07.736200  9072 raft_consensus.cc:3055] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.736478  9072 raft_consensus.cc:740] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:07.741303  9072 consensus_queue.cc:260] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.742492  9072 raft_consensus.cc:3060] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.743785  9219 raft_consensus.cc:2468] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.745273  9929 leader_election.cc:304] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:07.746145 10098 raft_consensus.cc:2804] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.746628 10098 raft_consensus.cc:697] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:07.747642 10098 consensus_queue.cc:237] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.749500  9072 raft_consensus.cc:2468] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.759670  8894 catalog_manager.cc:5697] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:07.806977 10092 raft_consensus.cc:993] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.807549 10091 raft_consensus.cc:1081] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.809831  9978 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:07.810407  9978 raft_consensus.cc:493] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.810850  9978 raft_consensus.cc:3060] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.817175  9978 raft_consensus.cc:515] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:07.820320 10089 raft_consensus.cc:993] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.820443  9978 leader_election.cc:290] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:22:07.821715 10090 raft_consensus.cc:1081] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.823717  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:07.824290  9072 raft_consensus.cc:3055] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.824570  9072 raft_consensus.cc:740] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:07.825358  9072 consensus_queue.cc:260] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:07.826637  9072 raft_consensus.cc:3060] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.827090  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34640
I20260709 12:22:07.827688  9979 raft_consensus.cc:493] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.828033  9979 raft_consensus.cc:3060] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.834095  9072 raft_consensus.cc:2468] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.836851  9928 leader_election.cc:304] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:07.836845  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b73b5cbf14c728935731f909f32b4" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:07.837769 10098 raft_consensus.cc:2804] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.838297 10098 raft_consensus.cc:697] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:07.839399  9145 raft_consensus.cc:3060] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.839224 10098 consensus_queue.cc:237] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:07.844421 10091 raft_consensus.cc:993] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.845041 10091 raft_consensus.cc:1081] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:07.847179  9977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:07.847681  9977 raft_consensus.cc:493] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.848011  9977 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:07.848910  9145 raft_consensus.cc:2468] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.843659  9979 raft_consensus.cc:515] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:22:07.853731  9979 leader_election.cc:290] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:07.854729  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:07.855341  9143 raft_consensus.cc:3060] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.855834  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39aad2922e1f459a8fbb5dc6e3ac7d7b" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:07.856433  9216 raft_consensus.cc:3055] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.856719  9216 raft_consensus.cc:740] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:22:07.857506  9216 consensus_queue.cc:260] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:22:07.858537  9977 raft_consensus.cc:515] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.858904  9216 raft_consensus.cc:3060] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.860666  9977 leader_election.cc:290] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:07.862964  8894 catalog_manager.cc:5697] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:07.864928  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:07.865694  9219 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:07.871165 10091 raft_consensus.cc:993] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: : Instructing follower aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:07.865020  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7931c26b00244fa1b699e0ce90f2b291" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:07.871948 10092 raft_consensus.cc:1081] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:07.872215  9072 raft_consensus.cc:3055] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:07.872548  9219 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:07.872617  9072 raft_consensus.cc:740] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:07.874348  9072 consensus_queue.cc:260] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.875098  9145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:46454
I20260709 12:22:07.875671  9145 raft_consensus.cc:493] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:07.875681  9143 raft_consensus.cc:2468] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.876077  9145 raft_consensus.cc:3060] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.875681  9072 raft_consensus.cc:3060] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:07.876897  9928 leader_election.cc:304] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:07.877535  9929 leader_election.cc:304] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:07.878492 10098 raft_consensus.cc:2804] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:07.878506 10099 raft_consensus.cc:2804] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.887861  9072 raft_consensus.cc:2468] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:07.890746  9216 raft_consensus.cc:2468] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:07.892285 10099 raft_consensus.cc:697] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:07.893090  9145 raft_consensus.cc:515] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:07.895779  9145 leader_election.cc:290] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:07.893846 10099 consensus_queue.cc:237] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:22:07.896962  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:07.897527  9072 raft_consensus.cc:3055] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:07.897823  9072 raft_consensus.cc:740] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:07.898604 10098 raft_consensus.cc:697] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:07.898737  9216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b89dc1b3074a578d898694dd05a74a" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:07.899333  9216 raft_consensus.cc:3060] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.899525 10098 consensus_queue.cc:237] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:07.898759  9072 consensus_queue.cc:260] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:07.903013  9072 raft_consensus.cc:3060] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:07.905464  9216 raft_consensus.cc:2468] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:07.907462  9113 leader_election.cc:304] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:22:07.908476 10086 raft_consensus.cc:2804] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:07.912847 10086 raft_consensus.cc:697] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:07.916337 10086 consensus_queue.cc:237] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:07.924345  9072 raft_consensus.cc:2468] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:07.937192  8894 catalog_manager.cc:5697] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:07.941352  8898 catalog_manager.cc:5697] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 2 to 3, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 3 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:07.944176  8898 catalog_manager.cc:5697] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:08.130857  9072 raft_consensus.cc:1275] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.132004 10099 consensus_queue.cc:1048] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.143625  9219 raft_consensus.cc:1275] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.145176 10098 consensus_queue.cc:1048] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.290364  9219 raft_consensus.cc:1275] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.291497 10099 consensus_queue.cc:1048] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.301008  9145 raft_consensus.cc:1275] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.303058 10110 consensus_queue.cc:1048] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:08.339540  9219 raft_consensus.cc:1275] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.341063 10098 consensus_queue.cc:1048] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.361083  9072 raft_consensus.cc:1275] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.362550 10111 consensus_queue.cc:1048] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.373560  9219 raft_consensus.cc:1275] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:08.375563 10111 consensus_queue.cc:1048] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:08.394542  9072 raft_consensus.cc:1275] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:08.399612 10111 consensus_queue.cc:1048] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:08.435293  9145 raft_consensus.cc:1275] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.436538 10111 consensus_queue.cc:1048] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.439074  9069 raft_consensus.cc:1275] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.440528 10114 consensus_queue.cc:1048] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.454959  9072 raft_consensus.cc:1275] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.456519 10099 consensus_queue.cc:1048] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.478495 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:08.480901  9219 raft_consensus.cc:1275] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:08.482496 10132 consensus_queue.cc:1048] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:08.550122  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:08.550773  9069 raft_consensus.cc:606] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:22:08.553135  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:08.553685  9069 raft_consensus.cc:606] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:08.556466  9072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31ddf308b7944a55845b113b19560440"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:08.557088  9072 raft_consensus.cc:606] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:22:08.559898  9072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:08.560452  9072 raft_consensus.cc:606] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:08.562609  9219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa"
dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:46316
I20260709 12:22:08.563068  9219 raft_consensus.cc:606] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:08.565356  9072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:08.565923  9072 raft_consensus.cc:606] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS aa2e7a4cc66248d4be460ea6a1fd48f2
I20260709 12:22:08.567170 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:08.568396 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:08.568975 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:08.650745 10107 raft_consensus.cc:993] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:08.651350 10107 raft_consensus.cc:1081] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:08.652930  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34640
I20260709 12:22:08.653488  9979 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.653592 10092 raft_consensus.cc:993] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:08.653905  9979 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.654098 10091 raft_consensus.cc:1081] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:08.655766  9977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:08.656287  9977 raft_consensus.cc:493] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.656577  9977 raft_consensus.cc:3060] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.660419  9979 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:08.662827  9979 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:08.664397 10080 raft_consensus.cc:993] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:08.665215  9977 raft_consensus.cc:515] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:08.666075 10135 raft_consensus.cc:1081] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:08.666335  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:08.667071  9072 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.667869  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:08.668594  9069 raft_consensus.cc:3055] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.668861  9069 raft_consensus.cc:740] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:08.669757  9977 leader_election.cc:290] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:22:08.669832  9069 consensus_queue.cc:260] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:08.671761  9069 raft_consensus.cc:3060] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.672849  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94657c9a3fe64516bd052cd08f818cfa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:08.673612  9219 raft_consensus.cc:3055] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:08.673977  9219 raft_consensus.cc:740] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:22:08.674537  9072 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:08.674927  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "185f6cd295f64fcfa3bf18036f9f75aa" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:08.675453  9143 raft_consensus.cc:3060] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.674960  9219 consensus_queue.cc:260] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:08.677134  9219 raft_consensus.cc:3060] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.678653  9928 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:08.679399 10099 raft_consensus.cc:2804] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:08.679332  9145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:46454
I20260709 12:22:08.680116 10099 raft_consensus.cc:697] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:08.680186  9145 raft_consensus.cc:493] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.680729  9145 raft_consensus.cc:3060] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.681046 10099 consensus_queue.cc:237] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:08.684594  9143 raft_consensus.cc:2468] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:08.685686  9928 leader_election.cc:304] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:08.686630 10098 raft_consensus.cc:2804] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.687096 10098 raft_consensus.cc:697] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:08.687943 10098 consensus_queue.cc:237] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:08.688233  9145 raft_consensus.cc:515] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.689975  9219 raft_consensus.cc:2468] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:08.691533  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:08.696077  9072 raft_consensus.cc:3055] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.697425  9072 raft_consensus.cc:740] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:08.697777  9145 leader_election.cc:290] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:08.699725  9072 consensus_queue.cc:260] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.700412  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3459d33314340e1b9fa3217c09920f5" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:08.701377  9219 raft_consensus.cc:3060] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.703239  9072 raft_consensus.cc:3060] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.704317  9069 raft_consensus.cc:2468] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:08.708084  9219 raft_consensus.cc:2468] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:08.707479  8898 catalog_manager.cc:5697] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 2 to 3, leader changed from 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 3 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.709355  9113 leader_election.cc:304] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:22:08.710775  9072 raft_consensus.cc:2468] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:08.711055 10086 raft_consensus.cc:2804] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.711555 10086 raft_consensus.cc:697] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:08.712574 10086 consensus_queue.cc:237] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.723875  8894 catalog_manager.cc:5697] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.731012  8898 catalog_manager.cc:5697] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.863771 10134 raft_consensus.cc:993] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: : Instructing follower aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:08.864312 10092 raft_consensus.cc:1081] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:08.865705  9143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:46454
I20260709 12:22:08.866209  9143 raft_consensus.cc:493] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.866628  9143 raft_consensus.cc:3060] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.873091  9143 raft_consensus.cc:515] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.875043  9143 leader_election.cc:290] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:08.875618  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:08.875907  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa24aacdc954777b693378dd54a79de" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:08.876224  9072 raft_consensus.cc:3055] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.876559  9219 raft_consensus.cc:3060] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.876581  9072 raft_consensus.cc:740] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:08.877547  9072 consensus_queue.cc:260] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.878713  9072 raft_consensus.cc:3060] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.882196  9219 raft_consensus.cc:2468] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:08.883365  9113 leader_election.cc:304] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:22:08.884030 10086 raft_consensus.cc:2804] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.884478 10086 raft_consensus.cc:697] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:08.885248 10086 consensus_queue.cc:237] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.885743  9072 raft_consensus.cc:2468] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:08.901074  8894 catalog_manager.cc:5697] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.953570 10092 raft_consensus.cc:993] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:08.954116 10134 raft_consensus.cc:1081] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:08.955549  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:08.955983  9979 raft_consensus.cc:493] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.956250  9979 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.960686  9979 raft_consensus.cc:515] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:08.962666  9979 leader_election.cc:290] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:08.963017  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:08.963410  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d74a33940c4299aa703e8d872d9375" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:08.963701  9072 raft_consensus.cc:3055] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:08.964085  9219 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.964139  9072 raft_consensus.cc:740] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:08.964732 10134 raft_consensus.cc:993] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:08.965286 10091 raft_consensus.cc:1081] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer aa2e7a4cc66248d4be460ea6a1fd48f2 to start an election
I20260709 12:22:08.965024  9072 consensus_queue.cc:260] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:08.966374  9072 raft_consensus.cc:3060] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.966804  9143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31ddf308b7944a55845b113b19560440"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
 from {username='slave'} at 127.0.0.1:46454
I20260709 12:22:08.967283  9143 raft_consensus.cc:493] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.967558  9143 raft_consensus.cc:3060] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.970726  9219 raft_consensus.cc:2468] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:08.971966  9929 leader_election.cc:304] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:08.972716 10098 raft_consensus.cc:2804] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:08.973192 10098 raft_consensus.cc:697] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:08.973619  9072 raft_consensus.cc:2468] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:08.973358  9143 raft_consensus.cc:515] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.975004 10098 consensus_queue.cc:237] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:08.976900  9143 leader_election.cc:290] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:08.978233  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:08.978870  9072 raft_consensus.cc:3055] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.979197  9072 raft_consensus.cc:740] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:08.980032  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ddf308b7944a55845b113b19560440" candidate_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:08.980255  9072 consensus_queue.cc:260] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:08.980990  9219 raft_consensus.cc:3060] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.984064  9072 raft_consensus.cc:3060] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.987757  9219 raft_consensus.cc:2468] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:08.988871  9113 leader_election.cc:304] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4, aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 
I20260709 12:22:08.989857 10086 raft_consensus.cc:2804] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.990285 10086 raft_consensus.cc:697] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Becoming Leader. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:08.991088 10086 consensus_queue.cc:237] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:09.008780  9072 raft_consensus.cc:2468] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2e7a4cc66248d4be460ea6a1fd48f2 in term 2.
I20260709 12:22:09.021152  8898 catalog_manager.cc:5697] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.022640  8894 catalog_manager.cc:5697] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 2 to 3, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 3 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.069484  9072 raft_consensus.cc:1275] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.070639 10098 consensus_queue.cc:1048] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.082367  9143 raft_consensus.cc:1275] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.083696 10098 consensus_queue.cc:1048] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.112426  9069 raft_consensus.cc:1275] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.113897 10086 consensus_queue.cc:1048] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.123507  9218 raft_consensus.cc:1275] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.125017 10086 consensus_queue.cc:1048] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.143311  9069 raft_consensus.cc:1275] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:09.146209 10098 consensus_queue.cc:1048] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:09.170674  9218 raft_consensus.cc:1275] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:09.172771 10099 consensus_queue.cc:1048] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:09.396082  9218 raft_consensus.cc:1275] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.397198 10131 consensus_queue.cc:1048] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.407555  9069 raft_consensus.cc:1275] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.409128 10086 consensus_queue.cc:1048] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.431365  9069 raft_consensus.cc:1275] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.432816 10086 consensus_queue.cc:1048] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.453361  9218 raft_consensus.cc:1275] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer aa2e7a4cc66248d4be460ea6a1fd48f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.454725 10086 consensus_queue.cc:1048] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.512391  9218 raft_consensus.cc:1275] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:09.513614 10099 consensus_queue.cc:1048] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:09.522159  9069 raft_consensus.cc:1275] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:09.523787 10144 consensus_queue.cc:1048] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:09.607139 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.650180  9218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39459159540e4827b6f57bb550d02a30"
dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:46316
I20260709 12:22:09.650818  9218 raft_consensus.cc:606] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:09.665838  9145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
mode: GRACEFUL
new_leader_uuid: "0ded781e200f40a1b78076731e570fb7"
 from {username='slave'} at 127.0.0.1:55288
I20260709 12:22:09.666359  9145 raft_consensus.cc:606] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Received request to transfer leadership to TS 0ded781e200f40a1b78076731e570fb7
I20260709 12:22:09.669579  9145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:55288
I20260709 12:22:09.670022  9145 raft_consensus.cc:606] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:09.672340  9219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f"
dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:46316
I20260709 12:22:09.672859  9219 raft_consensus.cc:606] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:09.675243  9069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:09.675724  9069 raft_consensus.cc:606] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:09.676939 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:09.678689 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:09.679436 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.759732 10091 raft_consensus.cc:993] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:09.760187 10134 raft_consensus.cc:1081] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:09.761426  9977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:09.761808  9977 raft_consensus.cc:493] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:09.762039  9977 raft_consensus.cc:3060] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.766578  9977 raft_consensus.cc:515] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:09.768209  9977 leader_election.cc:290] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:22:09.768855  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:09.769212  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fec75531e4924e7b9308bdd3a269b06e" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:09.769567  9069 raft_consensus.cc:3055] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:09.769781  9145 raft_consensus.cc:3060] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.769943  9069 raft_consensus.cc:740] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:09.770768  9069 consensus_queue.cc:260] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:09.771763  9069 raft_consensus.cc:3060] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.774447  9145 raft_consensus.cc:2468] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:09.775774  9928 leader_election.cc:304] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:09.776525 10144 raft_consensus.cc:2804] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:09.776782  9069 raft_consensus.cc:2468] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:09.777098 10144 raft_consensus.cc:697] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:09.778096 10144 consensus_queue.cc:237] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:09.788417  8894 catalog_manager.cc:5697] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.799130 10090 raft_consensus.cc:993] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:09.799628 10090 raft_consensus.cc:1081] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:09.801290  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39459159540e4827b6f57bb550d02a30"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34640
I20260709 12:22:09.801752  9979 raft_consensus.cc:493] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:09.802091  9979 raft_consensus.cc:3060] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.806504  9979 raft_consensus.cc:515] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:22:09.807791  9979 leader_election.cc:290] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:09.808418  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:09.808636  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39459159540e4827b6f57bb550d02a30" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:09.809000  9145 raft_consensus.cc:3060] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.809237  9219 raft_consensus.cc:3055] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:09.809599  9219 raft_consensus.cc:740] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:22:09.810196  9219 consensus_queue.cc:260] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:22:09.811173  9219 raft_consensus.cc:3060] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:09.814177  9145 raft_consensus.cc:2468] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:09.815403  9928 leader_election.cc:304] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:09.816262 10144 raft_consensus.cc:2804] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:09.816758 10144 raft_consensus.cc:697] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:09.817440 10144 consensus_queue.cc:237] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } }
I20260709 12:22:09.827801  9219 raft_consensus.cc:2468] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:09.829537  8894 catalog_manager.cc:5697] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:10.082129 10131 raft_consensus.cc:993] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:10.082661 10086 raft_consensus.cc:1081] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:10.084113  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34612
I20260709 12:22:10.084640  9979 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:10.084975  9979 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.092031  9979 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:10.094733  9979 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 6 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:22:10.094676  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:10.095516  9069 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.096177  9143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70307298bd54a0a8f754e99eb9cfda5" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:10.096810  9143 raft_consensus.cc:3055] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:10.097101  9143 raft_consensus.cc:740] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:10.097891  9143 consensus_queue.cc:260] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:10.099184  9143 raft_consensus.cc:3060] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.102164  9069 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 6.
I20260709 12:22:10.103308  9928 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:10.104198 10144 raft_consensus.cc:2804] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:10.104655 10144 raft_consensus.cc:697] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 6 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:10.105525 10144 consensus_queue.cc:237] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:10.105947  9143 raft_consensus.cc:2468] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 6.
I20260709 12:22:10.107364 10131 raft_consensus.cc:993] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: : Instructing follower 0ded781e200f40a1b78076731e570fb7 to start an election
I20260709 12:22:10.107892 10086 raft_consensus.cc:1081] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Signalling peer 0ded781e200f40a1b78076731e570fb7 to start an election
I20260709 12:22:10.109381  9069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
 from {username='slave'} at 127.0.0.1:38168
I20260709 12:22:10.110002  9069 raft_consensus.cc:493] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:10.110385  9069 raft_consensus.cc:3060] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:10.116593  9069 raft_consensus.cc:515] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:10.117086  8894 catalog_manager.cc:5697] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 5 to 6, leader changed from aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 6 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:10.118907  9069 leader_election.cc:290] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Requested vote from peers aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913), 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651)
I20260709 12:22:10.119447  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:10.119974  9145 raft_consensus.cc:3055] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:10.120262  9145 raft_consensus.cc:740] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:10.120476  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c48e852b04e0099cd6a98a6020814" candidate_uuid: "0ded781e200f40a1b78076731e570fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:10.121024  9219 raft_consensus.cc:3060] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:10.121022  9145 consensus_queue.cc:260] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:10.122484  9145 raft_consensus.cc:3060] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:10.130429  9219 raft_consensus.cc:2468] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:22:10.131897  9039 leader_election.cc:304] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, 343eaafd515a4dfc9de3df299924f0d4; no voters: 
I20260709 12:22:10.133453 10091 raft_consensus.cc:2804] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:10.133906 10091 raft_consensus.cc:697] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:10.134703 10091 consensus_queue.cc:237] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } }
I20260709 12:22:10.135372  9145 raft_consensus.cc:2468] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ded781e200f40a1b78076731e570fb7 in term 2.
I20260709 12:22:10.148881  8894 catalog_manager.cc:5697] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 reported cstate change: term changed from 1 to 2, leader changed from aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3) to 0ded781e200f40a1b78076731e570fb7 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "0ded781e200f40a1b78076731e570fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:10.195806 10171 raft_consensus.cc:993] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:10.196426 10107 raft_consensus.cc:1081] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:10.197940  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34640
I20260709 12:22:10.198529  9979 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:10.198882  9979 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.205291  9979 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:10.207511  9979 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Requested vote from peers 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4:38651), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:10.207706  9219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "343eaafd515a4dfc9de3df299924f0d4"
I20260709 12:22:10.208024  9069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78c5f0841e2455aa811815132278a1f" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:10.208400  9219 raft_consensus.cc:3055] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:10.208631  9069 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.208807  9219 raft_consensus.cc:740] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 343eaafd515a4dfc9de3df299924f0d4, State: Running, Role: LEADER
I20260709 12:22:10.211160  9219 consensus_queue.cc:260] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:10.212333  9219 raft_consensus.cc:3060] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.216173  9069 raft_consensus.cc:2468] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:10.217249  9928 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:10.217972 10144 raft_consensus.cc:2804] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:10.218475 10144 raft_consensus.cc:697] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:10.218608  9219 raft_consensus.cc:2468] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:10.219403 10144 consensus_queue.cc:237] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:10.231612  8894 catalog_manager.cc:5697] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 2 to 3, leader changed from 343eaafd515a4dfc9de3df299924f0d4 (127.30.25.4) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 3 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:10.243320  9145 raft_consensus.cc:1275] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:10.244714 10144 consensus_queue.cc:1048] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:10.267632  9069 raft_consensus.cc:1275] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:10.269294 10111 consensus_queue.cc:1048] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:10.276804  9145 raft_consensus.cc:1275] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:10.278123 10176 consensus_queue.cc:1048] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:10.288350  9219 raft_consensus.cc:1275] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:10.289637 10144 consensus_queue.cc:1048] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:10.567338  9218 raft_consensus.cc:1275] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:10.568632 10091 consensus_queue.cc:1048] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:10.578824  9143 raft_consensus.cc:1275] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer 0ded781e200f40a1b78076731e570fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:10.580240 10135 consensus_queue.cc:1048] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:10.643136  9218 raft_consensus.cc:1275] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:10.644484 10099 consensus_queue.cc:1048] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:10.652789  9069 raft_consensus.cc:1275] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:10.654422 10111 consensus_queue.cc:1048] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:10.714946 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:10.716432  9145 raft_consensus.cc:1275] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:10.717654 10176 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:10.732951  9069 raft_consensus.cc:1275] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 6 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:10.737780 10144 consensus_queue.cc:1048] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:10.775616  9072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94"
dest_uuid: "0ded781e200f40a1b78076731e570fb7"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:35432
I20260709 12:22:10.776130  9072 raft_consensus.cc:606] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:10.778541  9145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0"
dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
mode: GRACEFUL
new_leader_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:55288
I20260709 12:22:10.778946  9145 raft_consensus.cc:606] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Received request to transfer leadership to TS e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:10.779929 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:10.781116 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:10.781708 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:10.856681 10135 raft_consensus.cc:993] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:10.857228 10134 raft_consensus.cc:1081] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:10.858467  9977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:22:10.858853  9977 raft_consensus.cc:493] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:10.859122  9977 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.865336  9977 raft_consensus.cc:515] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:10.867676  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:10.868184  9072 raft_consensus.cc:3055] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:10.868382  9072 raft_consensus.cc:740] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ded781e200f40a1b78076731e570fb7, State: Running, Role: LEADER
I20260709 12:22:10.869063  9072 consensus_queue.cc:260] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:10.869854  9072 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.867182  9977 leader_election.cc:290] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:22:10.873773  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4462b201e045d69b521bfedf999c94" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:10.874462  9145 raft_consensus.cc:3060] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:10.874590  9072 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:10.876046  9928 leader_election.cc:304] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:10.877017 10099 raft_consensus.cc:2804] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:10.877521 10099 raft_consensus.cc:697] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:10.878250 10099 consensus_queue.cc:237] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:10.881178  9145 raft_consensus.cc:2468] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 3.
I20260709 12:22:10.889060  8897 catalog_manager.cc:5697] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 2 to 3, leader changed from 0ded781e200f40a1b78076731e570fb7 (127.30.25.2) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 3 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.136407 10086 raft_consensus.cc:993] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: : Instructing follower e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:11.137079 10183 raft_consensus.cc:1081] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Signalling peer e47ee72c4e114aef9818c7c324ed1938 to start an election
I20260709 12:22:11.138711  9977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0"
dest_uuid: "e47ee72c4e114aef9818c7c324ed1938"
 from {username='slave'} at 127.0.0.1:34612
I20260709 12:22:11.139230  9977 raft_consensus.cc:493] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.139510  9977 raft_consensus.cc:3060] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.144496  9977 raft_consensus.cc:515] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:11.147166  9977 leader_election.cc:290] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Requested vote from peers 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631), aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913)
I20260709 12:22:11.147904  9072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ded781e200f40a1b78076731e570fb7"
I20260709 12:22:11.148475  9072 raft_consensus.cc:3060] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.152109  9145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95536fefe39b43939fac53b1ab01dab0" candidate_uuid: "e47ee72c4e114aef9818c7c324ed1938" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2"
I20260709 12:22:11.152704  9145 raft_consensus.cc:3055] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:11.153004  9145 raft_consensus.cc:740] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa2e7a4cc66248d4be460ea6a1fd48f2, State: Running, Role: LEADER
I20260709 12:22:11.153712  9145 consensus_queue.cc:260] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:11.154932  9145 raft_consensus.cc:3060] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.156042  9072 raft_consensus.cc:2468] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:11.158677  9928 leader_election.cc:304] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938; no voters: 
I20260709 12:22:11.159613 10099 raft_consensus.cc:2804] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:11.160148 10099 raft_consensus.cc:697] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Becoming Leader. State: Replica: e47ee72c4e114aef9818c7c324ed1938, State: Running, Role: LEADER
I20260709 12:22:11.161587 10099 consensus_queue.cc:237] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:11.166584  9145 raft_consensus.cc:2468] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e47ee72c4e114aef9818c7c324ed1938 in term 2.
I20260709 12:22:11.174789  8897 catalog_manager.cc:5697] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 reported cstate change: term changed from 1 to 2, leader changed from aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3) to e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "e47ee72c4e114aef9818c7c324ed1938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.343816  9074 raft_consensus.cc:1275] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:11.345248 10099 consensus_queue.cc:1048] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:11.352365  9145 raft_consensus.cc:1275] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:11.355624 10176 consensus_queue.cc:1048] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:11.590000  9145 raft_consensus.cc:1275] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:11.591434 10198 consensus_queue.cc:1048] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:11.606287  9074 raft_consensus.cc:1275] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Refusing update from remote peer e47ee72c4e114aef9818c7c324ed1938: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:11.607719 10198 consensus_queue.cc:1048] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:11.817286 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:11.852957 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:11.853852 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:11.854274 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.037621 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.098070 30820 tablet_server.cc:179] TabletServer@127.30.25.1:34461 shutting down...
W20260709 12:22:12.115342  9113 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 59 similar messages]
W20260709 12:22:12.125376  9113 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.146035  9187 consensus_peers.cc:597] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.148208 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:12.150270 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.151895 30820 raft_consensus.cc:2243] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.153015 30820 raft_consensus.cc:2272] T ce01c7da004041a58a5a746d2f258ee9 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.157716 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.160594 30820 raft_consensus.cc:2243] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.161396 30820 raft_consensus.cc:2272] T f3f4d49acc064d6ea6cf83616117be16 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.164785 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.165752 30820 raft_consensus.cc:2243] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.166419 30820 raft_consensus.cc:2272] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.169916 30820 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:12.170497  9187 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.170774 30820 raft_consensus.cc:2243] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.172228 30820 raft_consensus.cc:2272] T 1b4462b201e045d69b521bfedf999c94 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.176295 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.177165 30820 raft_consensus.cc:2243] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.177603 30820 raft_consensus.cc:2272] T 013d7d1179ed4bffba8be127fe4a2d03 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.179948 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.180500 30820 raft_consensus.cc:2243] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.180861 30820 raft_consensus.cc:2272] T ed9ca59931ad482da231fc05861e3b93 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.184484 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.185443 30820 raft_consensus.cc:2243] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.186002 30820 raft_consensus.cc:2272] T bdb71a58f7994546bdcf177f0ec7b6b1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.187915  9187 consensus_peers.cc:597] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.189414 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.190673 30820 raft_consensus.cc:2243] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.192265 30820 raft_consensus.cc:2272] T 39459159540e4827b6f57bb550d02a30 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.195765 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.196748 30820 raft_consensus.cc:2243] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.197274 30820 raft_consensus.cc:2272] T d77c1d2e7167495fbe8556f3ba691a55 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.200781  9039 consensus_peers.cc:597] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.200836 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.201740 30820 raft_consensus.cc:2243] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.202416 30820 raft_consensus.cc:2272] T 25135f5b10d44185b9d6be0f2fb5a02c P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.203044  9187 consensus_peers.cc:597] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.205252 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.205869 30820 raft_consensus.cc:2243] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.206261 30820 raft_consensus.cc:2272] T c992391e9d4940a3b550c8c511b354d1 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.208911 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.209452 30820 raft_consensus.cc:2243] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.209848 30820 raft_consensus.cc:2272] T 2bd869b989df40608f17c7091c100fc7 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.212203 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.213003 30820 raft_consensus.cc:2243] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.214221 30820 raft_consensus.cc:2272] T 4cc9d94cd3304974abe8834d2f696077 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.218106 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.218761 30820 raft_consensus.cc:2243] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.219679 30820 raft_consensus.cc:2272] T 185f6cd295f64fcfa3bf18036f9f75aa P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.222525 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.223492 30820 raft_consensus.cc:2243] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.223968 30820 raft_consensus.cc:2272] T 44e11b03c5914827946641c953501ec8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.227105 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.227679 30820 raft_consensus.cc:2243] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.228601 30820 raft_consensus.cc:2272] T 76d74a33940c4299aa703e8d872d9375 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.231118 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.231798 30820 raft_consensus.cc:2243] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.232432 30820 raft_consensus.cc:2272] T 571bcffedf6f4da59792575fd301ed0d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.235945 30820 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:12.236536  9113 consensus_peers.cc:597] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.236738 30820 raft_consensus.cc:2243] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.237377 30820 raft_consensus.cc:2272] T 2426191b92b948e4b607c976696af51f P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.239365  9187 consensus_peers.cc:597] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.240572 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.241161 30820 raft_consensus.cc:2243] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.242250 30820 raft_consensus.cc:2272] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.244655 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.245168 30820 raft_consensus.cc:2243] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.246065 30820 raft_consensus.cc:2272] T fec75531e4924e7b9308bdd3a269b06e P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.248314 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.248878 30820 raft_consensus.cc:2243] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.249750 30820 raft_consensus.cc:2272] T 95536fefe39b43939fac53b1ab01dab0 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.252601 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.253342 30820 raft_consensus.cc:2243] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.254678 30820 raft_consensus.cc:2272] T c70307298bd54a0a8f754e99eb9cfda5 P e47ee72c4e114aef9818c7c324ed1938 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.257802 30820 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:12.258149  9187 consensus_peers.cc:597] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.258544 30820 raft_consensus.cc:2243] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.259140 30820 raft_consensus.cc:2272] T 2e2c3b1da14244ec82ac5d25e49e0862 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.262259 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.263088 30820 raft_consensus.cc:2243] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.264218 30820 raft_consensus.cc:2272] T 293b73b5cbf14c728935731f909f32b4 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.267800 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.268604 30820 raft_consensus.cc:2243] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:12.268600  9039 consensus_peers.cc:597] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.269208 30820 raft_consensus.cc:2272] T 39d47714f6214edbaabd7255530669f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.271552 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.272193 30820 raft_consensus.cc:2243] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.273015 30820 raft_consensus.cc:2272] T ed17e2a536b342e3b9222a83b27741c8 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.275436 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.275998 30820 raft_consensus.cc:2243] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.276700 30820 raft_consensus.cc:2272] T c7f9c182103549958915a7a7ed1eab44 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.280344 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.281042 30820 raft_consensus.cc:2243] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.281759 30820 raft_consensus.cc:2272] T 69899b55ef9441a39d4f4a192d838f17 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.285223 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.287156 30820 raft_consensus.cc:2243] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.287818 30820 raft_consensus.cc:2272] T 98c726df727346b694e0509fae44d789 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.291184 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.291793 30820 raft_consensus.cc:2243] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.292407 30820 raft_consensus.cc:2272] T 46df15ab8dec424780c28700160bbdb2 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.294804 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.295308 30820 raft_consensus.cc:2243] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.295786 30820 raft_consensus.cc:2272] T 149942a6b05d454496753897bc2ed0b8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.296804  9039 consensus_peers.cc:597] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.298640 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.299320 30820 raft_consensus.cc:2243] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.300132 30820 raft_consensus.cc:2272] T 94657c9a3fe64516bd052cd08f818cfa P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.302966 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.303615 30820 raft_consensus.cc:2243] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.305382 30820 raft_consensus.cc:2272] T f78c5f0841e2455aa811815132278a1f P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.308280 30820 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:12.308370  9187 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.309208 30820 raft_consensus.cc:2243] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.309943 30820 raft_consensus.cc:2272] T 72198f72a6de43099cba6530224ef11d P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.312932  9113 consensus_peers.cc:597] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.313320 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.314054 30820 raft_consensus.cc:2243] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.314649 30820 raft_consensus.cc:2272] T bae55b8962494f7582e7d5304cb39d13 P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.317827 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.318658 30820 raft_consensus.cc:2243] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.319557 30820 raft_consensus.cc:2272] T 7931c26b00244fa1b699e0ce90f2b291 P e47ee72c4e114aef9818c7c324ed1938 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.322062 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.322729 30820 raft_consensus.cc:2243] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.323218 30820 raft_consensus.cc:2272] T bec4c3600c2a4b22876a710877cf531d P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.326231 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.326822 30820 raft_consensus.cc:2243] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.327231 30820 raft_consensus.cc:2272] T 1f703417858b43a1b0d7affdd86231f6 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.329279 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.329798 30820 raft_consensus.cc:2243] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.330224 30820 raft_consensus.cc:2272] T eb7ab6e4c736485fbc461ccb19ea580a P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.332556 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.333055 30820 raft_consensus.cc:2243] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.333557 30820 raft_consensus.cc:2272] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P e47ee72c4e114aef9818c7c324ed1938 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.335749 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.336252 30820 raft_consensus.cc:2243] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.336689 30820 raft_consensus.cc:2272] T ba2d24a7495d4a42b94f6f25809db7f8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.338775 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.339267 30820 raft_consensus.cc:2243] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.339725 30820 raft_consensus.cc:2272] T 8984e03dbda34e0fb5f577335d3870ec P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.341841 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.342332 30820 raft_consensus.cc:2243] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.342809 30820 raft_consensus.cc:2272] T 45f96b48cfe14f749d63e8cb99981b8b P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.344779 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.345220 30820 raft_consensus.cc:2243] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.345604 30820 raft_consensus.cc:2272] T b6ee9ad5a10c4a739bf573cebe2cb609 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.348194 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:12.348794 30820 raft_consensus.cc:2243] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.349479 30820 raft_consensus.cc:2272] T 9418597cbc1943eda834dfaa445555a8 P e47ee72c4e114aef9818c7c324ed1938 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.352332  9039 consensus_peers.cc:597] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.353801  9039 consensus_peers.cc:597] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.374874  9113 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.426647  9039 consensus_peers.cc:597] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.432056 30820 tablet_server.cc:196] TabletServer@127.30.25.1:34461 shutdown complete.
W20260709 12:22:12.437479  9113 consensus_peers.cc:597] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.440310  9187 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.443567  9039 consensus_peers.cc:597] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.444154  9039 consensus_peers.cc:597] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.449926  9187 consensus_peers.cc:597] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.458029  9039 consensus_peers.cc:597] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.461299  9187 consensus_peers.cc:597] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.499466  9113 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.508993  9113 consensus_peers.cc:597] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.509419  9113 consensus_peers.cc:597] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.535162 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
W20260709 12:22:12.548377  9187 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.559206  9186 consensus_peers.cc:597] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.563047  9186 consensus_peers.cc:597] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.572959  9186 consensus_peers.cc:597] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.571733  9113 consensus_peers.cc:597] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.576660  9039 consensus_peers.cc:597] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.581915  9112 consensus_peers.cc:597] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.612114  9112 consensus_peers.cc:597] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.615504  9187 consensus_peers.cc:597] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 -> Peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Couldn't send request to peer e47ee72c4e114aef9818c7c324ed1938. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.618242 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:12.619633 30820 tablet_replica.cc:333] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.620817 30820 raft_consensus.cc:2243] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.621479 30820 raft_consensus.cc:2272] T 013d7d1179ed4bffba8be127fe4a2d03 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.624514 30820 tablet_replica.cc:333] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.625228 30820 raft_consensus.cc:2243] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.625716 30820 raft_consensus.cc:2272] T 1b4462b201e045d69b521bfedf999c94 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.627928 30820 tablet_replica.cc:333] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.628881 30820 raft_consensus.cc:2243] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.629415 30820 raft_consensus.cc:2272] T 69899b55ef9441a39d4f4a192d838f17 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.632645 30820 tablet_replica.cc:333] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.633534 30820 raft_consensus.cc:2243] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.634387 30820 raft_consensus.cc:2272] T 30a7af654bae45a4a4bae112448a8932 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.636754 30820 tablet_replica.cc:333] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.637653 30820 raft_consensus.cc:2243] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.638782 30820 raft_consensus.cc:2272] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.641842 30820 tablet_replica.cc:333] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.642608 30820 raft_consensus.cc:2243] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.643172 30820 raft_consensus.cc:2272] T 09b89dc1b3074a578d898694dd05a74a P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.645720 30820 tablet_replica.cc:333] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.646564 30820 raft_consensus.cc:2243] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.647147 30820 raft_consensus.cc:2272] T f78c5f0841e2455aa811815132278a1f P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.650094 30820 tablet_replica.cc:333] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.650846 30820 raft_consensus.cc:2243] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.651422 30820 raft_consensus.cc:2272] T 76d74a33940c4299aa703e8d872d9375 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.654363 30820 tablet_replica.cc:333] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.655246 30820 raft_consensus.cc:2243] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:22:12.655845 10208 consensus_queue.cc:1175] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:22:12.663617 30820 raft_consensus.cc:2272] T eb7ab6e4c736485fbc461ccb19ea580a P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.666744 30820 tablet_replica.cc:333] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.667543 30820 raft_consensus.cc:2243] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.668013 30820 raft_consensus.cc:2272] T 1f703417858b43a1b0d7affdd86231f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.670622 30820 tablet_replica.cc:333] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.671217 30820 raft_consensus.cc:2243] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.672247 30820 raft_consensus.cc:2272] T b6ee9ad5a10c4a739bf573cebe2cb609 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.675295 30820 tablet_replica.cc:333] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.676046 30820 raft_consensus.cc:2243] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:22:12.676627  9186 consensus_peers.cc:597] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.677862 30820 raft_consensus.cc:2272] T 45f96b48cfe14f749d63e8cb99981b8b P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.680528 30820 tablet_replica.cc:333] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.681151 30820 raft_consensus.cc:2243] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.681629 30820 raft_consensus.cc:2272] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.683611 30820 tablet_replica.cc:333] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.684266 30820 raft_consensus.cc:2243] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.685114 30820 raft_consensus.cc:2272] T 6f8c48e852b04e0099cd6a98a6020814 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.687382 30820 tablet_replica.cc:333] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.688009 30820 raft_consensus.cc:2243] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.688511 30820 raft_consensus.cc:2272] T ba2d24a7495d4a42b94f6f25809db7f8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.690706 30820 tablet_replica.cc:333] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.691296 30820 raft_consensus.cc:2243] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.692143 30820 raft_consensus.cc:2272] T 8984e03dbda34e0fb5f577335d3870ec P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.694523 30820 tablet_replica.cc:333] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.695075 30820 raft_consensus.cc:2243] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.695530 30820 raft_consensus.cc:2272] T b81bd1db68c34b918042fd1f8a0d980c P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.697765 30820 tablet_replica.cc:333] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.698408 30820 raft_consensus.cc:2243] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.698906 30820 raft_consensus.cc:2272] T d77c1d2e7167495fbe8556f3ba691a55 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.701197 30820 tablet_replica.cc:333] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.701778 30820 raft_consensus.cc:2243] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.702617 30820 raft_consensus.cc:2272] T bcf8624d5330403dbdebaeaabf396c07 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.704787 30820 tablet_replica.cc:333] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.705353 30820 raft_consensus.cc:2243] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.705788 30820 raft_consensus.cc:2272] T 45855a20f5a4404cb8ba32f93c294288 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.708166 30820 tablet_replica.cc:333] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.708773 30820 raft_consensus.cc:2243] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.709764 30820 raft_consensus.cc:2272] T 2426191b92b948e4b607c976696af51f P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.712462 30820 tablet_replica.cc:333] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.713151 30820 raft_consensus.cc:2243] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.714097 30820 raft_consensus.cc:2272] T 13219c476384456b808b69575b85edcd P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.715523  9186 consensus_peers.cc:597] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.716566 30820 tablet_replica.cc:333] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.717195 30820 raft_consensus.cc:2243] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.717958 30820 raft_consensus.cc:2272] T 39d47714f6214edbaabd7255530669f6 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.720089 30820 tablet_replica.cc:333] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.720592 30820 raft_consensus.cc:2243] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.720983 30820 raft_consensus.cc:2272] T ed17e2a536b342e3b9222a83b27741c8 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.723186 30820 tablet_replica.cc:333] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.723804 30820 raft_consensus.cc:2243] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.724221 30820 raft_consensus.cc:2272] T 31ddf308b7944a55845b113b19560440 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.726168 30820 tablet_replica.cc:333] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.726765 30820 raft_consensus.cc:2243] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.727145 30820 raft_consensus.cc:2272] T 95536fefe39b43939fac53b1ab01dab0 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.728890 30820 tablet_replica.cc:333] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.729405 30820 raft_consensus.cc:2243] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.729756 30820 raft_consensus.cc:2272] T 86996e32eee445b381d0391def34e7b1 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.732409 30820 tablet_replica.cc:333] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.733150 30820 raft_consensus.cc:2243] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.733727 30820 raft_consensus.cc:2272] T fec75531e4924e7b9308bdd3a269b06e P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.736537 30820 tablet_replica.cc:333] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.737274 30820 raft_consensus.cc:2243] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.738186 30820 raft_consensus.cc:2272] T 46df15ab8dec424780c28700160bbdb2 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.740446 30820 tablet_replica.cc:333] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.740972 30820 raft_consensus.cc:2243] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.741509 30820 raft_consensus.cc:2272] T 94657c9a3fe64516bd052cd08f818cfa P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.744427 30820 tablet_replica.cc:333] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.745217 30820 raft_consensus.cc:2243] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.745748 30820 raft_consensus.cc:2272] T 2290f27d484a4682a825fb8d1bb87924 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.748457 30820 tablet_replica.cc:333] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.748992 30820 raft_consensus.cc:2243] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.749482 30820 raft_consensus.cc:2272] T 149942a6b05d454496753897bc2ed0b8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.751868 30820 tablet_replica.cc:333] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.752377 30820 raft_consensus.cc:2243] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.752816 30820 raft_consensus.cc:2272] T 185f6cd295f64fcfa3bf18036f9f75aa P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.754848 30820 tablet_replica.cc:333] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.755545 30820 raft_consensus.cc:2243] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.756002 30820 raft_consensus.cc:2272] T 44e11b03c5914827946641c953501ec8 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.758567 30820 tablet_replica.cc:333] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.759243 30820 raft_consensus.cc:2243] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.759833 30820 raft_consensus.cc:2272] T 4cc9d94cd3304974abe8834d2f696077 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.762444 30820 tablet_replica.cc:333] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.763135 30820 raft_consensus.cc:2243] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.764144 30820 raft_consensus.cc:2272] T bae55b8962494f7582e7d5304cb39d13 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.766243 30820 tablet_replica.cc:333] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.766831 30820 raft_consensus.cc:2243] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.767302 30820 raft_consensus.cc:2272] T 72198f72a6de43099cba6530224ef11d P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.769586 30820 tablet_replica.cc:333] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.770272 30820 raft_consensus.cc:2243] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.770814 30820 raft_consensus.cc:2272] T d3459d33314340e1b9fa3217c09920f5 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.773239 30820 tablet_replica.cc:333] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.773895 30820 raft_consensus.cc:2243] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.774451 30820 raft_consensus.cc:2272] T 0fa24aacdc954777b693378dd54a79de P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.776481 30820 tablet_replica.cc:333] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.776988 30820 raft_consensus.cc:2243] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.777705 30820 raft_consensus.cc:2272] T bec4c3600c2a4b22876a710877cf531d P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.779755 30820 tablet_replica.cc:333] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.780220 30820 raft_consensus.cc:2243] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.780721 30820 raft_consensus.cc:2272] T 293b73b5cbf14c728935731f909f32b4 P 0ded781e200f40a1b78076731e570fb7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.784056  9112 consensus_peers.cc:597] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.784375 30820 tablet_replica.cc:333] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.785095 30820 raft_consensus.cc:2243] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.785676 30820 raft_consensus.cc:2272] T c70307298bd54a0a8f754e99eb9cfda5 P 0ded781e200f40a1b78076731e570fb7 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.788450 30820 tablet_replica.cc:333] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.789067 30820 raft_consensus.cc:2243] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:12.789603 30820 raft_consensus.cc:2272] T 7931c26b00244fa1b699e0ce90f2b291 P 0ded781e200f40a1b78076731e570fb7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:12.792443 30820 tablet_replica.cc:333] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7: stopping tablet replica
I20260709 12:22:12.793043 30820 raft_consensus.cc:2243] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:12.794109 30820 raft_consensus.cc:2272] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 0ded781e200f40a1b78076731e570fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:12.802254  9112 consensus_peers.cc:597] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.813164  9186 consensus_peers.cc:597] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.821043  9112 consensus_peers.cc:597] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.854365  9112 consensus_peers.cc:597] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.875638 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
W20260709 12:22:12.891641  9112 consensus_peers.cc:597] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.920992  9112 consensus_peers.cc:597] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:12.983084 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
W20260709 12:22:12.983307  9112 consensus_peers.cc:597] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:12.999256  9186 consensus_peers.cc:597] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.004616  9186 consensus_peers.cc:597] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.019655  9186 consensus_peers.cc:597] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.044874  9186 consensus_peers.cc:597] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Couldn't send request to peer 0ded781e200f40a1b78076731e570fb7. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.063796 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:13.064909 30820 tablet_replica.cc:333] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.065876 30820 raft_consensus.cc:2243] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:13.066846  9186 consensus_peers.cc:597] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.067346 30820 raft_consensus.cc:2272] T 39459159540e4827b6f57bb550d02a30 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.069867 30820 tablet_replica.cc:333] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.070502 30820 raft_consensus.cc:2243] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.071236 30820 raft_consensus.cc:2272] T 013d7d1179ed4bffba8be127fe4a2d03 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.073161 30820 tablet_replica.cc:333] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.073657 30820 raft_consensus.cc:2243] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.074452 30820 raft_consensus.cc:2272] T 09b89dc1b3074a578d898694dd05a74a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.076373 30820 tablet_replica.cc:333] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.076871 30820 raft_consensus.cc:2243] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.077247 30820 raft_consensus.cc:2272] T f3f4d49acc064d6ea6cf83616117be16 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.079187 30820 tablet_replica.cc:333] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.079722 30820 raft_consensus.cc:2243] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.080438 30820 raft_consensus.cc:2272] T 98c726df727346b694e0509fae44d789 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.082285 30820 tablet_replica.cc:333] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.082876 30820 raft_consensus.cc:2243] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.083587 30820 raft_consensus.cc:2272] T ce01c7da004041a58a5a746d2f258ee9 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.085779 30820 tablet_replica.cc:333] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.086388 30820 raft_consensus.cc:2243] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.086825 30820 raft_consensus.cc:2272] T 1b4462b201e045d69b521bfedf999c94 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.088837 30820 tablet_replica.cc:333] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.089387 30820 raft_consensus.cc:2243] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.089836 30820 raft_consensus.cc:2272] T bae55b8962494f7582e7d5304cb39d13 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.092155 30820 tablet_replica.cc:333] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.092747 30820 raft_consensus.cc:2243] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.093221 30820 raft_consensus.cc:2272] T 2bd869b989df40608f17c7091c100fc7 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.096211 30820 tablet_replica.cc:333] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.097036 30820 raft_consensus.cc:2243] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.097574 30820 raft_consensus.cc:2272] T 185f6cd295f64fcfa3bf18036f9f75aa P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.100243 30820 tablet_replica.cc:333] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.100814 30820 raft_consensus.cc:2243] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.101641 30820 raft_consensus.cc:2272] T b81bd1db68c34b918042fd1f8a0d980c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.103906 30820 tablet_replica.cc:333] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.104477 30820 raft_consensus.cc:2243] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.104847 30820 raft_consensus.cc:2272] T 9418597cbc1943eda834dfaa445555a8 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.106837 30820 tablet_replica.cc:333] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.107352 30820 raft_consensus.cc:2243] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.107724 30820 raft_consensus.cc:2272] T 8f97ca50b64e4e5eb86f9392e8a6d6cd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.109445 30820 tablet_replica.cc:333] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.109946 30820 raft_consensus.cc:2243] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.110500 30820 raft_consensus.cc:2272] T 95536fefe39b43939fac53b1ab01dab0 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.112419 30820 tablet_replica.cc:333] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.112926 30820 raft_consensus.cc:2243] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.113317 30820 raft_consensus.cc:2272] T fec75531e4924e7b9308bdd3a269b06e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.115401 30820 tablet_replica.cc:333] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.115902 30820 raft_consensus.cc:2243] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.116262 30820 raft_consensus.cc:2272] T 86996e32eee445b381d0391def34e7b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.118098 30820 tablet_replica.cc:333] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.118721 30820 raft_consensus.cc:2243] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.119089 30820 raft_consensus.cc:2272] T 45f96b48cfe14f749d63e8cb99981b8b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.121003 30820 tablet_replica.cc:333] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.121513 30820 raft_consensus.cc:2243] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.121848 30820 raft_consensus.cc:2272] T 4507e0b0b1a74cf98c0b22397b2d56f1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.123894 30820 tablet_replica.cc:333] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.124447 30820 raft_consensus.cc:2243] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.124797 30820 raft_consensus.cc:2272] T 6f8c48e852b04e0099cd6a98a6020814 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.126668 30820 tablet_replica.cc:333] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.127167 30820 raft_consensus.cc:2243] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.127519 30820 raft_consensus.cc:2272] T 25135f5b10d44185b9d6be0f2fb5a02c P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.129225 30820 tablet_replica.cc:333] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.129741 30820 raft_consensus.cc:2243] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.130110 30820 raft_consensus.cc:2272] T bdb71a58f7994546bdcf177f0ec7b6b1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.131919 30820 tablet_replica.cc:333] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.132463 30820 raft_consensus.cc:2243] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.132965 30820 raft_consensus.cc:2272] T ed9ca59931ad482da231fc05861e3b93 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.135442 30820 tablet_replica.cc:333] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.136252 30820 raft_consensus.cc:2243] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.137275 30820 raft_consensus.cc:2272] T d77c1d2e7167495fbe8556f3ba691a55 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.139691 30820 tablet_replica.cc:333] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.140251 30820 raft_consensus.cc:2243] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.140719 30820 raft_consensus.cc:2272] T bcf8624d5330403dbdebaeaabf396c07 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.143115 30820 tablet_replica.cc:333] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.143703 30820 raft_consensus.cc:2243] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.144150 30820 raft_consensus.cc:2272] T eb7ab6e4c736485fbc461ccb19ea580a P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.146423 30820 tablet_replica.cc:333] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.146966 30820 raft_consensus.cc:2243] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.147686 30820 raft_consensus.cc:2272] T 1f703417858b43a1b0d7affdd86231f6 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.149609 30820 tablet_replica.cc:333] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.150152 30820 raft_consensus.cc:2243] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.150611 30820 raft_consensus.cc:2272] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.152529 30820 tablet_replica.cc:333] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.153044 30820 raft_consensus.cc:2243] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.154000 30820 raft_consensus.cc:2272] T 45855a20f5a4404cb8ba32f93c294288 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.156910 30820 tablet_replica.cc:333] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.157645 30820 raft_consensus.cc:2243] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.158133 30820 raft_consensus.cc:2272] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.160544 30820 tablet_replica.cc:333] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.161028 30820 raft_consensus.cc:2243] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.161402 30820 raft_consensus.cc:2272] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.163277 30820 tablet_replica.cc:333] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.163739 30820 raft_consensus.cc:2243] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.164700 30820 raft_consensus.cc:2272] T 31ddf308b7944a55845b113b19560440 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.166935 30820 tablet_replica.cc:333] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.167488 30820 raft_consensus.cc:2243] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.167956 30820 raft_consensus.cc:2272] T 8984e03dbda34e0fb5f577335d3870ec P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.169076 10216 raft_consensus.cc:493] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:22:13.169555 10216 raft_consensus.cc:515] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } } peers { permanent_uuid: "aa2e7a4cc66248d4be460ea6a1fd48f2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 35913 } }
I20260709 12:22:13.170063 30820 tablet_replica.cc:333] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.170758 30820 raft_consensus.cc:2243] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.171274 30820 raft_consensus.cc:2272] T 30a7af654bae45a4a4bae112448a8932 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.171547 10216 leader_election.cc:290] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:13.173995 30820 tablet_replica.cc:333] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.174891 30820 raft_consensus.cc:2243] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.175909 30820 raft_consensus.cc:2272] T c992391e9d4940a3b550c8c511b354d1 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.176847  9113 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:22:13.177651  9112 leader_election.cc:336] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111)
I20260709 12:22:13.178064  9112 leader_election.cc:304] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa2e7a4cc66248d4be460ea6a1fd48f2; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:13.178817 30820 tablet_replica.cc:333] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.179040 10216 raft_consensus.cc:2749] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:13.179637 30820 raft_consensus.cc:2243] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.180286 30820 raft_consensus.cc:2272] T 2290f27d484a4682a825fb8d1bb87924 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.182375 30820 tablet_replica.cc:333] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.182906 30820 raft_consensus.cc:2243] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.183331 30820 raft_consensus.cc:2272] T 13219c476384456b808b69575b85edcd P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.184118  9186 consensus_peers.cc:597] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.185243 30820 tablet_replica.cc:333] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.185894 30820 raft_consensus.cc:2243] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.186765 30820 raft_consensus.cc:2272] T c7f9c182103549958915a7a7ed1eab44 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.189446 30820 tablet_replica.cc:333] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
W20260709 12:22:13.189442  9186 consensus_peers.cc:597] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.190011 30820 raft_consensus.cc:2243] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.190546 30820 raft_consensus.cc:2272] T 293b73b5cbf14c728935731f909f32b4 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.192540 30820 tablet_replica.cc:333] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.193050 30820 raft_consensus.cc:2243] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.193578 30820 raft_consensus.cc:2272] T c70307298bd54a0a8f754e99eb9cfda5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.195477 30820 tablet_replica.cc:333] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.195974 30820 raft_consensus.cc:2243] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.196684 30820 raft_consensus.cc:2272] T 2e2c3b1da14244ec82ac5d25e49e0862 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.198544 30820 tablet_replica.cc:333] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.199047 30820 raft_consensus.cc:2243] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.199805 30820 raft_consensus.cc:2272] T d3459d33314340e1b9fa3217c09920f5 P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.201740 30820 tablet_replica.cc:333] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.202245 30820 raft_consensus.cc:2243] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.203009 30820 raft_consensus.cc:2272] T 0fa24aacdc954777b693378dd54a79de P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.204962 30820 tablet_replica.cc:333] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.205459 30820 raft_consensus.cc:2243] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.205853 30820 raft_consensus.cc:2272] T bec4c3600c2a4b22876a710877cf531d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.208382 30820 tablet_replica.cc:333] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2: stopping tablet replica
I20260709 12:22:13.208947 30820 raft_consensus.cc:2243] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.209863 30820 raft_consensus.cc:2272] T 571bcffedf6f4da59792575fd301ed0d P aa2e7a4cc66248d4be460ea6a1fd48f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.270644 10194 raft_consensus.cc:493] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:22:13.271196 10194 raft_consensus.cc:515] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:13.273533 10194 leader_election.cc:290] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
W20260709 12:22:13.277073  9187 leader_election.cc:336] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:22:13.279713  9186 leader_election.cc:336] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111)
I20260709 12:22:13.280112  9186 leader_election.cc:304] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:13.280646 10194 raft_consensus.cc:2749] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:13.290568 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
W20260709 12:22:13.295990  9186 consensus_peers.cc:597] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35913: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.360728 30820 tablet_server.cc:179] TabletServer@127.30.25.4:0 shutting down...
W20260709 12:22:13.389380  9186 consensus_peers.cc:597] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35913: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.398936  9186 consensus_peers.cc:597] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35913: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.400877  9186 consensus_peers.cc:597] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35913: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.408885 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:13.410008 30820 tablet_replica.cc:333] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.411078 30820 raft_consensus.cc:2243] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.411705 30820 raft_consensus.cc:2272] T 39459159540e4827b6f57bb550d02a30 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.414054 30820 tablet_replica.cc:333] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.414681 30820 raft_consensus.cc:2243] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.415488 30820 raft_consensus.cc:2272] T f3f4d49acc064d6ea6cf83616117be16 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.417385 30820 tablet_replica.cc:333] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.417953 30820 raft_consensus.cc:2243] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.418767 30820 raft_consensus.cc:2272] T ba2d24a7495d4a42b94f6f25809db7f8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.420748 30820 tablet_replica.cc:333] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.421304 30820 raft_consensus.cc:2243] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.421749 30820 raft_consensus.cc:2272] T ce01c7da004041a58a5a746d2f258ee9 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.423713 30820 tablet_replica.cc:333] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.424311 30820 raft_consensus.cc:2243] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.424764 30820 raft_consensus.cc:2272] T d3459d33314340e1b9fa3217c09920f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.426981 30820 tablet_replica.cc:333] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.427627 30820 raft_consensus.cc:2243] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.428135 30820 raft_consensus.cc:2272] T 0fa24aacdc954777b693378dd54a79de P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.430224 30820 tablet_replica.cc:333] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.430418 10194 raft_consensus.cc:493] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:22:13.430913 30820 raft_consensus.cc:2243] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.430925 10194 raft_consensus.cc:515] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:13.431485 30820 raft_consensus.cc:2272] T 98c726df727346b694e0509fae44d789 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.433156 10194 leader_election.cc:290] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:13.434203 30820 tablet_replica.cc:333] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.435150 30820 raft_consensus.cc:2243] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.436179 30820 raft_consensus.cc:2272] T 69899b55ef9441a39d4f4a192d838f17 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.437213  9187 leader_election.cc:336] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:22:13.438014  9186 leader_election.cc:336] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111)
I20260709 12:22:13.438390  9186 leader_election.cc:304] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:13.438972 30820 tablet_replica.cc:333] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.438982 10194 raft_consensus.cc:2749] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:13.439857 30820 raft_consensus.cc:2243] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.440677 30820 raft_consensus.cc:2272] T 9418597cbc1943eda834dfaa445555a8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.442772 30820 tablet_replica.cc:333] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.443341 30820 raft_consensus.cc:2243] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.443825 30820 raft_consensus.cc:2272] T 76d74a33940c4299aa703e8d872d9375 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.445796 30820 tablet_replica.cc:333] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.446421 30820 raft_consensus.cc:2243] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.446916 30820 raft_consensus.cc:2272] T 09b89dc1b3074a578d898694dd05a74a P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.448849 30820 tablet_replica.cc:333] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.449421 30820 raft_consensus.cc:2243] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.449880 30820 raft_consensus.cc:2272] T f78c5f0841e2455aa811815132278a1f P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.451889 30820 tablet_replica.cc:333] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.452463 30820 raft_consensus.cc:2243] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.452947 30820 raft_consensus.cc:2272] T 2e2c3b1da14244ec82ac5d25e49e0862 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.453845 10194 raft_consensus.cc:493] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:22:13.454339 10194 raft_consensus.cc:515] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:13.455334 30820 tablet_replica.cc:333] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.456149 30820 raft_consensus.cc:2243] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.456271 10194 leader_election.cc:290] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:13.456797 30820 raft_consensus.cc:2272] T b6ee9ad5a10c4a739bf573cebe2cb609 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.459738 30820 tablet_replica.cc:333] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.460670 30820 raft_consensus.cc:2243] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.461831 30820 raft_consensus.cc:2272] T 86996e32eee445b381d0391def34e7b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.462036  9187 leader_election.cc:336] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
W20260709 12:22:13.463997  9186 leader_election.cc:336] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111)
I20260709 12:22:13.464279 10207 raft_consensus.cc:493] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e47ee72c4e114aef9818c7c324ed1938)
I20260709 12:22:13.464409  9186 leader_election.cc:304] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:13.464736 30820 tablet_replica.cc:333] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.465378 10194 raft_consensus.cc:2749] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:13.464982 10207 raft_consensus.cc:515] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "343eaafd515a4dfc9de3df299924f0d4" member_type: VOTER last_known_addr { host: "127.30.25.4" port: 38651 } } peers { permanent_uuid: "e47ee72c4e114aef9818c7c324ed1938" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 34461 } } peers { permanent_uuid: "0ded781e200f40a1b78076731e570fb7" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 43631 } }
I20260709 12:22:13.465752 30820 raft_consensus.cc:2243] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:22:13.465752  9186 consensus_peers.cc:597] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 -> Peer aa2e7a4cc66248d4be460ea6a1fd48f2 (127.30.25.3:35913): Couldn't send request to peer aa2e7a4cc66248d4be460ea6a1fd48f2. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.3:35913: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.466923 30820 raft_consensus.cc:2272] T 2bd869b989df40608f17c7091c100fc7 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.467080 10207 leader_election.cc:290] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461), 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631)
I20260709 12:22:13.469758 30820 tablet_replica.cc:333] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.470633 30820 raft_consensus.cc:2243] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:13.471172  9186 leader_election.cc:336] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ded781e200f40a1b78076731e570fb7 (127.30.25.2:43631): Network error: Client connection negotiation failed: client connection to 127.30.25.2:43631: connect: Connection refused (error 111)
I20260709 12:22:13.471239 30820 raft_consensus.cc:2272] T 4cc9d94cd3304974abe8834d2f696077 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.471714  9187 leader_election.cc:336] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e47ee72c4e114aef9818c7c324ed1938 (127.30.25.1:34461): Network error: Client connection negotiation failed: client connection to 127.30.25.1:34461: connect: Connection refused (error 111)
I20260709 12:22:13.472126  9187 leader_election.cc:304] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 343eaafd515a4dfc9de3df299924f0d4; no voters: 0ded781e200f40a1b78076731e570fb7, e47ee72c4e114aef9818c7c324ed1938
I20260709 12:22:13.472817 10207 raft_consensus.cc:2749] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:13.474023 30820 tablet_replica.cc:333] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.474715 30820 raft_consensus.cc:2243] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.475514 30820 raft_consensus.cc:2272] T 44e11b03c5914827946641c953501ec8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.477632 30820 tablet_replica.cc:333] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.478497 30820 raft_consensus.cc:2243] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.478919 30820 raft_consensus.cc:2272] T 45855a20f5a4404cb8ba32f93c294288 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.480873 30820 tablet_replica.cc:333] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.481420 30820 raft_consensus.cc:2243] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.481824 30820 raft_consensus.cc:2272] T 2426191b92b948e4b607c976696af51f P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.483759 30820 tablet_replica.cc:333] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.484298 30820 raft_consensus.cc:2243] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.484726 30820 raft_consensus.cc:2272] T 39aad2922e1f459a8fbb5dc6e3ac7d7b P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.486763 30820 tablet_replica.cc:333] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.487325 30820 raft_consensus.cc:2243] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.488060 30820 raft_consensus.cc:2272] T 4507e0b0b1a74cf98c0b22397b2d56f1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.489827 30820 tablet_replica.cc:333] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.490523 30820 raft_consensus.cc:2243] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.490952 30820 raft_consensus.cc:2272] T 6f8c48e852b04e0099cd6a98a6020814 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.492889 30820 tablet_replica.cc:333] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.493435 30820 raft_consensus.cc:2243] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.494175 30820 raft_consensus.cc:2272] T 25135f5b10d44185b9d6be0f2fb5a02c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.496023 30820 tablet_replica.cc:333] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.496548 30820 raft_consensus.cc:2243] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.497283 30820 raft_consensus.cc:2272] T ed9ca59931ad482da231fc05861e3b93 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.499181 30820 tablet_replica.cc:333] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.499704 30820 raft_consensus.cc:2243] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.500456 30820 raft_consensus.cc:2272] T bdb71a58f7994546bdcf177f0ec7b6b1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.502408 30820 tablet_replica.cc:333] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.502928 30820 raft_consensus.cc:2243] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.503459 30820 raft_consensus.cc:2272] T b81bd1db68c34b918042fd1f8a0d980c P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.505798 30820 tablet_replica.cc:333] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.506485 30820 raft_consensus.cc:2243] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.506944 30820 raft_consensus.cc:2272] T bcf8624d5330403dbdebaeaabf396c07 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.508828 30820 tablet_replica.cc:333] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.509339 30820 raft_consensus.cc:2243] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.509763 30820 raft_consensus.cc:2272] T 13219c476384456b808b69575b85edcd P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.511685 30820 tablet_replica.cc:333] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.512197 30820 raft_consensus.cc:2243] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.512600 30820 raft_consensus.cc:2272] T c7f9c182103549958915a7a7ed1eab44 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.514564 30820 tablet_replica.cc:333] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.515081 30820 raft_consensus.cc:2243] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.515496 30820 raft_consensus.cc:2272] T 39d47714f6214edbaabd7255530669f6 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.517211 30820 tablet_replica.cc:333] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.517719 30820 raft_consensus.cc:2243] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.518114 30820 raft_consensus.cc:2272] T ed17e2a536b342e3b9222a83b27741c8 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.519951 30820 tablet_replica.cc:333] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.520517 30820 raft_consensus.cc:2243] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.520912 30820 raft_consensus.cc:2272] T 30a7af654bae45a4a4bae112448a8932 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.522743 30820 tablet_replica.cc:333] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.523231 30820 raft_consensus.cc:2243] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.523630 30820 raft_consensus.cc:2272] T c992391e9d4940a3b550c8c511b354d1 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.525691 30820 tablet_replica.cc:333] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.526203 30820 raft_consensus.cc:2243] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.527029 30820 raft_consensus.cc:2272] T 8fdefe9cd9cb4f9f821cf256b4ec6f6e P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.528983 30820 tablet_replica.cc:333] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.529516 30820 raft_consensus.cc:2243] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.529908 30820 raft_consensus.cc:2272] T 31ddf308b7944a55845b113b19560440 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.531831 30820 tablet_replica.cc:333] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.532339 30820 raft_consensus.cc:2243] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.532735 30820 raft_consensus.cc:2272] T 571bcffedf6f4da59792575fd301ed0d P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.534667 30820 tablet_replica.cc:333] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.535182 30820 raft_consensus.cc:2243] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.535625 30820 raft_consensus.cc:2272] T 7931c26b00244fa1b699e0ce90f2b291 P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.537535 30820 tablet_replica.cc:333] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.538035 30820 raft_consensus.cc:2243] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.538460 30820 raft_consensus.cc:2272] T 2efbde85ba184e82a1ceb3e8ded5d1f5 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.540542 30820 tablet_replica.cc:333] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.541247 30820 raft_consensus.cc:2243] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.541669 30820 raft_consensus.cc:2272] T 46df15ab8dec424780c28700160bbdb2 P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.543608 30820 tablet_replica.cc:333] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.544116 30820 raft_consensus.cc:2243] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.544540 30820 raft_consensus.cc:2272] T 94657c9a3fe64516bd052cd08f818cfa P 343eaafd515a4dfc9de3df299924f0d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.546350 30820 tablet_replica.cc:333] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.546845 30820 raft_consensus.cc:2243] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.547585 30820 raft_consensus.cc:2272] T 2290f27d484a4682a825fb8d1bb87924 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.549376 30820 tablet_replica.cc:333] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.549866 30820 raft_consensus.cc:2243] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.550595 30820 raft_consensus.cc:2272] T 149942a6b05d454496753897bc2ed0b8 P 343eaafd515a4dfc9de3df299924f0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.552270 30820 tablet_replica.cc:333] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4: stopping tablet replica
I20260709 12:22:13.552704 30820 raft_consensus.cc:2243] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.553397 30820 raft_consensus.cc:2272] T 72198f72a6de43099cba6530224ef11d P 343eaafd515a4dfc9de3df299924f0d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.622928 30820 tablet_server.cc:196] TabletServer@127.30.25.4:0 shutdown complete.
I20260709 12:22:13.686964 30820 master.cc:562] Master@127.30.25.62:37527 shutting down...
I20260709 12:22:13.713135 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.713783 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.714143 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7a02ca4df2c94fc2b3ecca76eb0a1a9d: stopping tablet replica
I20260709 12:22:13.769670 30820 master.cc:584] Master@127.30.25.62:37527 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (40915 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:22:13.862582 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:44227
I20260709 12:22:13.863720 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:13.869369 10221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:13.869499 10222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:13.871011 10224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:13.871891 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:13.873095 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:13.873268 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:13.873467 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599733873429 us; error 0 us; skew 500 ppm
I20260709 12:22:13.874002 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:13.876577 30820 webserver.cc:533] Webserver started at http://127.30.25.62:45987/ using document root <none> and password file <none>
I20260709 12:22:13.877079 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:13.877233 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:13.877550 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:13.878759 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "c6955f88fdd24a41b8af969e8ce3db49"
format_stamp: "Formatted at 2026-07-09 12:22:13 on dist-test-slave-rldg"
I20260709 12:22:13.883008 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:22:13.886202 10229 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:13.887003 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:22:13.887277 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "c6955f88fdd24a41b8af969e8ce3db49"
format_stamp: "Formatted at 2026-07-09 12:22:13 on dist-test-slave-rldg"
I20260709 12:22:13.887571 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:13.901216 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:13.902518 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:13.940340 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:44227
I20260709 12:22:13.940464 10280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:44227 every 8 connection(s)
I20260709 12:22:13.944569 10281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:13.955139 10281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49: Bootstrap starting.
I20260709 12:22:13.960227 10281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:13.964447 10281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49: No bootstrap required, opened a new log
I20260709 12:22:13.966562 10281 raft_consensus.cc:359] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6955f88fdd24a41b8af969e8ce3db49" member_type: VOTER }
I20260709 12:22:13.966936 10281 raft_consensus.cc:385] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:13.967113 10281 raft_consensus.cc:740] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6955f88fdd24a41b8af969e8ce3db49, State: Initialized, Role: FOLLOWER
I20260709 12:22:13.967613 10281 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6955f88fdd24a41b8af969e8ce3db49" member_type: VOTER }
I20260709 12:22:13.968026 10281 raft_consensus.cc:399] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:13.968219 10281 raft_consensus.cc:493] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:13.968428 10281 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:13.973062 10281 raft_consensus.cc:515] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6955f88fdd24a41b8af969e8ce3db49" member_type: VOTER }
I20260709 12:22:13.973553 10281 leader_election.cc:304] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6955f88fdd24a41b8af969e8ce3db49; no voters: 
I20260709 12:22:13.974668 10281 leader_election.cc:290] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:13.975104 10284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:13.976552 10284 raft_consensus.cc:697] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 1 LEADER]: Becoming Leader. State: Replica: c6955f88fdd24a41b8af969e8ce3db49, State: Running, Role: LEADER
I20260709 12:22:13.977275 10284 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6955f88fdd24a41b8af969e8ce3db49" member_type: VOTER }
I20260709 12:22:13.977762 10281 sys_catalog.cc:565] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:13.982292 10285 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6955f88fdd24a41b8af969e8ce3db49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6955f88fdd24a41b8af969e8ce3db49" member_type: VOTER } }
I20260709 12:22:13.982481 10286 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6955f88fdd24a41b8af969e8ce3db49. Latest consensus state: current_term: 1 leader_uuid: "c6955f88fdd24a41b8af969e8ce3db49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6955f88fdd24a41b8af969e8ce3db49" member_type: VOTER } }
I20260709 12:22:13.983137 10286 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:13.984478 10285 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:13.985128 10290 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:13.990528 10290 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:13.996320 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:13.999745 10290 catalog_manager.cc:1383] Generated new cluster ID: d1e88cb46f814fbcb401f83c69a02af3
I20260709 12:22:13.999998 10290 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:14.021001 10290 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:14.022383 10290 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:14.041805 10290 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49: Generated new TSK 0
I20260709 12:22:14.042444 10290 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:14.063195 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.069375 10302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.070420 10303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.071938 10305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.072810 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:14.073804 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:14.074039 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:14.074210 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599734074188 us; error 0 us; skew 500 ppm
I20260709 12:22:14.074754 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:14.077009 30820 webserver.cc:533] Webserver started at http://127.30.25.1:39889/ using document root <none> and password file <none>
I20260709 12:22:14.077513 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:14.077723 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:14.078004 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:14.079239 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "8dec85d80d334e7d9f6c904a12fe3913"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:14.083613 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:14.086796 10310 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.087533 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:22:14.087816 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "8dec85d80d334e7d9f6c904a12fe3913"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:14.088117 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:14.113476 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:14.115134 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:14.116658 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:14.119007 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:14.119247 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.119513 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:14.119707 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.161387 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:41877
I20260709 12:22:14.161506 10372 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:41877 every 8 connection(s)
I20260709 12:22:14.166576 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.174489 10377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.176082 10378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.177130 10380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.178608 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:14.179911 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:14.180124 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:14.180302 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599734180284 us; error 0 us; skew 500 ppm
I20260709 12:22:14.180860 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:14.183466 30820 webserver.cc:533] Webserver started at http://127.30.25.2:44757/ using document root <none> and password file <none>
I20260709 12:22:14.183976 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:14.184191 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:14.184496 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:14.185832 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "25e1844b0ccb48adb22cc528da104bca"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:14.185902 10373 heartbeater.cc:344] Connected to a master server at 127.30.25.62:44227
I20260709 12:22:14.186443 10373 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:14.187155 10373 heartbeater.cc:507] Master 127.30.25.62:44227 requested a full tablet report, sending...
I20260709 12:22:14.189373 10246 ts_manager.cc:194] Registered new tserver with Master: 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:14.191234 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:14.191784 10246 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54396
I20260709 12:22:14.195271 10385 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.196079 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:22:14.196384 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "25e1844b0ccb48adb22cc528da104bca"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:14.196655 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:14.215080 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:14.216280 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:14.217717 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:14.219879 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:14.220052 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.220335 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:14.220506 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.261098 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:39477
I20260709 12:22:14.261211 10447 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:39477 every 8 connection(s)
I20260709 12:22:14.266003 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.273221 10451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.274288 10452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.276831 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:14.276701 10454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.278522 10448 heartbeater.cc:344] Connected to a master server at 127.30.25.62:44227
I20260709 12:22:14.278690 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:14.278925 10448 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:14.279026 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:14.279424 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599734279399 us; error 0 us; skew 500 ppm
I20260709 12:22:14.279940 10448 heartbeater.cc:507] Master 127.30.25.62:44227 requested a full tablet report, sending...
I20260709 12:22:14.280040 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:14.282140 10246 ts_manager.cc:194] Registered new tserver with Master: 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:14.282815 30820 webserver.cc:533] Webserver started at http://127.30.25.3:46687/ using document root <none> and password file <none>
I20260709 12:22:14.283391 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:14.283602 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:14.283675 10246 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54406
I20260709 12:22:14.283865 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:14.285251 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:14.290002 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:22:14.293494 10459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.294265 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:14.294683 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:14.295014 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:14.309525 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:14.310803 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:14.312283 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:14.314538 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:14.314776 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.315032 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:14.315215 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:14.355898 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:43805
I20260709 12:22:14.356002 10521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:43805 every 8 connection(s)
I20260709 12:22:14.370379 10522 heartbeater.cc:344] Connected to a master server at 127.30.25.62:44227
I20260709 12:22:14.370810 10522 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:14.371534 10522 heartbeater.cc:507] Master 127.30.25.62:44227 requested a full tablet report, sending...
I20260709 12:22:14.373445 10246 ts_manager.cc:194] Registered new tserver with Master: 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:14.373835 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014462521s
I20260709 12:22:14.374854 10246 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54418
I20260709 12:22:14.400130 10246 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54434:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:14.403059 10246 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:14.475720 10485 tablet_service.cc:1511] Processing CreateTablet for tablet 4faa9dae270549169ff4a40136ebd180 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:14.476867 10487 tablet_service.cc:1511] Processing CreateTablet for tablet 81cf0923653b483f9ba448b97810d958 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:14.477339 10485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4faa9dae270549169ff4a40136ebd180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.478353 10487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cf0923653b483f9ba448b97810d958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.475720 10486 tablet_service.cc:1511] Processing CreateTablet for tablet ebc6723eb06e4d09b65f8d0d739d35dd (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:14.479990 10486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc6723eb06e4d09b65f8d0d739d35dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.484292 10484 tablet_service.cc:1511] Processing CreateTablet for tablet c35fc1d28d25478388654b8b37d694ff (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:14.485755 10484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c35fc1d28d25478388654b8b37d694ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.490483 10483 tablet_service.cc:1511] Processing CreateTablet for tablet 901b273b616542b3a23c6aaf861d7b7a (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:14.491972 10483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 901b273b616542b3a23c6aaf861d7b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.495396 10482 tablet_service.cc:1511] Processing CreateTablet for tablet 643e68f4ae67439581a6a5a3c4df5ed6 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:14.496942 10482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643e68f4ae67439581a6a5a3c4df5ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.503589 10481 tablet_service.cc:1511] Processing CreateTablet for tablet 4556df24f388448b85b996e2f2ebe127 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:14.505124 10481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4556df24f388448b85b996e2f2ebe127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.508432 10480 tablet_service.cc:1511] Processing CreateTablet for tablet 73ec4c2a8f774679a048658e30d5708b (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:14.509900 10480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ec4c2a8f774679a048658e30d5708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.522666 10479 tablet_service.cc:1511] Processing CreateTablet for tablet 1220a04033fc478a909de6d8f46c51bf (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:14.524217 10479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1220a04033fc478a909de6d8f46c51bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.537993 10338 tablet_service.cc:1511] Processing CreateTablet for tablet 81cf0923653b483f9ba448b97810d958 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:14.539521 10338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cf0923653b483f9ba448b97810d958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.541028 10337 tablet_service.cc:1511] Processing CreateTablet for tablet ebc6723eb06e4d09b65f8d0d739d35dd (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:14.542336 10330 tablet_service.cc:1511] Processing CreateTablet for tablet 1220a04033fc478a909de6d8f46c51bf (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:14.543442 10336 tablet_service.cc:1511] Processing CreateTablet for tablet 4faa9dae270549169ff4a40136ebd180 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:14.545720 10335 tablet_service.cc:1511] Processing CreateTablet for tablet c35fc1d28d25478388654b8b37d694ff (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:14.547974 10334 tablet_service.cc:1511] Processing CreateTablet for tablet 901b273b616542b3a23c6aaf861d7b7a (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:14.550130 10333 tablet_service.cc:1511] Processing CreateTablet for tablet 643e68f4ae67439581a6a5a3c4df5ed6 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:14.552080 10329 tablet_service.cc:1511] Processing CreateTablet for tablet 29deab0b8b384dccbed4b383d96eb326 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:14.552470 10331 tablet_service.cc:1511] Processing CreateTablet for tablet 73ec4c2a8f774679a048658e30d5708b (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:14.553707 10337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc6723eb06e4d09b65f8d0d739d35dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.554421 10332 tablet_service.cc:1511] Processing CreateTablet for tablet 4556df24f388448b85b996e2f2ebe127 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:14.555677 10332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4556df24f388448b85b996e2f2ebe127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.574414 10484 tablet_service.cc:1511] Processing CreateTablet for tablet 29deab0b8b384dccbed4b383d96eb326 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:14.576093 10537 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:14.576396 10484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29deab0b8b384dccbed4b383d96eb326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.578450 10330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1220a04033fc478a909de6d8f46c51bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.582247 10413 tablet_service.cc:1511] Processing CreateTablet for tablet 81cf0923653b483f9ba448b97810d958 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:14.583117 10334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 901b273b616542b3a23c6aaf861d7b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.583722 10413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cf0923653b483f9ba448b97810d958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.583827 10329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29deab0b8b384dccbed4b383d96eb326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.587388 10336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4faa9dae270549169ff4a40136ebd180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.590718 10335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c35fc1d28d25478388654b8b37d694ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.592677 10412 tablet_service.cc:1511] Processing CreateTablet for tablet ebc6723eb06e4d09b65f8d0d739d35dd (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:14.594098 10412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc6723eb06e4d09b65f8d0d739d35dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.594472 10331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ec4c2a8f774679a048658e30d5708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.598397 10333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643e68f4ae67439581a6a5a3c4df5ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.603523 10411 tablet_service.cc:1511] Processing CreateTablet for tablet 4faa9dae270549169ff4a40136ebd180 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:14.604982 10411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4faa9dae270549169ff4a40136ebd180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.605729 10538 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:14.607442 10410 tablet_service.cc:1511] Processing CreateTablet for tablet c35fc1d28d25478388654b8b37d694ff (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:14.608850 10410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c35fc1d28d25478388654b8b37d694ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.611441 10409 tablet_service.cc:1511] Processing CreateTablet for tablet 901b273b616542b3a23c6aaf861d7b7a (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:14.612825 10409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 901b273b616542b3a23c6aaf861d7b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.615847 10408 tablet_service.cc:1511] Processing CreateTablet for tablet 643e68f4ae67439581a6a5a3c4df5ed6 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:14.617801 10408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643e68f4ae67439581a6a5a3c4df5ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.620733 10407 tablet_service.cc:1511] Processing CreateTablet for tablet 4556df24f388448b85b996e2f2ebe127 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:14.624859 10406 tablet_service.cc:1511] Processing CreateTablet for tablet 73ec4c2a8f774679a048658e30d5708b (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:14.628922 10405 tablet_service.cc:1511] Processing CreateTablet for tablet 1220a04033fc478a909de6d8f46c51bf (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:14.632524 10404 tablet_service.cc:1511] Processing CreateTablet for tablet 29deab0b8b384dccbed4b383d96eb326 (DEFAULT_TABLE table=test-workload [id=5c8e0dfabdcd47c59d1fa28d1ad32a84]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:14.634299 10407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4556df24f388448b85b996e2f2ebe127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.646878 10404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29deab0b8b384dccbed4b383d96eb326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.647535 10537 tablet_bootstrap.cc:654] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.650858 10406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ec4c2a8f774679a048658e30d5708b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.652721 10405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1220a04033fc478a909de6d8f46c51bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:14.694941 10538 tablet_bootstrap.cc:654] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.791888 10537 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:14.798080 10537 ts_tablet_manager.cc:1403] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.222s	user 0.051s	sys 0.073s
I20260709 12:22:14.800866 10538 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:14.801497 10538 ts_tablet_manager.cc:1403] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.196s	user 0.042s	sys 0.060s
I20260709 12:22:14.801577 10537 raft_consensus.cc:359] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.802454 10537 raft_consensus.cc:385] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.810460 10537 raft_consensus.cc:740] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.811391 10537 consensus_queue.cc:260] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.804590 10538 raft_consensus.cc:359] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.812885 10538 raft_consensus.cc:385] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.813197 10538 raft_consensus.cc:740] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.810081 10539 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:14.814055 10522 heartbeater.cc:499] Master 127.30.25.62:44227 was elected leader, sending a full tablet report...
I20260709 12:22:14.819506 10539 tablet_bootstrap.cc:654] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.819597 10537 ts_tablet_manager.cc:1434] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.021s	user 0.000s	sys 0.006s
I20260709 12:22:14.822022 10537 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:14.820182 10538 consensus_queue.cc:260] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.830682 10537 tablet_bootstrap.cc:654] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.852578 10538 ts_tablet_manager.cc:1434] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.027s
I20260709 12:22:14.852864 10373 heartbeater.cc:499] Master 127.30.25.62:44227 was elected leader, sending a full tablet report...
I20260709 12:22:14.854167 10539 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:14.854787 10539 ts_tablet_manager.cc:1403] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.022s
I20260709 12:22:14.855078 10538 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:14.856387 10537 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:14.856937 10537 ts_tablet_manager.cc:1403] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.035s	user 0.025s	sys 0.009s
I20260709 12:22:14.858428 10539 raft_consensus.cc:359] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:14.859289 10539 raft_consensus.cc:385] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.859622 10539 raft_consensus.cc:740] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
W20260709 12:22:14.860456 10523 tablet.cc:2406] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:14.860610 10539 consensus_queue.cc:260] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:14.860129 10537 raft_consensus.cc:359] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.861485 10538 tablet_bootstrap.cc:654] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.861681 10537 raft_consensus.cc:385] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.862130 10537 raft_consensus.cc:740] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.863227 10537 consensus_queue.cc:260] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.865801 10537 ts_tablet_manager.cc:1434] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:22:14.867137 10537 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:14.872937 10539 ts_tablet_manager.cc:1434] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.008s
I20260709 12:22:14.873771 10448 heartbeater.cc:499] Master 127.30.25.62:44227 was elected leader, sending a full tablet report...
I20260709 12:22:14.874440 10537 tablet_bootstrap.cc:654] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.880620 10542 raft_consensus.cc:493] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.881300 10542 raft_consensus.cc:515] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.884589 10539 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:14.893532 10542 leader_election.cc:290] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:14.896680 10538 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:14.897464 10538 ts_tablet_manager.cc:1403] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.025s
I20260709 12:22:14.900713 10538 raft_consensus.cc:359] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.901520 10538 raft_consensus.cc:385] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.901873 10538 raft_consensus.cc:740] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.902886 10538 consensus_queue.cc:260] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.905339 10539 tablet_bootstrap.cc:654] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.905669 10538 ts_tablet_manager.cc:1434] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:14.907915 10538 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:14.920367 10538 tablet_bootstrap.cc:654] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.920668 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:14.922264 10348 raft_consensus.cc:2468] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:14.924508 10461 leader_election.cc:304] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:14.927583 10537 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:14.928406 10537 ts_tablet_manager.cc:1403] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.062s	user 0.035s	sys 0.019s
I20260709 12:22:14.929436 10542 raft_consensus.cc:2804] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:14.929879 10542 raft_consensus.cc:493] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.930204 10542 raft_consensus.cc:3060] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:14.932567 10537 raft_consensus.cc:359] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.933621 10537 raft_consensus.cc:385] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.933986 10537 raft_consensus.cc:740] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.934295 10539 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:14.934898 10539 ts_tablet_manager.cc:1403] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.051s	user 0.005s	sys 0.010s
I20260709 12:22:14.935058 10537 consensus_queue.cc:260] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.937894 10539 raft_consensus.cc:359] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.938663 10539 raft_consensus.cc:385] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.938975 10539 raft_consensus.cc:740] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.940061 10538 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:14.939818 10542 raft_consensus.cc:515] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.940675 10538 ts_tablet_manager.cc:1403] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.007s
I20260709 12:22:14.943410 10553 raft_consensus.cc:493] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.939927 10539 consensus_queue.cc:260] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.943967 10553 raft_consensus.cc:515] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.944960 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:14.945650 10348 raft_consensus.cc:3060] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:14.945907 10539 ts_tablet_manager.cc:1434] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:22:14.947095 10347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901b273b616542b3a23c6aaf861d7b7a" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:14.947479 10553 leader_election.cc:290] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
W20260709 12:22:14.949689 10461 leader_election.cc:343] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:14.953081 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:14.953368 10539 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:14.953788 10348 raft_consensus.cc:2468] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:14.953802 10423 raft_consensus.cc:2468] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:14.944782 10538 raft_consensus.cc:359] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.954885 10538 raft_consensus.cc:385] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:14.955192 10538 raft_consensus.cc:740] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:14.955477 10461 leader_election.cc:304] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:14.941080 10537 ts_tablet_manager.cc:1434] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:22:14.954294 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901b273b616542b3a23c6aaf861d7b7a" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:14.955955 10538 consensus_queue.cc:260] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:14.957358 10537 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:14.958540 10538 ts_tablet_manager.cc:1434] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 12:22:14.959605 10538 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
W20260709 12:22:14.959606 10460 leader_election.cc:343] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:14.960896 10539 tablet_bootstrap.cc:654] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.960988 10460 leader_election.cc:304] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:14.962899 10542 leader_election.cc:290] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:14.963548 10554 raft_consensus.cc:2749] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:14.964107 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:14.964735 10422 raft_consensus.cc:3060] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:14.966279 10537 tablet_bootstrap.cc:654] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.972397 10538 tablet_bootstrap.cc:654] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:14.956295 10553 raft_consensus.cc:2804] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:14.987399 10553 raft_consensus.cc:697] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:14.994650 10422 raft_consensus.cc:2468] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
W20260709 12:22:14.995088 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:14.990940 10538 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:14.998848 10538 ts_tablet_manager.cc:1403] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.002s
I20260709 12:22:14.988595 10553 consensus_queue.cc:237] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.003507 10538 raft_consensus.cc:359] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.004575 10538 raft_consensus.cc:385] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.004969 10538 raft_consensus.cc:740] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.005903 10538 consensus_queue.cc:260] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.007072 10553 raft_consensus.cc:493] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.008311 10553 raft_consensus.cc:515] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.009662 10538 ts_tablet_manager.cc:1434] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:22:15.010543 10553 leader_election.cc:290] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.010782 10538 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:15.011822 10539 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.012367 10539 ts_tablet_manager.cc:1403] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.060s	user 0.030s	sys 0.021s
I20260709 12:22:15.013123 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4faa9dae270549169ff4a40136ebd180" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.017238 10539 raft_consensus.cc:359] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.017994 10538 tablet_bootstrap.cc:654] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.018194 10539 raft_consensus.cc:385] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.018612 10539 raft_consensus.cc:740] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.018853 10537 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.019382 10537 ts_tablet_manager.cc:1403] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.062s	user 0.026s	sys 0.019s
I20260709 12:22:15.019556 10539 consensus_queue.cc:260] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.022506 10537 raft_consensus.cc:359] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:15.023253 10537 raft_consensus.cc:385] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.023535 10537 raft_consensus.cc:740] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.024325 10537 consensus_queue.cc:260] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
W20260709 12:22:15.018690 10461 leader_election.cc:343] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.017138 10246 catalog_manager.cc:5697] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.027252 10537 ts_tablet_manager.cc:1434] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:15.027860 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4faa9dae270549169ff4a40136ebd180" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.029610 10422 raft_consensus.cc:2468] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.029808 10537 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:15.028301 10539 ts_tablet_manager.cc:1434] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:22:15.031277 10539 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.034605 10460 leader_election.cc:304] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:15.035867 10553 raft_consensus.cc:2804] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.036248 10553 raft_consensus.cc:493] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.036689 10553 raft_consensus.cc:3060] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.038558 10539 tablet_bootstrap.cc:654] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.039768 10537 tablet_bootstrap.cc:654] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.036291 10538 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:15.046797 10538 ts_tablet_manager.cc:1403] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.013s
I20260709 12:22:15.045819 10553 raft_consensus.cc:515] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.053632 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4faa9dae270549169ff4a40136ebd180" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
W20260709 12:22:15.055516 10461 leader_election.cc:343] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.058094 10553 leader_election.cc:290] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.058337 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4faa9dae270549169ff4a40136ebd180" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.058936 10422 raft_consensus.cc:3060] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.058986 10538 raft_consensus.cc:359] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:15.059778 10538 raft_consensus.cc:385] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.060070 10538 raft_consensus.cc:740] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.060859 10538 consensus_queue.cc:260] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:15.063112 10538 ts_tablet_manager.cc:1434] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:22:15.064186 10538 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:15.066668 10422 raft_consensus.cc:2468] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.068027 10460 leader_election.cc:304] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:15.069170 10553 raft_consensus.cc:2804] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.072584 10553 raft_consensus.cc:697] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:15.073145 10539 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.073628 10537 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.073860 10539 ts_tablet_manager.cc:1403] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.020s
I20260709 12:22:15.074275 10537 ts_tablet_manager.cc:1403] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.012s
I20260709 12:22:15.073460 10553 consensus_queue.cc:237] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.077307 10539 raft_consensus.cc:359] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.078121 10539 raft_consensus.cc:385] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.078938 10539 raft_consensus.cc:740] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.078513 10537 raft_consensus.cc:359] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.080715 10539 consensus_queue.cc:260] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.081362 10537 raft_consensus.cc:385] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.082698 10537 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.083078 10539 ts_tablet_manager.cc:1434] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:15.084203 10539 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.083706 10537 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.085572 10538 tablet_bootstrap.cc:654] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.086548 10537 ts_tablet_manager.cc:1434] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:22:15.087668 10537 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:15.092240 10539 tablet_bootstrap.cc:654] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.094089 10538 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:15.094631 10538 ts_tablet_manager.cc:1403] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.003s
I20260709 12:22:15.095155 10537 tablet_bootstrap.cc:654] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.097754 10538 raft_consensus.cc:359] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.098539 10538 raft_consensus.cc:385] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.098857 10538 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.099645 10538 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.101027 10245 catalog_manager.cc:5697] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.102028 10538 ts_tablet_manager.cc:1434] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:22:15.104555 10538 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:15.110453 10539 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.110778 10543 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.110934 10538 tablet_bootstrap.cc:654] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.111001 10539 ts_tablet_manager.cc:1403] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 12:22:15.111313 10543 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:15.114243 10539 raft_consensus.cc:359] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.114876 10537 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.114997 10539 raft_consensus.cc:385] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.115682 10539 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.115918 10537 ts_tablet_manager.cc:1403] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.000s
I20260709 12:22:15.116698 10539 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.119326 10539 ts_tablet_manager.cc:1434] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:22:15.119431 10537 raft_consensus.cc:359] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.120374 10537 raft_consensus.cc:385] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.120533 10539 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.120733 10537 raft_consensus.cc:740] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.121755 10537 consensus_queue.cc:260] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.124468 10537 ts_tablet_manager.cc:1434] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:15.125588 10537 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:15.126998 10539 tablet_bootstrap.cc:654] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.118212 10543 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.133044 10537 tablet_bootstrap.cc:654] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.138008 10546 raft_consensus.cc:493] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.138774 10546 raft_consensus.cc:515] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.140245 10538 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:15.140889 10538 ts_tablet_manager.cc:1403] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.000s
I20260709 12:22:15.143967 10546 leader_election.cc:290] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.144086 10538 raft_consensus.cc:359] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.145098 10538 raft_consensus.cc:385] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.145404 10538 raft_consensus.cc:740] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.147583 10539 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.148329 10539 ts_tablet_manager.cc:1403] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.007s
I20260709 12:22:15.147902 10538 consensus_queue.cc:260] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.161306 10497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.162218 10497 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 0.
I20260709 12:22:15.162508 10538 ts_tablet_manager.cc:1434] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:22:15.163647 10538 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:15.163806 10314 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.164876 10543 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.165242 10543 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.165558 10543 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.168798 10539 raft_consensus.cc:359] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.169584 10539 raft_consensus.cc:385] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.169886 10539 raft_consensus.cc:740] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.174441 10543 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:15.175534 10539 consensus_queue.cc:260] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.177438 10497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:15.178064 10497 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.179524 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.180176 10422 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 0.
I20260709 12:22:15.182336 10538 tablet_bootstrap.cc:654] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.185369 10539 ts_tablet_manager.cc:1434] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 12:22:15.186542 10539 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.187695 10497 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 1.
I20260709 12:22:15.188318 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.189031 10422 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.188202 10543 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Requested vote from peers 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.190440 10314 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.193687 10537 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.193949 10539 tablet_bootstrap.cc:654] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.194401 10537 ts_tablet_manager.cc:1403] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.023s
I20260709 12:22:15.196767 10543 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.196926 10422 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 1.
I20260709 12:22:15.200372 10537 raft_consensus.cc:359] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.201517 10537 raft_consensus.cc:385] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.201990 10537 raft_consensus.cc:740] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.204955 10537 consensus_queue.cc:260] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.205768 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901b273b616542b3a23c6aaf861d7b7a" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.208168 10348 raft_consensus.cc:2468] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 0.
I20260709 12:22:15.208940 10538 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:15.209152 10539 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.205768 10497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901b273b616542b3a23c6aaf861d7b7a" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.209503 10538 ts_tablet_manager.cc:1403] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.046s	user 0.018s	sys 0.001s
I20260709 12:22:15.209686 10539 ts_tablet_manager.cc:1403] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.007s
I20260709 12:22:15.209549 10387 leader_election.cc:304] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.210556 10497 raft_consensus.cc:2468] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 0.
I20260709 12:22:15.210973 10546 raft_consensus.cc:2804] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.211390 10546 raft_consensus.cc:493] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.211759 10546 raft_consensus.cc:3060] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.212946 10538 raft_consensus.cc:359] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.213658 10538 raft_consensus.cc:385] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.213932 10538 raft_consensus.cc:740] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.214440 10543 raft_consensus.cc:697] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 LEADER]: Becoming Leader. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Running, Role: LEADER
I20260709 12:22:15.214784 10538 consensus_queue.cc:260] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.216868 10538 ts_tablet_manager.cc:1434] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:15.215448 10543 consensus_queue.cc:237] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:15.217612 10537 ts_tablet_manager.cc:1434] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:22:15.217845 10538 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:15.218792 10537 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:15.218878 10546 raft_consensus.cc:515] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.221717 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901b273b616542b3a23c6aaf861d7b7a" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:15.222291 10348 raft_consensus.cc:3060] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.224198 10538 tablet_bootstrap.cc:654] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.225415 10537 tablet_bootstrap.cc:654] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.227739 10580 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.228271 10580 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.231190 10539 raft_consensus.cc:359] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.231983 10539 raft_consensus.cc:385] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.232301 10539 raft_consensus.cc:740] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.232860 10497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.233592 10497 raft_consensus.cc:2468] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 0.
I20260709 12:22:15.234648 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.236045 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901b273b616542b3a23c6aaf861d7b7a" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:15.236624 10546 leader_election.cc:290] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.236753 10496 raft_consensus.cc:3060] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.236184 10348 raft_consensus.cc:2468] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 1.
I20260709 12:22:15.238087 10387 leader_election.cc:304] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.239104 10546 raft_consensus.cc:2804] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.240500 10539 consensus_queue.cc:260] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.242435 10580 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.239032 10314 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
W20260709 12:22:15.243896 10311 leader_election.cc:343] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.245229 10539 ts_tablet_manager.cc:1434] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:22:15.246901 10580 raft_consensus.cc:2804] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.247316 10580 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.247702 10580 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.248441 10546 raft_consensus.cc:697] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:15.249375 10546 consensus_queue.cc:237] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.253796 10539 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.259200 10496 raft_consensus.cc:2468] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 1.
I20260709 12:22:15.261863 10580 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.267400 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.267817 10539 tablet_bootstrap.cc:654] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:15.268908 10311 leader_election.cc:343] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.271842 10245 catalog_manager.cc:5697] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 0 to 1, leader changed from <none> to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 1 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.263309 10538 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:15.276444 10538 ts_tablet_manager.cc:1403] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.020s
I20260709 12:22:15.279572 10538 raft_consensus.cc:359] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.279717 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:15.281390 10496 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.279520 10244 catalog_manager.cc:5697] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "8dec85d80d334e7d9f6c904a12fe3913" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.284034 10580 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Requested vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.285559 10538 raft_consensus.cc:385] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.285871 10538 raft_consensus.cc:740] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.293746 10538 consensus_queue.cc:260] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.295902 10538 ts_tablet_manager.cc:1434] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:22:15.296362 10496 raft_consensus.cc:2468] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 1.
I20260709 12:22:15.296923 10538 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:15.298702 10539 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.299229 10539 ts_tablet_manager.cc:1403] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.020s
I20260709 12:22:15.300690 10537 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.301379 10537 ts_tablet_manager.cc:1403] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.045s
I20260709 12:22:15.301923 10580 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.301923 10539 raft_consensus.cc:359] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.302910 10539 raft_consensus.cc:385] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.302668 10580 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.303203 10538 tablet_bootstrap.cc:654] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.304723 10580 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.304519 10537 raft_consensus.cc:359] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.301291 10314 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:15.305231 10537 raft_consensus.cc:385] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.305471 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.305531 10537 raft_consensus.cc:740] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.303205 10539 raft_consensus.cc:740] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.306228 10422 raft_consensus.cc:2468] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 0.
I20260709 12:22:15.306835 10580 raft_consensus.cc:2804] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.306550 10539 consensus_queue.cc:260] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.307387 10580 raft_consensus.cc:697] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 LEADER]: Becoming Leader. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Running, Role: LEADER
I20260709 12:22:15.306994 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.308243 10580 consensus_queue.cc:237] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
W20260709 12:22:15.308671 10314 leader_election.cc:343] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.306550 10537 consensus_queue.cc:260] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.313757 10537 ts_tablet_manager.cc:1434] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:22:15.314846 10537 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:15.314885 10311 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:15.316195 10543 raft_consensus.cc:2804] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.316567 10543 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.316895 10543 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.317169 10553 raft_consensus.cc:493] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.317914 10553 raft_consensus.cc:515] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.321326 10537 tablet_bootstrap.cc:654] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.308694 10539 ts_tablet_manager.cc:1434] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:22:15.321895 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.322925 10348 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.323240 10539 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.324767 10461 leader_election.cc:304] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.325971 10554 raft_consensus.cc:2804] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.326470 10554 raft_consensus.cc:493] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.326910 10554 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.329645 10539 tablet_bootstrap.cc:654] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.332301 10553 leader_election.cc:290] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.332540 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.333335 10422 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.336210 10543 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.340112 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.340862 10422 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.342592 10554 raft_consensus.cc:515] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.346602 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:15.347380 10348 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.353754 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:15.355953 10546 raft_consensus.cc:493] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.356619 10546 raft_consensus.cc:515] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.361366 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.362200 10496 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 0.
W20260709 12:22:15.363991 10314 leader_election.cc:343] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.364897 10347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.366650 10389 leader_election.cc:304] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 
I20260709 12:22:15.367853 10583 raft_consensus.cc:2804] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.368414 10583 raft_consensus.cc:493] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.368808 10583 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.370291 10580 raft_consensus.cc:493] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.371026 10580 raft_consensus.cc:515] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
W20260709 12:22:15.372105 10387 leader_election.cc:343] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.374727 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.375545 10546 leader_election.cc:290] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
W20260709 12:22:15.376823 10311 leader_election.cc:343] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.379561 10580 leader_election.cc:290] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.381109 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "8dec85d80d334e7d9f6c904a12fe3913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.381054 10348 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.387899 10539 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.385357 10461 leader_election.cc:304] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.388428 10539 ts_tablet_manager.cc:1403] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.022s
I20260709 12:22:15.388645 10553 raft_consensus.cc:2804] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.388953 10538 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: No bootstrap required, opened a new log
I20260709 12:22:15.389199 10553 raft_consensus.cc:697] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:15.389472 10538 ts_tablet_manager.cc:1403] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.093s	user 0.027s	sys 0.056s
I20260709 12:22:15.390116 10553 consensus_queue.cc:237] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.391557 10539 raft_consensus.cc:359] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.392228 10539 raft_consensus.cc:385] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.392486 10539 raft_consensus.cc:740] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.393090 10538 raft_consensus.cc:359] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.392699 10245 catalog_manager.cc:5697] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "8dec85d80d334e7d9f6c904a12fe3913" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.393227 10539 consensus_queue.cc:260] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.395555 10539 ts_tablet_manager.cc:1434] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:15.395998 10537 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.396457 10539 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca: Bootstrap starting.
I20260709 12:22:15.396502 10537 ts_tablet_manager.cc:1403] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.019s
I20260709 12:22:15.393801 10538 raft_consensus.cc:385] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.398448 10538 raft_consensus.cc:740] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.399366 10538 consensus_queue.cc:260] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.401604 10538 ts_tablet_manager.cc:1434] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
W20260709 12:22:15.402269 10314 leader_election.cc:343] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:15.402735 10539 tablet_bootstrap.cc:654] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.402868 10314 leader_election.cc:304] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8dec85d80d334e7d9f6c904a12fe3913; no voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:15.403282 10537 raft_consensus.cc:359] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.403889 10580 raft_consensus.cc:2749] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.404052 10537 raft_consensus.cc:385] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.409379 10537 raft_consensus.cc:740] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.408903 10543 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Requested vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.411382 10542 raft_consensus.cc:493] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.411518 10554 leader_election.cc:290] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.410140 10537 consensus_queue.cc:260] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.412194 10542 raft_consensus.cc:515] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.413939 10537 ts_tablet_manager.cc:1434] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:22:15.414706 10539 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca: No bootstrap required, opened a new log
I20260709 12:22:15.414959 10537 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40: Bootstrap starting.
I20260709 12:22:15.415231 10539 ts_tablet_manager.cc:1403] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.005s
I20260709 12:22:15.415099 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643e68f4ae67439581a6a5a3c4df5ed6" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.415506 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.415804 10348 raft_consensus.cc:2468] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.416143 10423 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.417171 10461 leader_election.cc:304] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.418135 10554 raft_consensus.cc:2804] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.418592 10554 raft_consensus.cc:493] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.418243 10539 raft_consensus.cc:359] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.418824 10422 raft_consensus.cc:2468] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dec85d80d334e7d9f6c904a12fe3913 in term 1.
I20260709 12:22:15.419009 10554 raft_consensus.cc:3060] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.419130 10539 raft_consensus.cc:385] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.419543 10539 raft_consensus.cc:740] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.420372 10539 consensus_queue.cc:260] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.422487 10539 ts_tablet_manager.cc:1434] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:15.422911 10542 leader_election.cc:290] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.424567 10421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643e68f4ae67439581a6a5a3c4df5ed6" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.425187 10421 raft_consensus.cc:2468] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.429004 10423 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.431480 10311 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:15.430538 10554 raft_consensus.cc:515] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.432758 10543 raft_consensus.cc:2804] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.433274 10543 raft_consensus.cc:697] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 LEADER]: Becoming Leader. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Running, Role: LEADER
I20260709 12:22:15.433797 10554 leader_election.cc:290] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.434130 10543 consensus_queue.cc:237] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.434700 10245 catalog_manager.cc:5697] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.438678 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643e68f4ae67439581a6a5a3c4df5ed6" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:15.431754 10537 tablet_bootstrap.cc:654] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.439273 10348 raft_consensus.cc:3060] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.441423 10583 raft_consensus.cc:515] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.443161 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643e68f4ae67439581a6a5a3c4df5ed6" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.443818 10423 raft_consensus.cc:3060] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.447451 10348 raft_consensus.cc:2468] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.448685 10347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:15.449375 10347 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.449738 10461 leader_election.cc:304] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.450670 10554 raft_consensus.cc:2804] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.451167 10554 raft_consensus.cc:697] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:15.452106 10554 consensus_queue.cc:237] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.461524 10347 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 1.
I20260709 12:22:15.463327 10387 leader_election.cc:304] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.464584 10546 raft_consensus.cc:2804] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.465297 10546 raft_consensus.cc:697] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:15.464738 10244 catalog_manager.cc:5697] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "8dec85d80d334e7d9f6c904a12fe3913" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.466549 10546 consensus_queue.cc:237] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.470079 10537 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40: No bootstrap required, opened a new log
I20260709 12:22:15.470647 10537 ts_tablet_manager.cc:1403] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent bootstrapping tablet: real 0.056s	user 0.021s	sys 0.015s
I20260709 12:22:15.473297 10537 raft_consensus.cc:359] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.474155 10537 raft_consensus.cc:385] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.474522 10537 raft_consensus.cc:740] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.475586 10553 raft_consensus.cc:493] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.476317 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:15.476948 10496 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.475360 10537 consensus_queue.cc:260] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.476145 10553 raft_consensus.cc:515] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.479084 10537 ts_tablet_manager.cc:1434] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:22:15.480551 10553 leader_election.cc:290] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.481598 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.482340 10348 raft_consensus.cc:2468] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.482450 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.483363 10422 raft_consensus.cc:2468] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 0.
I20260709 12:22:15.483745 10461 leader_election.cc:304] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.485360 10542 raft_consensus.cc:2804] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:15.485720 10542 raft_consensus.cc:493] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:15.486044 10542 raft_consensus.cc:3060] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.493069 10348 raft_consensus.cc:3060] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.497305 10553 consensus_queue.cc:1048] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.499390 10583 leader_election.cc:290] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.504081 10599 consensus_queue.cc:1048] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.500084 10496 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 1.
I20260709 12:22:15.499212 10542 raft_consensus.cc:515] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.505822 10553 consensus_queue.cc:1048] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.509523 10542 leader_election.cc:290] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.509339 10423 raft_consensus.cc:2468] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.511374 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:15.512147 10423 raft_consensus.cc:3060] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.508510 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:15.515091 10348 raft_consensus.cc:3060] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.522902 10244 catalog_manager.cc:5697] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 0 to 1, leader changed from <none> to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 1 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.552193 10348 raft_consensus.cc:2468] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.553755 10461 leader_election.cc:304] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:15.556149 10423 raft_consensus.cc:2468] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:15.558871 10553 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.562629 10553 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.565572 10553 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:15.567512 10245 catalog_manager.cc:5697] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.574156 10546 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.574728 10546 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.576843 10546 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:15.578716 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:15.579717 10496 raft_consensus.cc:2393] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e1844b0ccb48adb22cc528da104bca in current term 1: Already voted for candidate 8dec85d80d334e7d9f6c904a12fe3913 in this term.
I20260709 12:22:15.583840 10347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.558442 10599 raft_consensus.cc:2804] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.586221 10599 raft_consensus.cc:697] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:15.587205 10599 consensus_queue.cc:237] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:15.591840 10348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913" is_pre_election: true
I20260709 12:22:15.593883 10387 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca; no voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:15.596021 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:15.604614 10546 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.619459 10423 raft_consensus.cc:2393] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ae1ebdf2c004893b66a1097f2b0af40 in current term 1: Already voted for candidate 8dec85d80d334e7d9f6c904a12fe3913 in this term.
I20260709 12:22:15.623217 10546 raft_consensus.cc:2749] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.623822 10460 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:15.628162 10606 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.633198 10607 consensus_queue.cc:1048] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.646770 10606 raft_consensus.cc:2749] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:15.647006 10543 consensus_queue.cc:1048] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.659435 10245 catalog_manager.cc:5697] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.667872 10592 consensus_queue.cc:1048] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.678855 10546 consensus_queue.cc:1048] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.692466 10546 consensus_queue.cc:1048] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.844784 10607 consensus_queue.cc:1048] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.854835 10608 consensus_queue.cc:1048] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.881258 10592 consensus_queue.cc:1048] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.892352 10580 consensus_queue.cc:1048] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.904727 10543 consensus_queue.cc:1048] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.908409 10546 consensus_queue.cc:1048] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.928406 10583 consensus_queue.cc:1048] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:15.930276 10580 consensus_queue.cc:1048] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:15.934110 30820 ts_manager.cc:295] Set tserver state for 8dec85d80d334e7d9f6c904a12fe3913 to MAINTENANCE_MODE
I20260709 12:22:15.935055 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
I20260709 12:22:15.978940 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:15.979708 30820 tablet_replica.cc:333] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:15.980618 30820 raft_consensus.cc:2243] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:15.981186 30820 raft_consensus.cc:2272] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:15.983572 10461 consensus_peers.cc:597] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:15.984232 30820 tablet_replica.cc:333] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:15.984894 30820 raft_consensus.cc:2243] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:15.985311 30820 raft_consensus.cc:2272] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:15.987454 30820 tablet_replica.cc:333] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:15.988010 30820 raft_consensus.cc:2243] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:15.988389 30820 raft_consensus.cc:2272] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:15.990470 30820 tablet_replica.cc:333] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:15.991076 30820 raft_consensus.cc:2243] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:15.991494 30820 raft_consensus.cc:2272] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:15.993607 30820 tablet_replica.cc:333] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:15.994169 30820 raft_consensus.cc:2243] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:15.994673 30820 raft_consensus.cc:2272] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:15.996240 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:15.996966 30820 tablet_replica.cc:333] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:15.997604 30820 raft_consensus.cc:2243] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:15.998515 30820 raft_consensus.cc:2272] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.000537 30820 tablet_replica.cc:333] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:16.001082 30820 raft_consensus.cc:2243] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:16.001477 30820 raft_consensus.cc:2272] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.003593 30820 tablet_replica.cc:333] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:16.004128 30820 raft_consensus.cc:2243] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:16.004498 30820 raft_consensus.cc:2272] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.006150 30820 tablet_replica.cc:333] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:16.006682 30820 raft_consensus.cc:2243] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:16.007467 30820 raft_consensus.cc:2272] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.009857 30820 tablet_replica.cc:333] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: stopping tablet replica
I20260709 12:22:16.010474 30820 raft_consensus.cc:2243] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:22:16.010674 10461 consensus_peers.cc:597] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:16.011420 30820 raft_consensus.cc:2272] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:16.018374 10608 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:16.047498 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
W20260709 12:22:16.147408 10461 consensus_peers.cc:597] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:16.153460 10387 consensus_peers.cc:597] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:16.159598 10461 consensus_peers.cc:597] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:16.234817 10608 consensus_queue.cc:1048] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:22:16.426738 10461 consensus_peers.cc:597] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:16.427964 10387 consensus_peers.cc:597] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:16.997366 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:17.138784 10461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111) [suppressed 127 similar messages]
I20260709 12:22:17.171550 10631 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.171605 10632 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.172410 10631 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:17.172555 10632 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:17.174396 10631 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:17.175572 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:17.176083 10632 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:17.176232 10423 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:17.177438 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:17.177599 10460 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 
I20260709 12:22:17.178084 10496 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 1.
W20260709 12:22:17.178634 10461 leader_election.cc:336] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.178701 10631 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.179237 10631 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.179662 10631 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.179965 10389 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 
W20260709 12:22:17.183624 10387 leader_election.cc:336] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.188612 10631 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:17.189287 10632 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.189769 10632 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.190151 10632 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.191816 10631 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: Requested vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:17.191947 10423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
W20260709 12:22:17.195988 10461 leader_election.cc:336] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.197896 10632 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:17.198940 10423 raft_consensus.cc:2393] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ae1ebdf2c004893b66a1097f2b0af40 in current term 2: Already voted for candidate 25e1844b0ccb48adb22cc528da104bca in this term.
I20260709 12:22:17.200693 10632 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Requested vote from peers 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:17.200608 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:17.201599 10496 raft_consensus.cc:2393] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 25e1844b0ccb48adb22cc528da104bca in current term 2: Already voted for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in this term.
I20260709 12:22:17.203217 10460 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.203976 10631 raft_consensus.cc:2749] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:22:17.204231 10387 leader_election.cc:336] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.204748 10387 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca; no voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.205472 10632 raft_consensus.cc:2749] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:17.392072 10606 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.392711 10606 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.394721 10606 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:17.395780 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:17.396454 10422 raft_consensus.cc:2468] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 1.
I20260709 12:22:17.397296 10632 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.397819 10632 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.397941 10460 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 
I20260709 12:22:17.399026 10606 raft_consensus.cc:2804] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.399358 10606 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
W20260709 12:22:17.399580 10461 leader_election.cc:336] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.399736 10606 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.400876 10632 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:17.402040 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
W20260709 12:22:17.403625 10387 leader_election.cc:336] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.405793 10606 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.406811 10496 raft_consensus.cc:2393] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25e1844b0ccb48adb22cc528da104bca in current term 2: Already voted for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in this term.
I20260709 12:22:17.407770 10606 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:17.408953 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:17.408972 10389 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca; no voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.409657 10422 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:17.411563 10461 leader_election.cc:336] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.415936 10422 raft_consensus.cc:2468] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 2.
I20260709 12:22:17.416438 10632 raft_consensus.cc:2749] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:17.417074 10460 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.417877 10606 raft_consensus.cc:2804] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.418457 10606 raft_consensus.cc:697] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:17.419281 10606 consensus_queue.cc:237] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.426553 10244 catalog_manager.cc:5697] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 1 to 2, leader changed from 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1) to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.430433 10631 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.430923 10631 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.432767 10631 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:17.434271 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25e1844b0ccb48adb22cc528da104bca" is_pre_election: true
I20260709 12:22:17.437331 10632 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
W20260709 12:22:17.437548 10461 leader_election.cc:336] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.437856 10632 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.438144 10461 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.439054 10606 raft_consensus.cc:2749] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:17.441637 10632 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:17.442901 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:17.443626 10496 raft_consensus.cc:2468] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 1.
W20260709 12:22:17.444047 10387 leader_election.cc:336] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.444661 10389 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.445276 10632 raft_consensus.cc:2804] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.445616 10632 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8dec85d80d334e7d9f6c904a12fe3913)
I20260709 12:22:17.445859 10632 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.450668 10632 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.452215 10632 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:17.453298 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:17.453867 10496 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:17.455689 10387 leader_election.cc:336] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:17.459033 10496 raft_consensus.cc:2468] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:17.460148 10389 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:17.460834 10632 raft_consensus.cc:2804] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.461229 10632 raft_consensus.cc:697] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:17.462018 10632 consensus_queue.cc:237] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:17.469065 10244 catalog_manager.cc:5697] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 1 to 2, leader changed from 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 2 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.807040 10422 raft_consensus.cc:1275] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.808759 10606 consensus_queue.cc:1048] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:17.824101 10461 consensus_peers.cc:597] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:17.911548 10387 consensus_peers.cc:597] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:17.998409 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:18.073151 10496 raft_consensus.cc:1275] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.074561 10632 consensus_queue.cc:1048] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:18.313237 10461 consensus_peers.cc:597] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:18.444907 10461 consensus_peers.cc:597] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:18.654364 10461 consensus_peers.cc:597] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:18.703270 10387 consensus_peers.cc:597] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:18.735996 10461 consensus_peers.cc:597] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:18.922272 10646 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.922745 10646 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:18.924505 10646 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:18.925215 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" is_pre_election: true
I20260709 12:22:18.925861 10496 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:18.927037 10389 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 
I20260709 12:22:18.927814 10646 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.928186 10646 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:18.928566 10646 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:18.928655 10387 leader_election.cc:336] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:18.933410 10646 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:18.934989 10646 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: Requested vote from peers 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:18.935784 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:18.936327 10496 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:18.938638 10387 leader_election.cc:336] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111)
I20260709 12:22:18.941454 10496 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 3.
I20260709 12:22:18.942534 10389 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 5ae1ebdf2c004893b66a1097f2b0af40; no voters: 8dec85d80d334e7d9f6c904a12fe3913
I20260709 12:22:18.943126 10646 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.944025 10646 raft_consensus.cc:697] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:18.944689 10646 consensus_queue.cc:237] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:18.950940 10245 catalog_manager.cc:5697] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 1 to 3, leader changed from 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 3 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:18.991254 10461 consensus_peers.cc:597] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:18.999405 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:19.062974 10387 consensus_peers.cc:597] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:19.410770 10496 raft_consensus.cc:1275] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:19.412036 10646 consensus_queue.cc:1048] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:19.427380 10387 consensus_peers.cc:597] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.000304 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:20.216806 10461 consensus_peers.cc:597] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:20.498517 10387 consensus_peers.cc:597] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:20.666654 10461 consensus_peers.cc:597] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:20.964164 10461 consensus_peers.cc:597] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:21.001221 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:21.024587 10461 consensus_peers.cc:597] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:21.067204 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:21.074112 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:21.074381 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:21.075781 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:21.075974 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:21.076992 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:21.077253 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:21.077454 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599741077430 us; error 0 us; skew 500 ppm
I20260709 12:22:21.078049 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:21.081058 30820 webserver.cc:533] Webserver started at http://127.30.25.1:39889/ using document root <none> and password file <none>
I20260709 12:22:21.081691 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:21.081923 30820 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:22:21.086048 30820 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:22:21.090235 10667 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:21.091250 30820 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:21.091668 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "8dec85d80d334e7d9f6c904a12fe3913"
format_stamp: "Formatted at 2026-07-09 12:22:14 on dist-test-slave-rldg"
I20260709 12:22:21.092058 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:21.113368 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:21.114595 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:21.115959 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:21.119451 10674 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:22:21.167968 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:22:21.168301 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.050s	user 0.002s	sys 0.000s
I20260709 12:22:21.168625 30820 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:22:21.173106 10674 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.186023 10674 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.186862 10674 tablet_bootstrap.cc:492] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.187443 10674 ts_tablet_manager.cc:1403] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:22:21.189473 10674 raft_consensus.cc:359] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.190074 10674 raft_consensus.cc:740] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.190824 10674 consensus_queue.cc:260] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.192981 10674 ts_tablet_manager.cc:1434] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:22:21.193660 10674 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.205878 10674 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.206732 10674 tablet_bootstrap.cc:492] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.207331 10674 ts_tablet_manager.cc:1403] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.014s	user 0.003s	sys 0.009s
I20260709 12:22:21.207804 30820 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:22:21.208112 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.040s	user 0.038s	sys 0.000s
I20260709 12:22:21.209481 10674 raft_consensus.cc:359] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:21.210263 10674 raft_consensus.cc:740] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.211072 10674 consensus_queue.cc:260] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:21.213052 10674 ts_tablet_manager.cc:1434] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.002s
I20260709 12:22:21.213856 10674 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.229470 10674 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.230597 10674 tablet_bootstrap.cc:492] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.231348 10674 ts_tablet_manager.cc:1403] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 12:22:21.234230 10674 raft_consensus.cc:359] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.235105 10674 raft_consensus.cc:740] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.236656 10674 consensus_queue.cc:260] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
W20260709 12:22:21.237797 10461 consensus_peers.cc:597] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:21.238763 10674 ts_tablet_manager.cc:1434] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:22:21.239537 10674 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.249099 10674 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.249960 10674 tablet_bootstrap.cc:492] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.250730 10674 ts_tablet_manager.cc:1403] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:22:21.253206 10674 raft_consensus.cc:359] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.253973 10674 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.254738 10674 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
W20260709 12:22:21.255376 10387 consensus_peers.cc:597] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:21.256439 10674 ts_tablet_manager.cc:1434] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:22:21.257176 10674 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.269101 10674 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.269853 10674 tablet_bootstrap.cc:492] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.270573 10674 ts_tablet_manager.cc:1403] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:22:21.271350 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:41877
I20260709 12:22:21.271467 10736 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:41877 every 8 connection(s)
I20260709 12:22:21.272717 10674 raft_consensus.cc:359] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.273247 10674 raft_consensus.cc:740] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.273871 10674 consensus_queue.cc:260] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.275523 10674 ts_tablet_manager.cc:1434] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:21.275955 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.276372 10674 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.307063 10674 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.308076 10674 tablet_bootstrap.cc:492] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.308768 10674 ts_tablet_manager.cc:1403] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.004s
I20260709 12:22:21.311761 10737 heartbeater.cc:344] Connected to a master server at 127.30.25.62:44227
I20260709 12:22:21.312251 10737 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:21.313172 10737 heartbeater.cc:507] Master 127.30.25.62:44227 requested a full tablet report, sending...
I20260709 12:22:21.314725 10674 raft_consensus.cc:359] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.315376 10674 raft_consensus.cc:740] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.315436 10496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
mode: GRACEFUL
new_leader_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:39390
I20260709 12:22:21.316262 10496 raft_consensus.cc:606] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Received request to transfer leadership to TS 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:21.316963 10674 consensus_queue.cc:260] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.319365 10496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
mode: GRACEFUL
new_leader_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:39390
I20260709 12:22:21.319947 10244 ts_manager.cc:194] Re-registered known tserver with Master: 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:21.320022 10496 raft_consensus.cc:606] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Received request to transfer leadership to TS 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:21.320019 10674 ts_tablet_manager.cc:1434] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:22:21.322618 10674 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
W20260709 12:22:21.324024 10461 consensus_peers.cc:597] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:21.326707 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:21.327899 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:21.328755 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.330502 10244 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39186
I20260709 12:22:21.333700 10737 heartbeater.cc:499] Master 127.30.25.62:44227 was elected leader, sending a full tablet report...
I20260709 12:22:21.337947 10674 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.339113 10674 tablet_bootstrap.cc:492] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.339723 10674 ts_tablet_manager.cc:1403] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:22:21.341650 10674 raft_consensus.cc:359] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.342234 10674 raft_consensus.cc:740] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.342905 10674 consensus_queue.cc:260] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.344726 10674 ts_tablet_manager.cc:1434] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:21.345407 10674 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.358217 10674 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.359210 10674 tablet_bootstrap.cc:492] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.359907 10674 ts_tablet_manager.cc:1403] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 12:22:21.361804 10674 raft_consensus.cc:359] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.362437 10674 raft_consensus.cc:740] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.363176 10674 consensus_queue.cc:260] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.365044 10674 ts_tablet_manager.cc:1434] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:22:21.366125 10674 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.377698 10674 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.378624 10674 tablet_bootstrap.cc:492] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.379208 10674 ts_tablet_manager.cc:1403] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:22:21.381130 10674 raft_consensus.cc:359] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.381654 10674 raft_consensus.cc:740] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.382228 10674 consensus_queue.cc:260] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.383702 10674 ts_tablet_manager.cc:1434] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:22:21.384470 10674 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap starting.
I20260709 12:22:21.396478 10674 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:21.397276 10674 tablet_bootstrap.cc:492] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Bootstrap complete.
I20260709 12:22:21.397822 10674 ts_tablet_manager.cc:1403] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:22:21.399756 10674 raft_consensus.cc:359] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.400450 10674 raft_consensus.cc:740] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dec85d80d334e7d9f6c904a12fe3913, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.401089 10674 consensus_queue.cc:260] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.402899 10674 ts_tablet_manager.cc:1434] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:22:21.455226 10658 raft_consensus.cc:993] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: : Instructing follower 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:21.455721 10749 raft_consensus.cc:1081] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Signalling peer 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:21.457171 10423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:33602
I20260709 12:22:21.457629 10423 raft_consensus.cc:493] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:21.457896 10423 raft_consensus.cc:3060] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.462803 10423 raft_consensus.cc:515] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.464349 10423 leader_election.cc:290] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:21.465252 10710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:21.465463 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc6723eb06e4d09b65f8d0d739d35dd" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:21.466012 10710 raft_consensus.cc:3060] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.466008 10496 raft_consensus.cc:3055] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:21.466614 10496 raft_consensus.cc:740] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:21.467492 10496 consensus_queue.cc:260] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.468866 10496 raft_consensus.cc:3060] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.474042 10710 raft_consensus.cc:2468] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:21.475615 10387 leader_election.cc:304] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:21.476567 10657 raft_consensus.cc:2804] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:21.477993 10496 raft_consensus.cc:2468] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:21.479449 10657 raft_consensus.cc:697] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:21.481065 10657 consensus_queue.cc:237] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.489711 10245 catalog_manager.cc:5697] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 1 to 2, leader changed from 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 2 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:21.579419 10709 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.583124 10658 raft_consensus.cc:993] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: : Instructing follower 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:21.583706 10749 raft_consensus.cc:1081] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Signalling peer 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:21.585220 10423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:33602
I20260709 12:22:21.585778 10423 raft_consensus.cc:493] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:21.586159 10423 raft_consensus.cc:3060] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.592955 10423 raft_consensus.cc:515] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.594731 10423 leader_election.cc:290] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:21.595449 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:21.596105 10709 raft_consensus.cc:3060] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.596434 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1220a04033fc478a909de6d8f46c51bf" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:21.597036 10496 raft_consensus.cc:3055] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:21.597329 10496 raft_consensus.cc:740] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:21.598052 10496 consensus_queue.cc:260] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.599184 10496 raft_consensus.cc:3060] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.602782 10709 raft_consensus.cc:2468] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:21.604068 10387 leader_election.cc:304] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:21.604908 10657 raft_consensus.cc:2804] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:21.605407 10657 raft_consensus.cc:697] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:21.605486 10496 raft_consensus.cc:2468] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:21.606215 10657 consensus_queue.cc:237] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:21.612848 10245 catalog_manager.cc:5697] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 1 to 2, leader changed from 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 2 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:21.618366 10709 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.737931 10709 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.881095 10709 raft_consensus.cc:1275] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:21.882247 10764 consensus_queue.cc:1048] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:21.893417 10496 raft_consensus.cc:1275] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:21.894917 10657 consensus_queue.cc:1048] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:22.002653 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:22.203339 10709 raft_consensus.cc:1275] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:22.205019 10657 consensus_queue.cc:1048] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:22.216773 10496 raft_consensus.cc:1275] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:22.218554 10763 consensus_queue.cc:1048] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:22.342314 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:22.361423 10422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
mode: GRACEFUL
new_leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:35108
I20260709 12:22:22.362028 10422 raft_consensus.cc:606] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Received request to transfer leadership to TS 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:22.365149 10422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
mode: GRACEFUL
new_leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:35108
I20260709 12:22:22.365670 10422 raft_consensus.cc:606] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Received request to transfer leadership to TS 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:22.366839 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:22.367364 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:22.368119 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:22.392640 10752 raft_consensus.cc:993] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: : Instructing follower 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:22.393311 10657 raft_consensus.cc:1081] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Signalling peer 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:22.394816 10496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:58854
I20260709 12:22:22.395289 10496 raft_consensus.cc:493] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.395614 10496 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.403805 10496 raft_consensus.cc:515] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:22.405395 10496 leader_election.cc:290] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:22.406570 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:22.406703 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:22.407444 10709 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.407524 10422 raft_consensus.cc:3055] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:22.407997 10422 raft_consensus.cc:740] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:22.408968 10422 consensus_queue.cc:260] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:22.410166 10422 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.415899 10709 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 2.
I20260709 12:22:22.417132 10461 leader_election.cc:304] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:22.418048 10749 raft_consensus.cc:2804] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:22.418510 10749 raft_consensus.cc:697] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:22.418937 10422 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 2.
I20260709 12:22:22.419533 10749 consensus_queue.cc:237] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:22.426716 10245 catalog_manager.cc:5697] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 1 to 2, leader changed from 25e1844b0ccb48adb22cc528da104bca (127.30.25.2) to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:22.804350 10422 raft_consensus.cc:1275] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:22.805636 10749 consensus_queue.cc:1048] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:22.817646 10709 raft_consensus.cc:1275] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:22.819454 10780 consensus_queue.cc:1048] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:22.824743 10752 raft_consensus.cc:993] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: : Instructing follower 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:22.825290 10657 raft_consensus.cc:1081] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Signalling peer 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:22.827454 10496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:58854
I20260709 12:22:22.828060 10496 raft_consensus.cc:493] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.828315 10496 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.834452 10496 raft_consensus.cc:515] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:22.836643 10496 leader_election.cc:290] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 3 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:22.837349 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:22.836798 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ec4c2a8f774679a048658e30d5708b" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:22.837937 10422 raft_consensus.cc:3055] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:22.838194 10709 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.838369 10422 raft_consensus.cc:740] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:22.839365 10422 consensus_queue.cc:260] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:22.840677 10422 raft_consensus.cc:3060] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.845059 10709 raft_consensus.cc:2468] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 3.
I20260709 12:22:22.845553 10422 raft_consensus.cc:2468] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 3.
I20260709 12:22:22.846200 10461 leader_election.cc:304] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:22.846987 10780 raft_consensus.cc:2804] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.847450 10780 raft_consensus.cc:697] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:22.848349 10780 consensus_queue.cc:237] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:22.855553 10245 catalog_manager.cc:5697] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 2 to 3, leader changed from 25e1844b0ccb48adb22cc528da104bca (127.30.25.2) to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 3 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:23.003816 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:23.266199 10709 raft_consensus.cc:1275] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:23.267678 10780 consensus_queue.cc:1048] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:23.277905 10422 raft_consensus.cc:1275] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:23.280457 10787 consensus_queue.cc:1048] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:23.374228 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.382922 10496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
mode: GRACEFUL
new_leader_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:39390
I20260709 12:22:23.383558 10496 raft_consensus.cc:606] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Received request to transfer leadership to TS 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:23.386276 10496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
mode: GRACEFUL
new_leader_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:39390
I20260709 12:22:23.386860 10496 raft_consensus.cc:606] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Received request to transfer leadership to TS 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:23.388091 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:23.388610 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:23.389233 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.783231 10787 raft_consensus.cc:993] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: : Instructing follower 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:23.783746 10749 raft_consensus.cc:1081] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Signalling peer 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:23.785171 10422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:33602
I20260709 12:22:23.785604 10422 raft_consensus.cc:493] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:23.785868 10422 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.791518 10422 raft_consensus.cc:515] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:23.792968 10422 leader_election.cc:290] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:23.794072 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:23.794296 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:23.794868 10709 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.794906 10496 raft_consensus.cc:3055] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:23.795378 10496 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:23.796169 10496 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:23.797137 10496 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.800768 10709 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:23.801925 10387 leader_election.cc:304] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:23.802668 10752 raft_consensus.cc:2804] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:23.802881 10496 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 2.
I20260709 12:22:23.803133 10752 raft_consensus.cc:697] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:23.804327 10752 consensus_queue.cc:237] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:23.808940 10780 raft_consensus.cc:993] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: : Instructing follower 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:23.809446 10780 raft_consensus.cc:1081] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Signalling peer 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:23.812207 10422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:33602
I20260709 12:22:23.812907 10422 raft_consensus.cc:493] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:23.813426 10422 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.821616 10422 raft_consensus.cc:515] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:23.823791 10422 leader_election.cc:290] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:23.823052 10244 catalog_manager.cc:5697] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 1 to 2, leader changed from 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 2 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.824162 10710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:23.825080 10710 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.825230 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29deab0b8b384dccbed4b383d96eb326" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:23.826609 10496 raft_consensus.cc:3055] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:23.826891 10496 raft_consensus.cc:740] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:23.827642 10496 consensus_queue.cc:260] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:23.828794 10496 raft_consensus.cc:3060] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.833431 10710 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 3.
I20260709 12:22:23.834842 10387 leader_election.cc:304] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:23.835603 10496 raft_consensus.cc:2468] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 3.
I20260709 12:22:23.835642 10752 raft_consensus.cc:2804] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:23.836499 10752 raft_consensus.cc:697] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:23.837375 10752 consensus_queue.cc:237] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:23.844008 10244 catalog_manager.cc:5697] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 2 to 3, leader changed from 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 3 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:24.004824 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:24.201014 10710 raft_consensus.cc:1275] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:24.202410 10752 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:24.213986 10496 raft_consensus.cc:1275] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:24.218192 10797 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:24.326869 10496 raft_consensus.cc:1275] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:24.328190 10797 consensus_queue.cc:1048] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:24.337366 10710 raft_consensus.cc:1275] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:24.339059 10657 consensus_queue.cc:1048] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:24.396643 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.403803 10422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81cf0923653b483f9ba448b97810d958"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
mode: GRACEFUL
new_leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:35108
I20260709 12:22:24.404330 10422 raft_consensus.cc:606] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Received request to transfer leadership to TS 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:24.406975 10422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
mode: GRACEFUL
new_leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:35108
I20260709 12:22:24.407454 10422 raft_consensus.cc:606] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Received request to transfer leadership to TS 5ae1ebdf2c004893b66a1097f2b0af40
I20260709 12:22:24.408562 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:24.408998 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:24.409495 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.573730 10752 raft_consensus.cc:993] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: : Instructing follower 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:24.574333 10657 raft_consensus.cc:1081] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Signalling peer 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:24.576071 10496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81cf0923653b483f9ba448b97810d958"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:58854
I20260709 12:22:24.576571 10496 raft_consensus.cc:493] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:24.576820 10496 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.581609 10496 raft_consensus.cc:515] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:24.583375 10496 leader_election.cc:290] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 4 election: Requested vote from peers 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477), 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877)
I20260709 12:22:24.584180 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:24.584443 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cf0923653b483f9ba448b97810d958" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:24.584928 10422 raft_consensus.cc:3055] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:24.585111 10709 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.585282 10422 raft_consensus.cc:740] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:24.586256 10422 consensus_queue.cc:260] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:24.587281 10422 raft_consensus.cc:3060] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.591506 10709 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 4.
I20260709 12:22:24.592803 10461 leader_election.cc:304] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:24.593014 10422 raft_consensus.cc:2468] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 4.
I20260709 12:22:24.593647 10749 raft_consensus.cc:2804] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:24.594229 10749 raft_consensus.cc:697] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:24.595419 10749 consensus_queue.cc:237] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } }
I20260709 12:22:24.606156 10245 catalog_manager.cc:5697] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 3 to 4, leader changed from 25e1844b0ccb48adb22cc528da104bca (127.30.25.2) to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 4 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:24.704493 10752 raft_consensus.cc:993] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: : Instructing follower 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:24.704998 10657 raft_consensus.cc:1081] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Signalling peer 5ae1ebdf2c004893b66a1097f2b0af40 to start an election
I20260709 12:22:24.706564 10496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
 from {username='slave'} at 127.0.0.1:58854
I20260709 12:22:24.707168 10496 raft_consensus.cc:493] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:24.707506 10496 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.713567 10496 raft_consensus.cc:515] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:24.715049 10496 leader_election.cc:290] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 3 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 25e1844b0ccb48adb22cc528da104bca (127.30.25.2:39477)
I20260709 12:22:24.715876 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:24.716066 10422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
I20260709 12:22:24.716629 10709 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.716718 10422 raft_consensus.cc:3055] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:24.717130 10422 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:24.717811 10422 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:24.718703 10422 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.722540 10709 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 3.
I20260709 12:22:24.723346 10422 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1ebdf2c004893b66a1097f2b0af40 in term 3.
I20260709 12:22:24.723731 10461 leader_election.cc:304] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1ebdf2c004893b66a1097f2b0af40, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:24.724689 10749 raft_consensus.cc:2804] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:24.725122 10749 raft_consensus.cc:697] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Becoming Leader. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:24.725848 10749 consensus_queue.cc:237] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:24.731961 10245 catalog_manager.cc:5697] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 reported cstate change: term changed from 2 to 3, leader changed from 25e1844b0ccb48adb22cc528da104bca (127.30.25.2) to 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3). New cstate: current_term: 3 leader_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:25.005743 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:25.127266 10422 raft_consensus.cc:1275] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:25.128778 10803 consensus_queue.cc:1048] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.140059 10709 raft_consensus.cc:1275] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:25.141824 10749 consensus_queue.cc:1048] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.186563 10422 raft_consensus.cc:1275] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.187754 10749 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.195431 10709 raft_consensus.cc:1275] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Refusing update from remote peer 5ae1ebdf2c004893b66a1097f2b0af40: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.197139 10811 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.415751 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.423121 10496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
mode: GRACEFUL
new_leader_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:39390
I20260709 12:22:25.423666 10496 raft_consensus.cc:606] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Received request to transfer leadership to TS 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:25.426102 10496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff"
dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
mode: GRACEFUL
new_leader_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:39390
I20260709 12:22:25.426590 10496 raft_consensus.cc:606] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Received request to transfer leadership to TS 25e1844b0ccb48adb22cc528da104bca
I20260709 12:22:25.427512 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:25.428017 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.428597 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.493121 10811 raft_consensus.cc:993] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: : Instructing follower 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:25.493647 10803 raft_consensus.cc:1081] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Signalling peer 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:25.495155 10422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:33602
I20260709 12:22:25.495632 10422 raft_consensus.cc:493] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.495867 10422 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.501169 10422 raft_consensus.cc:515] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:25.502734 10422 leader_election.cc:290] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:25.503453 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:25.503784 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4556df24f388448b85b996e2f2ebe127" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:25.504097 10709 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.504374 10496 raft_consensus.cc:3055] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:25.504699 10496 raft_consensus.cc:740] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:25.505360 10496 consensus_queue.cc:260] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:25.506278 10496 raft_consensus.cc:3060] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.509250 10709 raft_consensus.cc:2468] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 3.
I20260709 12:22:25.510357 10387 leader_election.cc:304] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:25.510980 10496 raft_consensus.cc:2468] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 3.
I20260709 12:22:25.511137 10752 raft_consensus.cc:2804] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:25.512306 10752 raft_consensus.cc:697] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:25.513190 10752 consensus_queue.cc:237] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:25.520555 10244 catalog_manager.cc:5697] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 2 to 3, leader changed from 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 3 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.684494 10811 raft_consensus.cc:993] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: : Instructing follower 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:25.684894 10803 raft_consensus.cc:1081] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Signalling peer 25e1844b0ccb48adb22cc528da104bca to start an election
I20260709 12:22:25.686283 10422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff"
dest_uuid: "25e1844b0ccb48adb22cc528da104bca"
 from {username='slave'} at 127.0.0.1:33602
I20260709 12:22:25.686774 10422 raft_consensus.cc:493] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.687095 10422 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.692885 10422 raft_consensus.cc:515] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:25.694317 10422 leader_election.cc:290] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 4 election: Requested vote from peers 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877), 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3:43805)
I20260709 12:22:25.695521 10496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5ae1ebdf2c004893b66a1097f2b0af40"
I20260709 12:22:25.696151 10496 raft_consensus.cc:3055] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:25.695966 10709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c35fc1d28d25478388654b8b37d694ff" candidate_uuid: "25e1844b0ccb48adb22cc528da104bca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8dec85d80d334e7d9f6c904a12fe3913"
I20260709 12:22:25.697126 10709 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.696552 10496 raft_consensus.cc:740] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5ae1ebdf2c004893b66a1097f2b0af40, State: Running, Role: LEADER
I20260709 12:22:25.698589 10496 consensus_queue.cc:260] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:25.699896 10496 raft_consensus.cc:3060] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.703423 10709 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 4.
I20260709 12:22:25.704576 10387 leader_election.cc:304] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e1844b0ccb48adb22cc528da104bca, 8dec85d80d334e7d9f6c904a12fe3913; no voters: 
I20260709 12:22:25.705302 10752 raft_consensus.cc:2804] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:25.705698 10496 raft_consensus.cc:2468] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e1844b0ccb48adb22cc528da104bca in term 4.
I20260709 12:22:25.705854 10752 raft_consensus.cc:697] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 4 LEADER]: Becoming Leader. State: Replica: 25e1844b0ccb48adb22cc528da104bca, State: Running, Role: LEADER
I20260709 12:22:25.706898 10752 consensus_queue.cc:237] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } }
I20260709 12:22:25.713333 10244 catalog_manager.cc:5697] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca reported cstate change: term changed from 3 to 4, leader changed from 5ae1ebdf2c004893b66a1097f2b0af40 (127.30.25.3) to 25e1844b0ccb48adb22cc528da104bca (127.30.25.2). New cstate: current_term: 4 leader_uuid: "25e1844b0ccb48adb22cc528da104bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e1844b0ccb48adb22cc528da104bca" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.927798 10496 raft_consensus.cc:1275] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.929046 10752 consensus_queue.cc:1048] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.940339 10709 raft_consensus.cc:1275] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.941767 10824 consensus_queue.cc:1048] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:26.006858 10294 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:26.191725 10710 raft_consensus.cc:1275] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:26.193076 10816 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dec85d80d334e7d9f6c904a12fe3913" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:26.201134 10496 raft_consensus.cc:1275] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Refusing update from remote peer 25e1844b0ccb48adb22cc528da104bca: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:26.202558 10824 consensus_queue.cc:1048] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1ebdf2c004893b66a1097f2b0af40" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 43805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:26.434623 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:26.463042 30820 tablet_server.cc:179] TabletServer@127.30.25.1:41877 shutting down...
I20260709 12:22:26.492524 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:26.493400 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.494277 30820 raft_consensus.cc:2243] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.494858 30820 raft_consensus.cc:2272] T ebc6723eb06e4d09b65f8d0d739d35dd P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.497437 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.498150 30820 raft_consensus.cc:2243] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.498687 30820 raft_consensus.cc:2272] T 1220a04033fc478a909de6d8f46c51bf P 8dec85d80d334e7d9f6c904a12fe3913 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.500898 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.501510 30820 raft_consensus.cc:2243] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.501977 30820 raft_consensus.cc:2272] T 643e68f4ae67439581a6a5a3c4df5ed6 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.504377 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.504948 30820 raft_consensus.cc:2243] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.505393 30820 raft_consensus.cc:2272] T 4556df24f388448b85b996e2f2ebe127 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.508337 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.508924 30820 raft_consensus.cc:2243] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.509368 30820 raft_consensus.cc:2272] T 73ec4c2a8f774679a048658e30d5708b P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.511562 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.512123 30820 raft_consensus.cc:2243] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.512588 30820 raft_consensus.cc:2272] T 81cf0923653b483f9ba448b97810d958 P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.514793 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.515364 30820 raft_consensus.cc:2243] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.515815 30820 raft_consensus.cc:2272] T c35fc1d28d25478388654b8b37d694ff P 8dec85d80d334e7d9f6c904a12fe3913 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.518527 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.519073 30820 raft_consensus.cc:2243] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.519520 30820 raft_consensus.cc:2272] T 4faa9dae270549169ff4a40136ebd180 P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.522218 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.522936 30820 raft_consensus.cc:2243] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.523511 30820 raft_consensus.cc:2272] T 29deab0b8b384dccbed4b383d96eb326 P 8dec85d80d334e7d9f6c904a12fe3913 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:26.523600 10461 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 90 similar messages]
I20260709 12:22:26.525871 30820 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:26.526824 30820 raft_consensus.cc:2243] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:26.526881 10461 consensus_peers.cc:597] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:26.527374 30820 raft_consensus.cc:2272] T 901b273b616542b3a23c6aaf861d7b7a P 8dec85d80d334e7d9f6c904a12fe3913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.561956 30820 tablet_server.cc:196] TabletServer@127.30.25.1:41877 shutdown complete.
W20260709 12:22:26.583163 10387 consensus_peers.cc:597] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:26.594198 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
W20260709 12:22:26.599421 10387 consensus_peers.cc:597] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:26.618369 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:26.619262 30820 tablet_replica.cc:333] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.620179 30820 raft_consensus.cc:2243] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.620996 30820 raft_consensus.cc:2272] T 4556df24f388448b85b996e2f2ebe127 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.623838 30820 tablet_replica.cc:333] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.624505 30820 raft_consensus.cc:2243] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.625324 30820 raft_consensus.cc:2272] T 1220a04033fc478a909de6d8f46c51bf P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.627782 30820 tablet_replica.cc:333] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.628355 30820 raft_consensus.cc:2243] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.628871 30820 raft_consensus.cc:2272] T 643e68f4ae67439581a6a5a3c4df5ed6 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.631192 30820 tablet_replica.cc:333] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.631757 30820 raft_consensus.cc:2243] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.632494 30820 raft_consensus.cc:2272] T 29deab0b8b384dccbed4b383d96eb326 P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.634866 30820 tablet_replica.cc:333] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.635398 30820 raft_consensus.cc:2243] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.635813 30820 raft_consensus.cc:2272] T 73ec4c2a8f774679a048658e30d5708b P 25e1844b0ccb48adb22cc528da104bca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.637920 30820 tablet_replica.cc:333] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.638496 30820 raft_consensus.cc:2243] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.638916 30820 raft_consensus.cc:2272] T 81cf0923653b483f9ba448b97810d958 P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.641063 30820 tablet_replica.cc:333] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.641662 30820 raft_consensus.cc:2243] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.642563 30820 raft_consensus.cc:2272] T c35fc1d28d25478388654b8b37d694ff P 25e1844b0ccb48adb22cc528da104bca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.645426 30820 tablet_replica.cc:333] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.645960 30820 raft_consensus.cc:2243] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.646826 30820 raft_consensus.cc:2272] T 901b273b616542b3a23c6aaf861d7b7a P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.648968 30820 tablet_replica.cc:333] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.649466 30820 raft_consensus.cc:2243] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.649952 30820 raft_consensus.cc:2272] T 4faa9dae270549169ff4a40136ebd180 P 25e1844b0ccb48adb22cc528da104bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.652199 30820 tablet_replica.cc:333] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca: stopping tablet replica
I20260709 12:22:26.652698 30820 raft_consensus.cc:2243] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.653532 30820 raft_consensus.cc:2272] T ebc6723eb06e4d09b65f8d0d739d35dd P 25e1844b0ccb48adb22cc528da104bca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.687439 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
I20260709 12:22:26.712661 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
I20260709 12:22:26.737906 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:26.738708 30820 tablet_replica.cc:333] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.739567 30820 raft_consensus.cc:2243] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:22:26.740572 10461 consensus_peers.cc:597] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 -> Peer 8dec85d80d334e7d9f6c904a12fe3913 (127.30.25.1:41877): Couldn't send request to peer 8dec85d80d334e7d9f6c904a12fe3913. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:41877: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:26.740610 30820 raft_consensus.cc:2272] T 73ec4c2a8f774679a048658e30d5708b P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.743189 30820 tablet_replica.cc:333] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.743832 30820 raft_consensus.cc:2243] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.744261 30820 raft_consensus.cc:2272] T 29deab0b8b384dccbed4b383d96eb326 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.746290 30820 tablet_replica.cc:333] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.746958 30820 raft_consensus.cc:2243] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.747507 30820 raft_consensus.cc:2272] T 1220a04033fc478a909de6d8f46c51bf P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.750200 30820 tablet_replica.cc:333] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.750944 30820 raft_consensus.cc:2243] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.751873 30820 raft_consensus.cc:2272] T 643e68f4ae67439581a6a5a3c4df5ed6 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.754009 30820 tablet_replica.cc:333] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.754601 30820 raft_consensus.cc:2243] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.755018 30820 raft_consensus.cc:2272] T 4556df24f388448b85b996e2f2ebe127 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.757448 30820 tablet_replica.cc:333] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.758008 30820 raft_consensus.cc:2243] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.758471 30820 raft_consensus.cc:2272] T c35fc1d28d25478388654b8b37d694ff P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.761057 30820 tablet_replica.cc:333] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.761615 30820 raft_consensus.cc:2243] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.762468 30820 raft_consensus.cc:2272] T 81cf0923653b483f9ba448b97810d958 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.764534 30820 tablet_replica.cc:333] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.765062 30820 raft_consensus.cc:2243] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.765831 30820 raft_consensus.cc:2272] T 4faa9dae270549169ff4a40136ebd180 P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.768098 30820 tablet_replica.cc:333] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.768620 30820 raft_consensus.cc:2243] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.769069 30820 raft_consensus.cc:2272] T ebc6723eb06e4d09b65f8d0d739d35dd P 5ae1ebdf2c004893b66a1097f2b0af40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.771234 30820 tablet_replica.cc:333] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40: stopping tablet replica
I20260709 12:22:26.771723 30820 raft_consensus.cc:2243] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:26.772171 30820 raft_consensus.cc:2272] T 901b273b616542b3a23c6aaf861d7b7a P 5ae1ebdf2c004893b66a1097f2b0af40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.803715 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
I20260709 12:22:26.831022 30820 master.cc:562] Master@127.30.25.62:44227 shutting down...
I20260709 12:22:26.855806 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:26.856410 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:26.856782 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P c6955f88fdd24a41b8af969e8ce3db49: stopping tablet replica
I20260709 12:22:26.877003 30820 master.cc:584] Master@127.30.25.62:44227 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13063 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:22:26.925844 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:42837
I20260709 12:22:26.927067 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:26.932502 10835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:26.933336 10836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:26.933852 10838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:26.934724 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:26.935634 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:26.935853 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:26.936018 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599746936002 us; error 0 us; skew 500 ppm
I20260709 12:22:26.936586 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:26.939111 30820 webserver.cc:533] Webserver started at http://127.30.25.62:39573/ using document root <none> and password file <none>
I20260709 12:22:26.939631 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:26.939839 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:26.940119 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:26.941272 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "22f5f8b277a146d3b9b1059cc4122e09"
format_stamp: "Formatted at 2026-07-09 12:22:26 on dist-test-slave-rldg"
I20260709 12:22:26.945940 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:22:26.949644 10843 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:26.950627 30820 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:26.950903 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "22f5f8b277a146d3b9b1059cc4122e09"
format_stamp: "Formatted at 2026-07-09 12:22:26 on dist-test-slave-rldg"
I20260709 12:22:26.951145 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:26.971235 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:26.972513 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:27.011866 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:42837
I20260709 12:22:27.011997 10894 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:42837 every 8 connection(s)
I20260709 12:22:27.016306 10895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.029093 10895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09: Bootstrap starting.
I20260709 12:22:27.034469 10895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.039127 10895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09: No bootstrap required, opened a new log
I20260709 12:22:27.041477 10895 raft_consensus.cc:359] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f5f8b277a146d3b9b1059cc4122e09" member_type: VOTER }
I20260709 12:22:27.041909 10895 raft_consensus.cc:385] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.042162 10895 raft_consensus.cc:740] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f5f8b277a146d3b9b1059cc4122e09, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.042898 10895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f5f8b277a146d3b9b1059cc4122e09" member_type: VOTER }
I20260709 12:22:27.043388 10895 raft_consensus.cc:399] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:27.043629 10895 raft_consensus.cc:493] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:27.043910 10895 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.049031 10895 raft_consensus.cc:515] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f5f8b277a146d3b9b1059cc4122e09" member_type: VOTER }
I20260709 12:22:27.049633 10895 leader_election.cc:304] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22f5f8b277a146d3b9b1059cc4122e09; no voters: 
I20260709 12:22:27.050940 10895 leader_election.cc:290] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:27.051375 10898 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:27.052783 10898 raft_consensus.cc:697] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 1 LEADER]: Becoming Leader. State: Replica: 22f5f8b277a146d3b9b1059cc4122e09, State: Running, Role: LEADER
I20260709 12:22:27.053488 10898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f5f8b277a146d3b9b1059cc4122e09" member_type: VOTER }
I20260709 12:22:27.054201 10895 sys_catalog.cc:565] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:27.056329 10899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22f5f8b277a146d3b9b1059cc4122e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f5f8b277a146d3b9b1059cc4122e09" member_type: VOTER } }
I20260709 12:22:27.056352 10900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22f5f8b277a146d3b9b1059cc4122e09. Latest consensus state: current_term: 1 leader_uuid: "22f5f8b277a146d3b9b1059cc4122e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f5f8b277a146d3b9b1059cc4122e09" member_type: VOTER } }
I20260709 12:22:27.057138 10899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:27.057250 10900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:27.061362 10903 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:27.066264 10903 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:27.070206 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:27.075879 10903 catalog_manager.cc:1383] Generated new cluster ID: 7ac8e72992524b73a4e2f9275b53acdc
I20260709 12:22:27.076238 10903 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:27.094965 10903 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:27.096318 10903 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:27.108135 10903 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09: Generated new TSK 0
I20260709 12:22:27.108779 10903 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:27.137252 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:27.143949 10916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:27.145149 10917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.147043 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:27.147408 10919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.148406 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:27.148667 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:27.148841 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599747148820 us; error 0 us; skew 500 ppm
I20260709 12:22:27.149398 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:27.151844 30820 webserver.cc:533] Webserver started at http://127.30.25.1:33133/ using document root <none> and password file <none>
I20260709 12:22:27.152369 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:27.152583 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:27.152851 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:27.153977 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
format_stamp: "Formatted at 2026-07-09 12:22:27 on dist-test-slave-rldg"
I20260709 12:22:27.158687 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:22:27.161885 10924 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.162755 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:27.163053 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
format_stamp: "Formatted at 2026-07-09 12:22:27 on dist-test-slave-rldg"
I20260709 12:22:27.163349 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:27.177464 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:27.178741 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:27.180179 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:27.182471 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:27.182689 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.182930 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:27.183130 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.237996 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:35771
I20260709 12:22:27.238117 10986 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:35771 every 8 connection(s)
I20260709 12:22:27.243696 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:27.251624 10991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:27.254724 10992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.263149 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:27.264668 10994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.264835 10987 heartbeater.cc:344] Connected to a master server at 127.30.25.62:42837
I20260709 12:22:27.265448 10987 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:27.265960 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:27.266265 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:27.266496 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599747266477 us; error 0 us; skew 500 ppm
I20260709 12:22:27.266443 10987 heartbeater.cc:507] Master 127.30.25.62:42837 requested a full tablet report, sending...
I20260709 12:22:27.267416 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:27.269115 10860 ts_manager.cc:194] Registered new tserver with Master: ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771)
I20260709 12:22:27.270251 30820 webserver.cc:533] Webserver started at http://127.30.25.2:45831/ using document root <none> and password file <none>
I20260709 12:22:27.270980 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:27.271211 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:27.271487 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:27.271689 10860 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54916
I20260709 12:22:27.273108 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "dd504b787ff14f17a500784956acb300"
format_stamp: "Formatted at 2026-07-09 12:22:27 on dist-test-slave-rldg"
I20260709 12:22:27.277562 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 12:22:27.280841 10999 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.281616 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:22:27.281913 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "dd504b787ff14f17a500784956acb300"
format_stamp: "Formatted at 2026-07-09 12:22:27 on dist-test-slave-rldg"
I20260709 12:22:27.282202 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:27.299435 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:27.300659 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:27.302148 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:27.304337 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:27.304513 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.304690 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:27.304816 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.344282 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:35615
I20260709 12:22:27.344442 11061 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:35615 every 8 connection(s)
I20260709 12:22:27.349275 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:27.357198 11066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.360994 11062 heartbeater.cc:344] Connected to a master server at 127.30.25.62:42837
W20260709 12:22:27.361037 11065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.361737 11062 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:27.363082 11062 heartbeater.cc:507] Master 127.30.25.62:42837 requested a full tablet report, sending...
I20260709 12:22:27.364421 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:27.364604 11068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:27.365692 10860 ts_manager.cc:194] Registered new tserver with Master: dd504b787ff14f17a500784956acb300 (127.30.25.2:35615)
I20260709 12:22:27.365831 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:27.366173 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:27.366385 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599747366365 us; error 0 us; skew 500 ppm
I20260709 12:22:27.367054 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:27.367316 10860 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54932
I20260709 12:22:27.369838 30820 webserver.cc:533] Webserver started at http://127.30.25.3:38589/ using document root <none> and password file <none>
I20260709 12:22:27.370398 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:27.370575 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:27.370782 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:27.371898 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "2b7e983896b14c1d9949d52ace680648"
format_stamp: "Formatted at 2026-07-09 12:22:27 on dist-test-slave-rldg"
I20260709 12:22:27.376230 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:27.379436 11073 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.380194 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:27.380514 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "2b7e983896b14c1d9949d52ace680648"
format_stamp: "Formatted at 2026-07-09 12:22:27 on dist-test-slave-rldg"
I20260709 12:22:27.380791 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:27.390137 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:27.391430 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:27.392863 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:27.395093 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:27.395332 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.395582 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:27.395761 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:27.436220 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:41453
I20260709 12:22:27.436331 11135 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:41453 every 8 connection(s)
I20260709 12:22:27.450474 11136 heartbeater.cc:344] Connected to a master server at 127.30.25.62:42837
I20260709 12:22:27.450824 11136 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:27.451472 11136 heartbeater.cc:507] Master 127.30.25.62:42837 requested a full tablet report, sending...
I20260709 12:22:27.453378 10860 ts_manager.cc:194] Registered new tserver with Master: 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:27.454147 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014645072s
I20260709 12:22:27.454814 10860 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54942
I20260709 12:22:27.479336 10860 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54952:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:27.481436 10860 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:27.532214 10952 tablet_service.cc:1511] Processing CreateTablet for tablet 0c51ea123dd84efababd1793c641959b (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=05b3e633e7e84207b3db6e065e68bd8b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:27.532886 11101 tablet_service.cc:1511] Processing CreateTablet for tablet 0c51ea123dd84efababd1793c641959b (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=05b3e633e7e84207b3db6e065e68bd8b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:27.533762 10952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c51ea123dd84efababd1793c641959b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.533499 11100 tablet_service.cc:1511] Processing CreateTablet for tablet 2770f6bc84ec439ea651895ecc1fd3d9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=05b3e633e7e84207b3db6e065e68bd8b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:27.534448 11101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c51ea123dd84efababd1793c641959b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.535138 11100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2770f6bc84ec439ea651895ecc1fd3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.532989 10951 tablet_service.cc:1511] Processing CreateTablet for tablet 2770f6bc84ec439ea651895ecc1fd3d9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=05b3e633e7e84207b3db6e065e68bd8b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:27.536664 10951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2770f6bc84ec439ea651895ecc1fd3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.555454 11151 tablet_bootstrap.cc:492] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4: Bootstrap starting.
I20260709 12:22:27.556246 11027 tablet_service.cc:1511] Processing CreateTablet for tablet 0c51ea123dd84efababd1793c641959b (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=05b3e633e7e84207b3db6e065e68bd8b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:27.557660 11027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c51ea123dd84efababd1793c641959b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.559605 11026 tablet_service.cc:1511] Processing CreateTablet for tablet 2770f6bc84ec439ea651895ecc1fd3d9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=05b3e633e7e84207b3db6e065e68bd8b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:27.561110 11026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2770f6bc84ec439ea651895ecc1fd3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.564224 11151 tablet_bootstrap.cc:654] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.578685 11151 tablet_bootstrap.cc:492] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4: No bootstrap required, opened a new log
I20260709 12:22:27.579214 11151 ts_tablet_manager.cc:1403] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.011s
I20260709 12:22:27.582461 11151 raft_consensus.cc:359] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.583309 11151 raft_consensus.cc:385] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.583645 11151 raft_consensus.cc:740] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.584708 11151 consensus_queue.cc:260] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.602794 11152 tablet_bootstrap.cc:492] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648: Bootstrap starting.
I20260709 12:22:27.604020 10987 heartbeater.cc:499] Master 127.30.25.62:42837 was elected leader, sending a full tablet report...
I20260709 12:22:27.604561 11151 ts_tablet_manager.cc:1434] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.017s
I20260709 12:22:27.605607 11151 tablet_bootstrap.cc:492] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4: Bootstrap starting.
I20260709 12:22:27.611405 11152 tablet_bootstrap.cc:654] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.612458 11151 tablet_bootstrap.cc:654] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.622165 11151 tablet_bootstrap.cc:492] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4: No bootstrap required, opened a new log
I20260709 12:22:27.622807 11151 ts_tablet_manager.cc:1403] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.003s
I20260709 12:22:27.623294 11152 tablet_bootstrap.cc:492] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648: No bootstrap required, opened a new log
I20260709 12:22:27.623917 11152 ts_tablet_manager.cc:1403] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.012s
I20260709 12:22:27.624791 11155 tablet_bootstrap.cc:492] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300: Bootstrap starting.
I20260709 12:22:27.626084 11151 raft_consensus.cc:359] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.626969 11151 raft_consensus.cc:385] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.627338 11151 raft_consensus.cc:740] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.628626 11151 consensus_queue.cc:260] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.627291 11152 raft_consensus.cc:359] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.630205 11152 raft_consensus.cc:385] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.630635 11152 raft_consensus.cc:740] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7e983896b14c1d9949d52ace680648, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.631156 11151 ts_tablet_manager.cc:1434] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:27.631494 11152 consensus_queue.cc:260] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.633013 11155 tablet_bootstrap.cc:654] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.634133 11136 heartbeater.cc:499] Master 127.30.25.62:42837 was elected leader, sending a full tablet report...
I20260709 12:22:27.634683 11152 ts_tablet_manager.cc:1434] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:22:27.635690 11152 tablet_bootstrap.cc:492] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648: Bootstrap starting.
I20260709 12:22:27.641563 11155 tablet_bootstrap.cc:492] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300: No bootstrap required, opened a new log
I20260709 12:22:27.642238 11155 ts_tablet_manager.cc:1403] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:22:27.642467 11152 tablet_bootstrap.cc:654] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.645421 11155 raft_consensus.cc:359] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.646301 11155 raft_consensus.cc:385] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.646624 11155 raft_consensus.cc:740] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd504b787ff14f17a500784956acb300, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.647408 11155 consensus_queue.cc:260] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.648149 11152 tablet_bootstrap.cc:492] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648: No bootstrap required, opened a new log
I20260709 12:22:27.648730 11152 ts_tablet_manager.cc:1403] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:22:27.651556 11062 heartbeater.cc:499] Master 127.30.25.62:42837 was elected leader, sending a full tablet report...
I20260709 12:22:27.651932 11155 ts_tablet_manager.cc:1434] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:22:27.651880 11152 raft_consensus.cc:359] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.652757 11152 raft_consensus.cc:385] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.653162 11155 tablet_bootstrap.cc:492] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300: Bootstrap starting.
I20260709 12:22:27.653101 11152 raft_consensus.cc:740] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7e983896b14c1d9949d52ace680648, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.654016 11152 consensus_queue.cc:260] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.656368 11152 ts_tablet_manager.cc:1434] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:27.660818 11155 tablet_bootstrap.cc:654] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:27.666534 11155 tablet_bootstrap.cc:492] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300: No bootstrap required, opened a new log
I20260709 12:22:27.667040 11155 ts_tablet_manager.cc:1403] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 12:22:27.669255 11155 raft_consensus.cc:359] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.669808 11155 raft_consensus.cc:385] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:27.670056 11155 raft_consensus.cc:740] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd504b787ff14f17a500784956acb300, State: Initialized, Role: FOLLOWER
I20260709 12:22:27.670710 11155 consensus_queue.cc:260] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.672363 11155 ts_tablet_manager.cc:1434] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:27.680761 11154 raft_consensus.cc:493] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.681334 11154 raft_consensus.cc:515] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.683532 11154 leader_election.cc:290] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453), dd504b787ff14f17a500784956acb300 (127.30.25.2:35615)
I20260709 12:22:27.693727 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c51ea123dd84efababd1793c641959b" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7e983896b14c1d9949d52ace680648" is_pre_election: true
I20260709 12:22:27.694614 11111 raft_consensus.cc:2468] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 0.
I20260709 12:22:27.696060 10927 leader_election.cc:304] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7e983896b14c1d9949d52ace680648, ae725ac6f2b442c49daf84ac0eb7fcf4; no voters: 
I20260709 12:22:27.696110 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c51ea123dd84efababd1793c641959b" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300" is_pre_election: true
I20260709 12:22:27.696975 11037 raft_consensus.cc:2468] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 0.
I20260709 12:22:27.697054 11154 raft_consensus.cc:2804] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:27.697562 11154 raft_consensus.cc:493] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.697924 11154 raft_consensus.cc:3060] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.703655 11154 raft_consensus.cc:515] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.705346 11154 leader_election.cc:290] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 election: Requested vote from peers 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453), dd504b787ff14f17a500784956acb300 (127.30.25.2:35615)
I20260709 12:22:27.706391 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c51ea123dd84efababd1793c641959b" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7e983896b14c1d9949d52ace680648"
I20260709 12:22:27.706491 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c51ea123dd84efababd1793c641959b" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300"
I20260709 12:22:27.707055 11111 raft_consensus.cc:3060] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.707132 11037 raft_consensus.cc:3060] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.712778 11037 raft_consensus.cc:2468] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 1.
I20260709 12:22:27.712779 11111 raft_consensus.cc:2468] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 1.
I20260709 12:22:27.713874 10928 leader_election.cc:304] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae725ac6f2b442c49daf84ac0eb7fcf4, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:27.714638 11154 raft_consensus.cc:2804] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:27.715709 11154 raft_consensus.cc:697] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 LEADER]: Becoming Leader. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Running, Role: LEADER
I20260709 12:22:27.716393 11154 consensus_queue.cc:237] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } }
I20260709 12:22:27.723214 10859 catalog_manager.cc:5697] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:27.805951 11158 raft_consensus.cc:493] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.806586 11158 raft_consensus.cc:515] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.808900 11158 leader_election.cc:290] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771), dd504b787ff14f17a500784956acb300 (127.30.25.2:35615)
I20260709 12:22:27.819196 10962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2770f6bc84ec439ea651895ecc1fd3d9" candidate_uuid: "2b7e983896b14c1d9949d52ace680648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" is_pre_election: true
I20260709 12:22:27.820068 10962 raft_consensus.cc:2468] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b7e983896b14c1d9949d52ace680648 in term 0.
I20260709 12:22:27.820438 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2770f6bc84ec439ea651895ecc1fd3d9" candidate_uuid: "2b7e983896b14c1d9949d52ace680648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300" is_pre_election: true
I20260709 12:22:27.821019 11037 raft_consensus.cc:2468] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b7e983896b14c1d9949d52ace680648 in term 0.
I20260709 12:22:27.821148 11074 leader_election.cc:304] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7e983896b14c1d9949d52ace680648, ae725ac6f2b442c49daf84ac0eb7fcf4; no voters: 
I20260709 12:22:27.822005 11158 raft_consensus.cc:2804] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:27.822417 11158 raft_consensus.cc:493] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.822718 11158 raft_consensus.cc:3060] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.828063 11158 raft_consensus.cc:515] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.829658 11158 leader_election.cc:290] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 election: Requested vote from peers ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771), dd504b787ff14f17a500784956acb300 (127.30.25.2:35615)
I20260709 12:22:27.830494 10962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2770f6bc84ec439ea651895ecc1fd3d9" candidate_uuid: "2b7e983896b14c1d9949d52ace680648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
I20260709 12:22:27.830790 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2770f6bc84ec439ea651895ecc1fd3d9" candidate_uuid: "2b7e983896b14c1d9949d52ace680648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300"
I20260709 12:22:27.831162 10962 raft_consensus.cc:3060] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.831368 11037 raft_consensus.cc:3060] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:27.838531 11037 raft_consensus.cc:2468] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b7e983896b14c1d9949d52ace680648 in term 1.
I20260709 12:22:27.838531 10962 raft_consensus.cc:2468] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b7e983896b14c1d9949d52ace680648 in term 1.
I20260709 12:22:27.839812 11077 leader_election.cc:304] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7e983896b14c1d9949d52ace680648, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:27.840525 11158 raft_consensus.cc:2804] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:27.841583 11158 raft_consensus.cc:697] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 1 LEADER]: Becoming Leader. State: Replica: 2b7e983896b14c1d9949d52ace680648, State: Running, Role: LEADER
I20260709 12:22:27.842262 11158 consensus_queue.cc:237] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:27.849368 10859 catalog_manager.cc:5697] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b7e983896b14c1d9949d52ace680648 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "2b7e983896b14c1d9949d52ace680648" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:27.949682 10859 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54962:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:27.951756 10859 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:27.974627 10952 tablet_service.cc:1511] Processing CreateTablet for tablet ed46727f21e14628a595be080f251a51 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2d9df76fd53f463b860b36697d402890]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:27.975961 10952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed46727f21e14628a595be080f251a51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.977152 11101 tablet_service.cc:1511] Processing CreateTablet for tablet ed46727f21e14628a595be080f251a51 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2d9df76fd53f463b860b36697d402890]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:27.978427 11101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed46727f21e14628a595be080f251a51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.981071 11027 tablet_service.cc:1511] Processing CreateTablet for tablet 448642666d7e404fa7986321486c4d1a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2d9df76fd53f463b860b36697d402890]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:27.981981 11100 tablet_service.cc:1511] Processing CreateTablet for tablet 448642666d7e404fa7986321486c4d1a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2d9df76fd53f463b860b36697d402890]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:27.975963 11026 tablet_service.cc:1511] Processing CreateTablet for tablet ed46727f21e14628a595be080f251a51 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2d9df76fd53f463b860b36697d402890]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:27.982366 11027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448642666d7e404fa7986321486c4d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.983326 11100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448642666d7e404fa7986321486c4d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.984006 11026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed46727f21e14628a595be080f251a51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:27.983961 10951 tablet_service.cc:1511] Processing CreateTablet for tablet 448642666d7e404fa7986321486c4d1a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=2d9df76fd53f463b860b36697d402890]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:27.985364 10951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448642666d7e404fa7986321486c4d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:28.000420 11155 tablet_bootstrap.cc:492] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: Bootstrap starting.
I20260709 12:22:28.000648 11152 tablet_bootstrap.cc:492] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648: Bootstrap starting.
I20260709 12:22:28.007635 11152 tablet_bootstrap.cc:654] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:28.008981 11155 tablet_bootstrap.cc:654] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:28.012625 11151 tablet_bootstrap.cc:492] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: Bootstrap starting.
I20260709 12:22:28.019876 11151 tablet_bootstrap.cc:654] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:28.023715 11152 tablet_bootstrap.cc:492] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648: No bootstrap required, opened a new log
I20260709 12:22:28.023808 11155 tablet_bootstrap.cc:492] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: No bootstrap required, opened a new log
I20260709 12:22:28.024339 11152 ts_tablet_manager.cc:1403] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 12:22:28.024461 11155 ts_tablet_manager.cc:1403] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.004s
I20260709 12:22:28.025524 11151 tablet_bootstrap.cc:492] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: No bootstrap required, opened a new log
I20260709 12:22:28.026034 11151 ts_tablet_manager.cc:1403] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 12:22:28.027099 11152 raft_consensus.cc:359] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.027724 11152 raft_consensus.cc:385] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:28.028081 11152 raft_consensus.cc:740] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7e983896b14c1d9949d52ace680648, State: Initialized, Role: FOLLOWER
I20260709 12:22:28.027557 11155 raft_consensus.cc:359] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.028538 11155 raft_consensus.cc:385] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:28.028851 11155 raft_consensus.cc:740] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd504b787ff14f17a500784956acb300, State: Initialized, Role: FOLLOWER
I20260709 12:22:28.028867 11151 raft_consensus.cc:359] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.030085 11151 raft_consensus.cc:385] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:28.030483 11151 raft_consensus.cc:740] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Initialized, Role: FOLLOWER
I20260709 12:22:28.029395 11152 consensus_queue.cc:260] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.030468 11155 consensus_queue.cc:260] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.031545 11151 consensus_queue.cc:260] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.033763 11151 ts_tablet_manager.cc:1434] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:22:28.034822 11151 tablet_bootstrap.cc:492] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4: Bootstrap starting.
I20260709 12:22:28.036144 11152 ts_tablet_manager.cc:1434] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:22:28.037278 11152 tablet_bootstrap.cc:492] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648: Bootstrap starting.
I20260709 12:22:28.038936 11158 raft_consensus.cc:493] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.039534 11158 raft_consensus.cc:515] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.041420 11151 tablet_bootstrap.cc:654] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:28.041865 11158 leader_election.cc:290] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771), dd504b787ff14f17a500784956acb300 (127.30.25.2:35615)
I20260709 12:22:28.043989 11152 tablet_bootstrap.cc:654] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:28.044982 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448642666d7e404fa7986321486c4d1a" candidate_uuid: "2b7e983896b14c1d9949d52ace680648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300" is_pre_election: true
I20260709 12:22:28.045114 10962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448642666d7e404fa7986321486c4d1a" candidate_uuid: "2b7e983896b14c1d9949d52ace680648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" is_pre_election: true
W20260709 12:22:28.046666 11077 leader_election.cc:343] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd504b787ff14f17a500784956acb300 (127.30.25.2:35615): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:28.047525 11074 leader_election.cc:343] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:28.048133 11074 leader_election.cc:304] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b7e983896b14c1d9949d52ace680648; no voters: ae725ac6f2b442c49daf84ac0eb7fcf4, dd504b787ff14f17a500784956acb300
I20260709 12:22:28.049741 11167 raft_consensus.cc:2749] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:28.055892 11152 tablet_bootstrap.cc:492] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648: No bootstrap required, opened a new log
I20260709 12:22:28.056630 11152 ts_tablet_manager.cc:1403] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 12:22:28.057555 11155 ts_tablet_manager.cc:1434] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:22:28.061363 11155 tablet_bootstrap.cc:492] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300: Bootstrap starting.
I20260709 12:22:28.059522 11152 raft_consensus.cc:359] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.062566 11152 raft_consensus.cc:385] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:28.062971 11152 raft_consensus.cc:740] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7e983896b14c1d9949d52ace680648, State: Initialized, Role: FOLLOWER
W20260709 12:22:28.065088 10908 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:28.064893 11152 consensus_queue.cc:260] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.067487 11152 ts_tablet_manager.cc:1434] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:28.070142 11155 tablet_bootstrap.cc:654] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:28.072357 11151 tablet_bootstrap.cc:492] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4: No bootstrap required, opened a new log
I20260709 12:22:28.072962 11151 ts_tablet_manager.cc:1403] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.016s
I20260709 12:22:28.075945 11155 tablet_bootstrap.cc:492] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300: No bootstrap required, opened a new log
I20260709 12:22:28.076006 11151 raft_consensus.cc:359] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.076627 11155 ts_tablet_manager.cc:1403] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 12:22:28.076979 11151 raft_consensus.cc:385] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:28.077311 11151 raft_consensus.cc:740] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Initialized, Role: FOLLOWER
I20260709 12:22:28.078110 11151 consensus_queue.cc:260] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.079849 11155 raft_consensus.cc:359] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.080401 11151 ts_tablet_manager.cc:1434] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:28.080626 11155 raft_consensus.cc:385] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:28.081017 11155 raft_consensus.cc:740] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd504b787ff14f17a500784956acb300, State: Initialized, Role: FOLLOWER
I20260709 12:22:28.081703 11155 consensus_queue.cc:260] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.083863 11155 ts_tablet_manager.cc:1434] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:28.103197 11154 consensus_queue.cc:1048] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.103946 11164 raft_consensus.cc:493] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.104453 11164 raft_consensus.cc:515] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.106616 11164 leader_election.cc:290] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd504b787ff14f17a500784956acb300 (127.30.25.2:35615), 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:28.107511 11036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448642666d7e404fa7986321486c4d1a" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300" is_pre_election: true
I20260709 12:22:28.107755 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448642666d7e404fa7986321486c4d1a" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7e983896b14c1d9949d52ace680648" is_pre_election: true
I20260709 12:22:28.108263 11036 raft_consensus.cc:2468] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 0.
I20260709 12:22:28.108366 11111 raft_consensus.cc:2468] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 0.
I20260709 12:22:28.109674 10928 leader_election.cc:304] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae725ac6f2b442c49daf84ac0eb7fcf4, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:28.110478 11164 raft_consensus.cc:2804] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:28.110832 11164 raft_consensus.cc:493] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.111162 11164 raft_consensus.cc:3060] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:28.122045 11154 consensus_queue.cc:1048] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:28.120062 11164 raft_consensus.cc:515] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.126917 11164 leader_election.cc:290] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 election: Requested vote from peers dd504b787ff14f17a500784956acb300 (127.30.25.2:35615), 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:28.128278 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448642666d7e404fa7986321486c4d1a" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd504b787ff14f17a500784956acb300"
I20260709 12:22:28.129031 11037 raft_consensus.cc:3060] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:28.129581 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448642666d7e404fa7986321486c4d1a" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7e983896b14c1d9949d52ace680648"
I20260709 12:22:28.130196 11111 raft_consensus.cc:3060] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:28.136503 11037 raft_consensus.cc:2468] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 1.
I20260709 12:22:28.137671 10928 leader_election.cc:304] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae725ac6f2b442c49daf84ac0eb7fcf4, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:28.138610 11154 raft_consensus.cc:2804] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:28.139078 11154 raft_consensus.cc:697] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 LEADER]: Becoming Leader. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Running, Role: LEADER
I20260709 12:22:28.139420 11111 raft_consensus.cc:2468] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 1.
I20260709 12:22:28.139917 11154 consensus_queue.cc:237] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.148398 10858 catalog_manager.cc:5697] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:28.222142 11161 raft_consensus.cc:493] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.222723 11161 raft_consensus.cc:515] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.224920 11161 leader_election.cc:290] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771), 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:28.234596 10962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "dd504b787ff14f17a500784956acb300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" is_pre_election: true
I20260709 12:22:28.235167 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "dd504b787ff14f17a500784956acb300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7e983896b14c1d9949d52ace680648" is_pre_election: true
I20260709 12:22:28.235342 10962 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd504b787ff14f17a500784956acb300 in term 0.
I20260709 12:22:28.235862 11111 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd504b787ff14f17a500784956acb300 in term 0.
I20260709 12:22:28.236486 11000 leader_election.cc:304] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae725ac6f2b442c49daf84ac0eb7fcf4, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:28.237319 11161 raft_consensus.cc:2804] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:28.237735 11161 raft_consensus.cc:493] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.238009 11161 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:28.243149 11161 raft_consensus.cc:515] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.244791 11161 leader_election.cc:290] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [CANDIDATE]: Term 1 election: Requested vote from peers ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771), 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:28.245702 10962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "dd504b787ff14f17a500784956acb300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
I20260709 12:22:28.245891 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "dd504b787ff14f17a500784956acb300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7e983896b14c1d9949d52ace680648"
I20260709 12:22:28.246366 10962 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:28.246519 11111 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:28.251279 10962 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd504b787ff14f17a500784956acb300 in term 1.
I20260709 12:22:28.251348 11111 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd504b787ff14f17a500784956acb300 in term 1.
I20260709 12:22:28.252274 11000 leader_election.cc:304] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae725ac6f2b442c49daf84ac0eb7fcf4, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:28.253003 11161 raft_consensus.cc:2804] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:28.253919 11161 raft_consensus.cc:697] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 LEADER]: Becoming Leader. State: Replica: dd504b787ff14f17a500784956acb300, State: Running, Role: LEADER
I20260709 12:22:28.254715 11161 consensus_queue.cc:237] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.261560 10858 catalog_manager.cc:5697] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd504b787ff14f17a500784956acb300 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "dd504b787ff14f17a500784956acb300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:28.316131 11158 consensus_queue.cc:1048] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.331506 11158 consensus_queue.cc:1048] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.352152 11037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed46727f21e14628a595be080f251a51"
dest_uuid: "dd504b787ff14f17a500784956acb300"
mode: GRACEFUL
new_leader_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
 from {username='slave'} at 127.0.0.1:41064
I20260709 12:22:28.352736 11037 raft_consensus.cc:606] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 LEADER]: Received request to transfer leadership to TS ae725ac6f2b442c49daf84ac0eb7fcf4
I20260709 12:22:28.533193 11154 consensus_queue.cc:1048] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.546164 11154 consensus_queue.cc:1048] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.743175 11161 consensus_queue.cc:1048] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.753232 11161 consensus_queue.cc:1048] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:28.760895 11161 raft_consensus.cc:993] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: : Instructing follower ae725ac6f2b442c49daf84ac0eb7fcf4 to start an election
I20260709 12:22:28.761381 11202 raft_consensus.cc:1081] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 LEADER]: Signalling peer ae725ac6f2b442c49daf84ac0eb7fcf4 to start an election
I20260709 12:22:28.763041 10962 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed46727f21e14628a595be080f251a51"
dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
 from {username='slave'} at 127.0.0.1:58018
I20260709 12:22:28.763684 10962 raft_consensus.cc:493] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:28.764005 10962 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.769435 10962 raft_consensus.cc:515] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.771567 10962 leader_election.cc:290] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 2 election: Requested vote from peers dd504b787ff14f17a500784956acb300 (127.30.25.2:35615), 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:28.771946 11037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd504b787ff14f17a500784956acb300"
I20260709 12:22:28.772153 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2b7e983896b14c1d9949d52ace680648"
I20260709 12:22:28.772615 11037 raft_consensus.cc:3055] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:28.772826 11111 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.773025 11037 raft_consensus.cc:740] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd504b787ff14f17a500784956acb300, State: Running, Role: LEADER
I20260709 12:22:28.774000 11037 consensus_queue.cc:260] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.775401 11037 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.778997 11111 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 2.
I20260709 12:22:28.780246 10927 leader_election.cc:304] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7e983896b14c1d9949d52ace680648, ae725ac6f2b442c49daf84ac0eb7fcf4; no voters: 
I20260709 12:22:28.780967 11164 raft_consensus.cc:2804] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:28.781456 11164 raft_consensus.cc:697] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 LEADER]: Becoming Leader. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Running, Role: LEADER
I20260709 12:22:28.782209 11037 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae725ac6f2b442c49daf84ac0eb7fcf4 in term 2.
I20260709 12:22:28.782266 11164 consensus_queue.cc:237] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:28.788811 10859 catalog_manager.cc:5697] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 reported cstate change: term changed from 1 to 2, leader changed from dd504b787ff14f17a500784956acb300 (127.30.25.2) to ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:29.080956 30820 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:22:29.082679 30820 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:22:29.083133 30820 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:22:29.098121 10962 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed46727f21e14628a595be080f251a51"
dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
mode: GRACEFUL
new_leader_uuid: "dd504b787ff14f17a500784956acb300"
 from {username='slave'} at 127.0.0.1:58028
I20260709 12:22:29.098773 10962 raft_consensus.cc:606] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 LEADER]: Received request to transfer leadership to TS dd504b787ff14f17a500784956acb300
I20260709 12:22:29.099915 30820 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:22:29.236734 11111 raft_consensus.cc:1275] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 2 FOLLOWER]: Refusing update from remote peer ae725ac6f2b442c49daf84ac0eb7fcf4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:29.238185 11164 consensus_queue.cc:1048] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:29.248497 11037 raft_consensus.cc:1275] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 2 FOLLOWER]: Refusing update from remote peer ae725ac6f2b442c49daf84ac0eb7fcf4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:29.249866 11211 consensus_queue.cc:1048] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:29.256637 11185 raft_consensus.cc:993] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: : Instructing follower dd504b787ff14f17a500784956acb300 to start an election
I20260709 12:22:29.257148 11211 raft_consensus.cc:1081] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 LEADER]: Signalling peer dd504b787ff14f17a500784956acb300 to start an election
I20260709 12:22:29.258775 11037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed46727f21e14628a595be080f251a51"
dest_uuid: "dd504b787ff14f17a500784956acb300"
 from {username='slave'} at 127.0.0.1:41054
I20260709 12:22:29.259246 11037 raft_consensus.cc:493] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:29.259559 11037 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.264830 11037 raft_consensus.cc:515] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:29.266919 11037 leader_election.cc:290] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [CANDIDATE]: Term 3 election: Requested vote from peers ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771), 2b7e983896b14c1d9949d52ace680648 (127.30.25.3:41453)
I20260709 12:22:29.267241 10962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "dd504b787ff14f17a500784956acb300" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4"
I20260709 12:22:29.267557 11111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed46727f21e14628a595be080f251a51" candidate_uuid: "dd504b787ff14f17a500784956acb300" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2b7e983896b14c1d9949d52ace680648"
I20260709 12:22:29.267932 10962 raft_consensus.cc:3055] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:29.268148 11111 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.270416 10962 raft_consensus.cc:740] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ae725ac6f2b442c49daf84ac0eb7fcf4, State: Running, Role: LEADER
I20260709 12:22:29.271332 10962 consensus_queue.cc:260] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:29.272698 10962 raft_consensus.cc:3060] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.274683 11111 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd504b787ff14f17a500784956acb300 in term 3.
I20260709 12:22:29.275902 11002 leader_election.cc:304] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7e983896b14c1d9949d52ace680648, dd504b787ff14f17a500784956acb300; no voters: 
I20260709 12:22:29.276628 11202 raft_consensus.cc:2804] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:29.277590 11202 raft_consensus.cc:697] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 3 LEADER]: Becoming Leader. State: Replica: dd504b787ff14f17a500784956acb300, State: Running, Role: LEADER
I20260709 12:22:29.278437 11202 consensus_queue.cc:237] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } }
I20260709 12:22:29.278971 10962 raft_consensus.cc:2468] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd504b787ff14f17a500784956acb300 in term 3.
I20260709 12:22:29.284560 10858 catalog_manager.cc:5697] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 reported cstate change: term changed from 2 to 3, leader changed from ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1) to dd504b787ff14f17a500784956acb300 (127.30.25.2). New cstate: current_term: 3 leader_uuid: "dd504b787ff14f17a500784956acb300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae725ac6f2b442c49daf84ac0eb7fcf4" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd504b787ff14f17a500784956acb300" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 35615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b7e983896b14c1d9949d52ace680648" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:29.305547 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
I20260709 12:22:29.324366 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:29.324950 30820 tablet_replica.cc:333] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4: stopping tablet replica
I20260709 12:22:29.325515 30820 raft_consensus.cc:2243] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.326295 30820 raft_consensus.cc:2272] T 448642666d7e404fa7986321486c4d1a P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.328888 30820 tablet_replica.cc:333] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4: stopping tablet replica
I20260709 12:22:29.329490 30820 raft_consensus.cc:2243] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.330174 30820 raft_consensus.cc:2272] T ed46727f21e14628a595be080f251a51 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.332764 30820 tablet_replica.cc:333] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4: stopping tablet replica
I20260709 12:22:29.333240 30820 raft_consensus.cc:2243] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.333604 30820 raft_consensus.cc:2272] T 2770f6bc84ec439ea651895ecc1fd3d9 P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.335831 30820 tablet_replica.cc:333] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4: stopping tablet replica
I20260709 12:22:29.336472 30820 raft_consensus.cc:2243] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.337420 30820 raft_consensus.cc:2272] T 0c51ea123dd84efababd1793c641959b P ae725ac6f2b442c49daf84ac0eb7fcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:29.345659 11074 consensus_peers.cc:597] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 -> Peer ae725ac6f2b442c49daf84ac0eb7fcf4 (127.30.25.1:35771): Couldn't send request to peer ae725ac6f2b442c49daf84ac0eb7fcf4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:29.361038 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
I20260709 12:22:29.373487 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
I20260709 12:22:29.392000 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:29.392627 30820 tablet_replica.cc:333] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300: stopping tablet replica
I20260709 12:22:29.393194 30820 raft_consensus.cc:2243] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.393676 30820 raft_consensus.cc:2272] T 448642666d7e404fa7986321486c4d1a P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.395931 30820 tablet_replica.cc:333] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: stopping tablet replica
I20260709 12:22:29.396466 30820 raft_consensus.cc:2243] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.397090 30820 pending_rounds.cc:70] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: Trying to abort 1 pending ops.
I20260709 12:22:29.397284 30820 pending_rounds.cc:77] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305624573052534784 op_type: NO_OP noop_request { }
I20260709 12:22:29.397578 30820 raft_consensus.cc:2889] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:22:29.397853 30820 raft_consensus.cc:2272] T ed46727f21e14628a595be080f251a51 P dd504b787ff14f17a500784956acb300 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.399979 30820 tablet_replica.cc:333] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300: stopping tablet replica
I20260709 12:22:29.400513 30820 raft_consensus.cc:2243] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.400935 30820 raft_consensus.cc:2272] T 0c51ea123dd84efababd1793c641959b P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.402752 30820 tablet_replica.cc:333] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300: stopping tablet replica
I20260709 12:22:29.403199 30820 raft_consensus.cc:2243] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.403594 30820 raft_consensus.cc:2272] T 2770f6bc84ec439ea651895ecc1fd3d9 P dd504b787ff14f17a500784956acb300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.425347 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
I20260709 12:22:29.438243 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
I20260709 12:22:29.455904 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:29.456552 30820 tablet_replica.cc:333] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648: stopping tablet replica
I20260709 12:22:29.457135 30820 raft_consensus.cc:2243] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.457608 30820 raft_consensus.cc:2272] T ed46727f21e14628a595be080f251a51 P 2b7e983896b14c1d9949d52ace680648 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.460018 30820 tablet_replica.cc:333] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648: stopping tablet replica
I20260709 12:22:29.460546 30820 raft_consensus.cc:2243] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.460964 30820 raft_consensus.cc:2272] T 448642666d7e404fa7986321486c4d1a P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.463187 30820 tablet_replica.cc:333] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648: stopping tablet replica
I20260709 12:22:29.463683 30820 raft_consensus.cc:2243] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.464504 30820 raft_consensus.cc:2272] T 2770f6bc84ec439ea651895ecc1fd3d9 P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.466477 30820 tablet_replica.cc:333] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648: stopping tablet replica
I20260709 12:22:29.466961 30820 raft_consensus.cc:2243] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:29.467351 30820 raft_consensus.cc:2272] T 0c51ea123dd84efababd1793c641959b P 2b7e983896b14c1d9949d52ace680648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.488801 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
I20260709 12:22:29.501775 30820 master.cc:562] Master@127.30.25.62:42837 shutting down...
I20260709 12:22:29.517021 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:29.517598 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:29.517939 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P 22f5f8b277a146d3b9b1059cc4122e09: stopping tablet replica
I20260709 12:22:29.536613 30820 master.cc:584] Master@127.30.25.62:42837 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2637 ms)
[----------] 6 tests from LeaderRebalancerTest (410690 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:22:29.564502 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:43419
I20260709 12:22:29.566042 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:29.571641 11215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.571841 11216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.579236 11218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.580914 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:29.581801 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:29.581996 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:29.582108 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599749582097 us; error 0 us; skew 500 ppm
I20260709 12:22:29.582647 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:29.584849 30820 webserver.cc:533] Webserver started at http://127.30.25.62:39063/ using document root <none> and password file <none>
I20260709 12:22:29.585309 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:29.585462 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:29.585667 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:29.586804 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "4c3b84b09ea14fff9561745b1d7e3ac8"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.591230 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:22:29.594570 11223 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.595314 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:22:29.595561 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "4c3b84b09ea14fff9561745b1d7e3ac8"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.595798 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:29.618278 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:29.619292 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:29.654079 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:43419
I20260709 12:22:29.654184 11274 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:43419 every 8 connection(s)
I20260709 12:22:29.657862 11275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:29.668038 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8: Bootstrap starting.
I20260709 12:22:29.672320 11275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:29.676182 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8: No bootstrap required, opened a new log
I20260709 12:22:29.678097 11275 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" member_type: VOTER }
I20260709 12:22:29.678483 11275 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:29.678658 11275 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3b84b09ea14fff9561745b1d7e3ac8, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.679136 11275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" member_type: VOTER }
I20260709 12:22:29.679538 11275 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:29.679723 11275 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:29.679922 11275 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:29.684818 11275 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" member_type: VOTER }
I20260709 12:22:29.685299 11275 leader_election.cc:304] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c3b84b09ea14fff9561745b1d7e3ac8; no voters: 
I20260709 12:22:29.686472 11275 leader_election.cc:290] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:29.686790 11278 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:29.688194 11278 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 1 LEADER]: Becoming Leader. State: Replica: 4c3b84b09ea14fff9561745b1d7e3ac8, State: Running, Role: LEADER
I20260709 12:22:29.688846 11278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" member_type: VOTER }
I20260709 12:22:29.689580 11275 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:29.691893 11280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c3b84b09ea14fff9561745b1d7e3ac8. Latest consensus state: current_term: 1 leader_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" member_type: VOTER } }
I20260709 12:22:29.692446 11280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:29.691946 11279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3b84b09ea14fff9561745b1d7e3ac8" member_type: VOTER } }
I20260709 12:22:29.693467 11279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:29.703816 11283 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:29.711802 11283 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:29.712810 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:29.721036 11283 catalog_manager.cc:1383] Generated new cluster ID: 722ba1e4999e4661b16a9d24d1a0adc9
I20260709 12:22:29.721367 11283 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:29.738082 11283 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:29.739495 11283 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:29.752899 11283 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8: Generated new TSK 0
I20260709 12:22:29.753544 11283 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:29.779471 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:29.785831 11296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.787230 11297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.789116 11299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.789479 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:29.790520 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:29.790724 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:29.790903 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599749790882 us; error 0 us; skew 500 ppm
I20260709 12:22:29.791630 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:29.793848 30820 webserver.cc:533] Webserver started at http://127.30.25.1:45505/ using document root <none> and password file <none>
I20260709 12:22:29.794400 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:29.794595 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:29.794883 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:29.795940 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "408b52ec62254d8e99818a554befd6e3"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.800383 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:29.803714 11304 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.804486 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:29.804777 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "408b52ec62254d8e99818a554befd6e3"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.805073 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:29.829505 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:29.830668 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:29.832129 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:29.834399 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:29.834620 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.834863 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:29.835033 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.872712 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:43947
I20260709 12:22:29.872833 11366 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:43947 every 8 connection(s)
I20260709 12:22:29.878052 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:29.885289 11371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.886868 11372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.892223 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:29.892736 11374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.893858 11367 heartbeater.cc:344] Connected to a master server at 127.30.25.62:43419
I20260709 12:22:29.894270 11367 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:29.894419 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:29.894733 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:29.894909 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599749894882 us; error 0 us; skew 500 ppm
I20260709 12:22:29.895195 11367 heartbeater.cc:507] Master 127.30.25.62:43419 requested a full tablet report, sending...
I20260709 12:22:29.895573 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:29.897557 11240 ts_manager.cc:194] Registered new tserver with Master: 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947)
I20260709 12:22:29.898430 30820 webserver.cc:533] Webserver started at http://127.30.25.2:34151/ using document root <none> and password file <none>
I20260709 12:22:29.899128 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:29.899382 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:29.899677 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:29.900038 11240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43240
I20260709 12:22:29.901119 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "7be56f2c081b41b6939235ccfd24abb1"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.905599 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:22:29.908869 11379 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.909658 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:22:29.909940 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "7be56f2c081b41b6939235ccfd24abb1"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.910211 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:29.923710 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:29.924788 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:29.926148 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:29.928392 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:29.928612 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.928850 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:29.929019 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.966217 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:46599
I20260709 12:22:29.966377 11441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:46599 every 8 connection(s)
I20260709 12:22:29.971101 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:29.977896 11445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.978832 11446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.981531 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:29.982686 11449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.983585 11442 heartbeater.cc:344] Connected to a master server at 127.30.25.62:43419
I20260709 12:22:29.984017 11442 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:29.983966 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:29.984500 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:29.984727 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599749984709 us; error 0 us; skew 500 ppm
I20260709 12:22:29.985083 11442 heartbeater.cc:507] Master 127.30.25.62:43419 requested a full tablet report, sending...
I20260709 12:22:29.985544 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:29.987577 11240 ts_manager.cc:194] Registered new tserver with Master: 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:29.988432 30820 webserver.cc:533] Webserver started at http://127.30.25.3:41163/ using document root <none> and password file <none>
I20260709 12:22:29.989131 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:29.989368 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:29.989472 11240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43256
I20260709 12:22:29.989704 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:29.991323 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "4abc425940dd4061af4d588e0cf78ba2"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:29.996399 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:29.999627 11453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:30.000471 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:30.000805 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "4abc425940dd4061af4d588e0cf78ba2"
format_stamp: "Formatted at 2026-07-09 12:22:29 on dist-test-slave-rldg"
I20260709 12:22:30.001096 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:30.048871 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:30.050033 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:30.051537 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:30.053791 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:30.054026 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:30.054270 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:30.054467 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:22:30.090919 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:34613
I20260709 12:22:30.091027 11515 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:34613 every 8 connection(s)
I20260709 12:22:30.104364 11516 heartbeater.cc:344] Connected to a master server at 127.30.25.62:43419
I20260709 12:22:30.104831 11516 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:30.105572 11516 heartbeater.cc:507] Master 127.30.25.62:43419 requested a full tablet report, sending...
I20260709 12:22:30.107508 11240 ts_manager.cc:194] Registered new tserver with Master: 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.107648 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013386518s
I20260709 12:22:30.109297 11240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43266
I20260709 12:22:30.132184 11240 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43268:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:30.134670 11240 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:30.195927 11407 tablet_service.cc:1511] Processing CreateTablet for tablet 97de4708d35e47afb683b92846175828 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:30.197722 11407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97de4708d35e47afb683b92846175828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.206713 11406 tablet_service.cc:1511] Processing CreateTablet for tablet 1998e87dc4b14e578102b92e9bfa9f03 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:30.208735 11406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1998e87dc4b14e578102b92e9bfa9f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.221369 11405 tablet_service.cc:1511] Processing CreateTablet for tablet 07af2adb5e0745088a20cdf76ead9d5c (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:30.221658 11481 tablet_service.cc:1511] Processing CreateTablet for tablet 97de4708d35e47afb683b92846175828 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:30.223101 11481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97de4708d35e47afb683b92846175828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.223222 11405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07af2adb5e0745088a20cdf76ead9d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.227774 11331 tablet_service.cc:1511] Processing CreateTablet for tablet 1998e87dc4b14e578102b92e9bfa9f03 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:30.228116 11332 tablet_service.cc:1511] Processing CreateTablet for tablet 97de4708d35e47afb683b92846175828 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:30.229347 11331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1998e87dc4b14e578102b92e9bfa9f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.231091 11329 tablet_service.cc:1511] Processing CreateTablet for tablet 9a35d4770d87426aab884733ee2c3d59 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:30.231346 11330 tablet_service.cc:1511] Processing CreateTablet for tablet 07af2adb5e0745088a20cdf76ead9d5c (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:30.235241 11404 tablet_service.cc:1511] Processing CreateTablet for tablet 9a35d4770d87426aab884733ee2c3d59 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:30.235566 11328 tablet_service.cc:1511] Processing CreateTablet for tablet b0b2319fba4945e2a673d0c96fe90161 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:30.236794 11404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a35d4770d87426aab884733ee2c3d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.236935 11480 tablet_service.cc:1511] Processing CreateTablet for tablet 1998e87dc4b14e578102b92e9bfa9f03 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:30.238386 11480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1998e87dc4b14e578102b92e9bfa9f03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.239918 11403 tablet_service.cc:1511] Processing CreateTablet for tablet b0b2319fba4945e2a673d0c96fe90161 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:30.240171 11327 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd96fbe8d2944b5859796e118975195 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:30.241384 11403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b2319fba4945e2a673d0c96fe90161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.243234 11329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a35d4770d87426aab884733ee2c3d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.244850 11326 tablet_service.cc:1511] Processing CreateTablet for tablet e8315b03aa974b2b8cc5c155f18f2248 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:30.246165 11326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8315b03aa974b2b8cc5c155f18f2248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.248351 11402 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd96fbe8d2944b5859796e118975195 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:30.249709 11402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd96fbe8d2944b5859796e118975195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.249428 11325 tablet_service.cc:1511] Processing CreateTablet for tablet 5d1eda1818ba4ecdb846e13977678111 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:30.250794 11479 tablet_service.cc:1511] Processing CreateTablet for tablet 07af2adb5e0745088a20cdf76ead9d5c (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:30.251394 11401 tablet_service.cc:1511] Processing CreateTablet for tablet e8315b03aa974b2b8cc5c155f18f2248 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:30.252213 11479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07af2adb5e0745088a20cdf76ead9d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.252720 11401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8315b03aa974b2b8cc5c155f18f2248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.254120 11324 tablet_service.cc:1511] Processing CreateTablet for tablet 7ec255b57d0c4527aae93cdfaa757e51 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:30.250782 11325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d1eda1818ba4ecdb846e13977678111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.260486 11399 tablet_service.cc:1511] Processing CreateTablet for tablet 7ec255b57d0c4527aae93cdfaa757e51 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:30.256364 11400 tablet_service.cc:1511] Processing CreateTablet for tablet 5d1eda1818ba4ecdb846e13977678111 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:30.264262 11327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd96fbe8d2944b5859796e118975195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.266631 11328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b2319fba4945e2a673d0c96fe90161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.269243 11332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97de4708d35e47afb683b92846175828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.270654 11324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec255b57d0c4527aae93cdfaa757e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.274814 11330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07af2adb5e0745088a20cdf76ead9d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.278893 11399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec255b57d0c4527aae93cdfaa757e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.282660 11400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d1eda1818ba4ecdb846e13977678111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.308445 11478 tablet_service.cc:1511] Processing CreateTablet for tablet 9a35d4770d87426aab884733ee2c3d59 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:30.309994 11478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a35d4770d87426aab884733ee2c3d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.330744 11531 tablet_bootstrap.cc:492] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.346099 11477 tablet_service.cc:1511] Processing CreateTablet for tablet b0b2319fba4945e2a673d0c96fe90161 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:30.347730 11477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b2319fba4945e2a673d0c96fe90161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.351895 11476 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd96fbe8d2944b5859796e118975195 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:30.353868 11476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd96fbe8d2944b5859796e118975195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.355832 11475 tablet_service.cc:1511] Processing CreateTablet for tablet e8315b03aa974b2b8cc5c155f18f2248 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:30.359019 11474 tablet_service.cc:1511] Processing CreateTablet for tablet 5d1eda1818ba4ecdb846e13977678111 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:30.361972 11473 tablet_service.cc:1511] Processing CreateTablet for tablet 7ec255b57d0c4527aae93cdfaa757e51 (DEFAULT_TABLE table=test-workload [id=87060bccda5241168f035bf9d49ec00c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:30.365115 11475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8315b03aa974b2b8cc5c155f18f2248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.370663 11474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d1eda1818ba4ecdb846e13977678111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.372541 11473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec255b57d0c4527aae93cdfaa757e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:30.399127 11532 tablet_bootstrap.cc:492] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.402954 11531 tablet_bootstrap.cc:654] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.403374 11533 tablet_bootstrap.cc:492] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.410853 11533 tablet_bootstrap.cc:654] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.422395 11532 tablet_bootstrap.cc:654] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.428870 11533 tablet_bootstrap.cc:492] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.429239 11531 tablet_bootstrap.cc:492] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.429380 11533 ts_tablet_manager.cc:1403] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.000s
I20260709 12:22:30.430048 11531 ts_tablet_manager.cc:1403] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.100s	user 0.009s	sys 0.007s
I20260709 12:22:30.431902 11532 tablet_bootstrap.cc:492] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.432442 11532 ts_tablet_manager.cc:1403] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.006s
I20260709 12:22:30.432770 11533 raft_consensus.cc:359] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.433137 11531 raft_consensus.cc:359] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.434029 11531 raft_consensus.cc:385] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.434415 11531 raft_consensus.cc:740] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.434603 11533 raft_consensus.cc:385] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.435029 11533 raft_consensus.cc:740] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.435356 11531 consensus_queue.cc:260] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.435462 11532 raft_consensus.cc:359] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.436371 11532 raft_consensus.cc:385] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.435943 11533 consensus_queue.cc:260] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.436693 11532 raft_consensus.cc:740] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.437696 11532 consensus_queue.cc:260] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.439801 11516 heartbeater.cc:499] Master 127.30.25.62:43419 was elected leader, sending a full tablet report...
I20260709 12:22:30.440399 11533 ts_tablet_manager.cc:1434] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:22:30.441809 11533 tablet_bootstrap.cc:492] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.443382 11367 heartbeater.cc:499] Master 127.30.25.62:43419 was elected leader, sending a full tablet report...
I20260709 12:22:30.444442 11532 ts_tablet_manager.cc:1434] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.005s
I20260709 12:22:30.445523 11532 tablet_bootstrap.cc:492] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.451442 11442 heartbeater.cc:499] Master 127.30.25.62:43419 was elected leader, sending a full tablet report...
I20260709 12:22:30.451972 11531 ts_tablet_manager.cc:1434] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.010s
I20260709 12:22:30.453038 11531 tablet_bootstrap.cc:492] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.453529 11533 tablet_bootstrap.cc:654] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.458652 11532 tablet_bootstrap.cc:654] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.459707 11531 tablet_bootstrap.cc:654] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.465822 11533 tablet_bootstrap.cc:492] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.466503 11533 ts_tablet_manager.cc:1403] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.008s
I20260709 12:22:30.469744 11533 raft_consensus.cc:359] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.470487 11533 raft_consensus.cc:385] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.470677 11532 tablet_bootstrap.cc:492] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.470777 11533 raft_consensus.cc:740] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.471252 11532 ts_tablet_manager.cc:1403] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.006s
I20260709 12:22:30.471889 11533 consensus_queue.cc:260] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.474118 11533 ts_tablet_manager.cc:1434] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:30.474234 11532 raft_consensus.cc:359] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.475057 11532 raft_consensus.cc:385] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.475128 11533 tablet_bootstrap.cc:492] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.475414 11532 raft_consensus.cc:740] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.476336 11532 consensus_queue.cc:260] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.478852 11532 ts_tablet_manager.cc:1434] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:22:30.479877 11532 tablet_bootstrap.cc:492] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.482350 11533 tablet_bootstrap.cc:654] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.486639 11532 tablet_bootstrap.cc:654] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.496764 11533 tablet_bootstrap.cc:492] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.497380 11533 ts_tablet_manager.cc:1403] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 12:22:30.499630 11532 tablet_bootstrap.cc:492] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.500254 11532 ts_tablet_manager.cc:1403] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:22:30.500646 11533 raft_consensus.cc:359] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.501516 11533 raft_consensus.cc:385] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.501842 11533 raft_consensus.cc:740] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.502996 11533 consensus_queue.cc:260] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.503453 11532 raft_consensus.cc:359] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.504441 11532 raft_consensus.cc:385] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.504777 11532 raft_consensus.cc:740] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.505556 11533 ts_tablet_manager.cc:1434] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:30.505714 11532 consensus_queue.cc:260] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.506878 11533 tablet_bootstrap.cc:492] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.508790 11532 ts_tablet_manager.cc:1434] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:22:30.510079 11532 tablet_bootstrap.cc:492] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.513453 11531 tablet_bootstrap.cc:492] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.514076 11531 ts_tablet_manager.cc:1403] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.042s
I20260709 12:22:30.514279 11533 tablet_bootstrap.cc:654] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.516726 11532 tablet_bootstrap.cc:654] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.517453 11531 raft_consensus.cc:359] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.518329 11531 raft_consensus.cc:385] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.518646 11531 raft_consensus.cc:740] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.519551 11531 consensus_queue.cc:260] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.522058 11531 ts_tablet_manager.cc:1434] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:22:30.523211 11531 tablet_bootstrap.cc:492] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.526297 11538 raft_consensus.cc:493] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.527076 11538 raft_consensus.cc:515] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.529913 11532 tablet_bootstrap.cc:492] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.529963 11531 tablet_bootstrap.cc:654] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.530628 11532 ts_tablet_manager.cc:1403] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:22:30.534050 11532 raft_consensus.cc:359] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.534958 11532 raft_consensus.cc:385] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.535290 11532 raft_consensus.cc:740] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.535311 11538 leader_election.cc:290] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:30.536579 11532 consensus_queue.cc:260] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.540288 11532 ts_tablet_manager.cc:1434] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:22:30.541877 11532 tablet_bootstrap.cc:492] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.545208 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af2adb5e0745088a20cdf76ead9d5c" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
W20260709 12:22:30.547187 11455 leader_election.cc:343] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.549669 11532 tablet_bootstrap.cc:654] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.557299 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af2adb5e0745088a20cdf76ead9d5c" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.558207 11539 raft_consensus.cc:493] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:30.559329 11456 leader_election.cc:343] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.558974 11539 raft_consensus.cc:515] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.559895 11456 leader_election.cc:304] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:30.560928 11538 raft_consensus.cc:2749] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:30.562813 11533 tablet_bootstrap.cc:492] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.563380 11533 ts_tablet_manager.cc:1403] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.013s
I20260709 12:22:30.570298 11533 raft_consensus.cc:359] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.571357 11533 raft_consensus.cc:385] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.571647 11533 raft_consensus.cc:740] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.572350 11533 consensus_queue.cc:260] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.573398 11531 tablet_bootstrap.cc:492] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.573894 11539 leader_election.cc:290] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.574169 11531 ts_tablet_manager.cc:1403] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.051s	user 0.025s	sys 0.006s
I20260709 12:22:30.574597 11533 ts_tablet_manager.cc:1434] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:22:30.575922 11533 tablet_bootstrap.cc:492] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.578722 11531 raft_consensus.cc:359] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.579527 11531 raft_consensus.cc:385] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.580624 11532 tablet_bootstrap.cc:492] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.580502 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.582245 11532 ts_tablet_manager.cc:1403] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.004s
W20260709 12:22:30.582577 11307 leader_election.cc:343] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.585424 11532 raft_consensus.cc:359] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.579893 11531 raft_consensus.cc:740] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.586233 11532 raft_consensus.cc:385] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.586743 11532 raft_consensus.cc:740] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.587136 11531 consensus_queue.cc:260] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.587571 11532 consensus_queue.cc:260] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.590173 11533 tablet_bootstrap.cc:654] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.590723 11539 raft_consensus.cc:493] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.591259 11539 raft_consensus.cc:515] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.592456 11532 ts_tablet_manager.cc:1434] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:30.593552 11532 tablet_bootstrap.cc:492] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.594069 11539 leader_election.cc:290] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.597836 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.599177 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
W20260709 12:22:30.599483 11307 leader_election.cc:343] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.600112 11532 tablet_bootstrap.cc:654] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.600004 11491 raft_consensus.cc:2468] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:30.600888 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.601595 11539 raft_consensus.cc:493] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.601740 11308 leader_election.cc:304] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:30.602175 11539 raft_consensus.cc:515] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.603888 11553 raft_consensus.cc:2804] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:30.604022 11308 leader_election.cc:343] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.604681 11539 leader_election.cc:290] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.604769 11308 leader_election.cc:304] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:30.605420 11553 raft_consensus.cc:493] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.606283 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.608106 11417 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:30.608333 11553 raft_consensus.cc:3060] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.608074 11539 raft_consensus.cc:2749] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:30.610293 11307 leader_election.cc:304] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:30.612107 11531 ts_tablet_manager.cc:1434] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 12:22:30.613201 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.617718 11490 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:30.618409 11531 tablet_bootstrap.cc:492] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.615192 11555 raft_consensus.cc:2804] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:30.617092 11533 tablet_bootstrap.cc:492] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.617707 11532 tablet_bootstrap.cc:492] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.619549 11553 raft_consensus.cc:515] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.627277 11532 ts_tablet_manager.cc:1403] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.016s
I20260709 12:22:30.630110 11531 tablet_bootstrap.cc:654] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.629878 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:30.630473 11555 raft_consensus.cc:493] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.630904 11555 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.630774 11532 raft_consensus.cc:359] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.631719 11532 raft_consensus.cc:385] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.632095 11532 raft_consensus.cc:740] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.627004 11533 ts_tablet_manager.cc:1403] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.004s
I20260709 12:22:30.633304 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:30.632995 11532 consensus_queue.cc:260] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.633905 11490 raft_consensus.cc:3060] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.636874 11533 raft_consensus.cc:359] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
W20260709 12:22:30.632556 11307 leader_election.cc:343] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.638062 11532 ts_tablet_manager.cc:1434] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:22:30.639523 11532 tablet_bootstrap.cc:492] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.645152 11537 raft_consensus.cc:493] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.645713 11537 raft_consensus.cc:515] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.646373 11532 tablet_bootstrap.cc:654] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.648375 11555 raft_consensus.cc:515] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.638003 11533 raft_consensus.cc:385] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.649672 11533 raft_consensus.cc:740] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.650524 11533 consensus_queue.cc:260] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.652580 11539 raft_consensus.cc:493] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.652959 11533 ts_tablet_manager.cc:1434] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:22:30.653144 11539 raft_consensus.cc:515] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.654109 11533 tablet_bootstrap.cc:492] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.655314 11539 leader_election.cc:290] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.657357 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1998e87dc4b14e578102b92e9bfa9f03" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.658077 11491 raft_consensus.cc:2468] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:30.658860 11537 leader_election.cc:290] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613), 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947)
I20260709 12:22:30.661139 11308 leader_election.cc:304] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:30.662348 11539 raft_consensus.cc:2804] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:30.662837 11539 raft_consensus.cc:493] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.661149 11533 tablet_bootstrap.cc:654] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.663245 11539 raft_consensus.cc:3060] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.672125 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.672976 11491 raft_consensus.cc:2468] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:30.674708 11383 leader_election.cc:304] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:30.675601 11555 leader_election.cc:290] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.677132 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:30.678061 11491 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.679728 11532 tablet_bootstrap.cc:492] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.680161 11537 raft_consensus.cc:2804] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:30.680337 11532 ts_tablet_manager.cc:1403] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.002s
I20260709 12:22:30.680677 11537 raft_consensus.cc:493] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.681138 11537 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.683389 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1998e87dc4b14e578102b92e9bfa9f03" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.683508 11532 raft_consensus.cc:359] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.684302 11532 raft_consensus.cc:385] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.684654 11532 raft_consensus.cc:740] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
W20260709 12:22:30.685683 11307 leader_election.cc:343] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.685726 11532 consensus_queue.cc:260] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.687405 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:30.688112 11417 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.689635 11532 ts_tablet_manager.cc:1434] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:22:30.690762 11532 tablet_bootstrap.cc:492] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.692018 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
W20260709 12:22:30.698964 11381 leader_election.cc:343] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.701431 11532 tablet_bootstrap.cc:654] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:30.705065 11288 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:30.714789 11561 raft_consensus.cc:493] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.716784 11561 raft_consensus.cc:515] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.720822 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
W20260709 12:22:30.722424 11381 leader_election.cc:343] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.725389 11490 raft_consensus.cc:2468] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:30.726013 11489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.726832 11308 leader_election.cc:304] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 7be56f2c081b41b6939235ccfd24abb1
W20260709 12:22:30.727666 11383 leader_election.cc:343] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.728533 11383 leader_election.cc:304] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7be56f2c081b41b6939235ccfd24abb1; no voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:30.727890 11553 leader_election.cc:290] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.727698 11555 raft_consensus.cc:2804] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:30.736195 11555 raft_consensus.cc:697] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:30.737506 11555 consensus_queue.cc:237] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.739637 11561 leader_election.cc:290] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.740301 11561 raft_consensus.cc:2749] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:30.744155 11537 raft_consensus.cc:515] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.745985 11537 leader_election.cc:290] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613), 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947)
I20260709 12:22:30.729928 11417 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:30.747848 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:30.749684 11539 raft_consensus.cc:515] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.749735 11307 leader_election.cc:304] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
W20260709 12:22:30.751145 11381 leader_election.cc:343] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.752005 11539 leader_election.cc:290] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.752591 11539 raft_consensus.cc:2804] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:30.753064 11539 raft_consensus.cc:697] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:30.753757 11489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1998e87dc4b14e578102b92e9bfa9f03" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:30.754446 11489 raft_consensus.cc:3060] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.756912 11491 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:30.763091 11532 tablet_bootstrap.cc:492] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.763635 11532 ts_tablet_manager.cc:1403] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.034s
I20260709 12:22:30.764240 11489 raft_consensus.cc:2468] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:30.767115 11308 leader_election.cc:304] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:30.768219 11564 raft_consensus.cc:2804] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:30.768756 11564 raft_consensus.cc:697] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:30.769580 11532 raft_consensus.cc:359] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.768971 11237 catalog_manager.cc:5697] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.770367 11532 raft_consensus.cc:385] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.769894 11564 consensus_queue.cc:237] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.770709 11532 raft_consensus.cc:740] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.765024 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1998e87dc4b14e578102b92e9bfa9f03" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:30.771435 11532 consensus_queue.cc:260] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.765780 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:30.772686 11490 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.773479 11532 ts_tablet_manager.cc:1434] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 12:22:30.769095 11533 tablet_bootstrap.cc:492] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.774778 11533 ts_tablet_manager.cc:1403] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.026s
W20260709 12:22:30.774893 11307 leader_election.cc:343] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.753890 11539 consensus_queue.cc:237] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.781553 11533 raft_consensus.cc:359] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.782382 11533 raft_consensus.cc:385] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.782685 11533 raft_consensus.cc:740] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.783497 11533 consensus_queue.cc:260] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.784616 11490 raft_consensus.cc:2468] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:30.785043 11531 tablet_bootstrap.cc:492] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.785696 11531 ts_tablet_manager.cc:1403] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.168s	user 0.047s	sys 0.084s
I20260709 12:22:30.785889 11533 ts_tablet_manager.cc:1434] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:22:30.786043 11383 leader_election.cc:304] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 408b52ec62254d8e99818a554befd6e3
I20260709 12:22:30.787106 11537 raft_consensus.cc:2804] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:30.786450 11532 tablet_bootstrap.cc:492] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:30.788311 11537 raft_consensus.cc:697] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:30.789047 11531 raft_consensus.cc:359] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.789224 11537 consensus_queue.cc:237] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.789986 11531 raft_consensus.cc:385] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.790408 11531 raft_consensus.cc:740] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.796036 11531 consensus_queue.cc:260] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.798786 11531 ts_tablet_manager.cc:1434] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:22:30.800204 11531 tablet_bootstrap.cc:492] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.793653 11532 tablet_bootstrap.cc:654] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.794124 11533 tablet_bootstrap.cc:492] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.812858 11532 tablet_bootstrap.cc:492] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:30.813468 11532 ts_tablet_manager.cc:1403] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.004s
I20260709 12:22:30.813516 11533 tablet_bootstrap.cc:654] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.815390 11531 tablet_bootstrap.cc:654] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.816574 11532 raft_consensus.cc:359] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.817412 11532 raft_consensus.cc:385] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.817749 11532 raft_consensus.cc:740] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.818617 11532 consensus_queue.cc:260] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:30.820983 11532 ts_tablet_manager.cc:1434] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:22:30.824270 11539 raft_consensus.cc:493] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.824904 11539 raft_consensus.cc:515] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.824433 11237 catalog_manager.cc:5697] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.826670 11531 tablet_bootstrap.cc:492] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.827350 11531 ts_tablet_manager.cc:1403] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.006s
I20260709 12:22:30.828377 11533 tablet_bootstrap.cc:492] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.828980 11533 ts_tablet_manager.cc:1403] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.007s
I20260709 12:22:30.829221 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.829638 11237 catalog_manager.cc:5697] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.830533 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.830734 11531 raft_consensus.cc:359] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.831634 11531 raft_consensus.cc:385] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.832013 11531 raft_consensus.cc:740] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
W20260709 12:22:30.831873 11308 leader_election.cc:343] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.832072 11533 raft_consensus.cc:359] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.833058 11533 raft_consensus.cc:385] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.832099 11539 leader_election.cc:290] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.833537 11533 raft_consensus.cc:740] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.833117 11531 consensus_queue.cc:260] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.829242 11240 catalog_manager.cc:5697] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.834828 11533 consensus_queue.cc:260] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.836072 11531 ts_tablet_manager.cc:1434] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:30.834964 11417 raft_consensus.cc:2468] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:30.837157 11531 tablet_bootstrap.cc:492] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.840660 11307 leader_election.cc:304] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:30.841691 11533 ts_tablet_manager.cc:1434] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:22:30.842001 11539 raft_consensus.cc:2804] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:30.842484 11539 raft_consensus.cc:493] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.842859 11533 tablet_bootstrap.cc:492] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.842883 11539 raft_consensus.cc:3060] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.844219 11531 tablet_bootstrap.cc:654] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.849793 11533 tablet_bootstrap.cc:654] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.856146 11531 tablet_bootstrap.cc:492] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.856724 11531 ts_tablet_manager.cc:1403] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.009s
I20260709 12:22:30.862601 11531 raft_consensus.cc:359] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.863358 11531 raft_consensus.cc:385] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.863692 11531 raft_consensus.cc:740] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.864494 11531 consensus_queue.cc:260] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.865397 11539 raft_consensus.cc:515] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.867123 11531 ts_tablet_manager.cc:1434] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:22:30.868283 11531 tablet_bootstrap.cc:492] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.868415 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:30.869047 11417 raft_consensus.cc:3060] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:30.874830 11531 tablet_bootstrap.cc:654] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.877049 11417 raft_consensus.cc:2468] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:30.879076 11307 leader_election.cc:304] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:30.880246 11564 raft_consensus.cc:2804] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:30.880858 11564 raft_consensus.cc:697] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:30.881934 11564 consensus_queue.cc:237] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.887887 11531 tablet_bootstrap.cc:492] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.888660 11531 ts_tablet_manager.cc:1403] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:22:30.892820 11531 raft_consensus.cc:359] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.893829 11531 raft_consensus.cc:385] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.894258 11531 raft_consensus.cc:740] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.895361 11531 consensus_queue.cc:260] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.899137 11531 ts_tablet_manager.cc:1434] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 12:22:30.900640 11531 tablet_bootstrap.cc:492] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.903625 11564 raft_consensus.cc:493] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.904304 11564 raft_consensus.cc:515] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.904414 11237 catalog_manager.cc:5697] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.906113 11539 leader_election.cc:290] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.907361 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:30.908278 11531 tablet_bootstrap.cc:654] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.908746 11533 tablet_bootstrap.cc:492] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
W20260709 12:22:30.909034 11308 leader_election.cc:343] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.910645 11533 ts_tablet_manager.cc:1403] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.068s	user 0.039s	sys 0.020s
I20260709 12:22:30.912778 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:30.913686 11533 raft_consensus.cc:359] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
W20260709 12:22:30.914301 11308 leader_election.cc:343] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.914669 11533 raft_consensus.cc:385] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.915017 11564 leader_election.cc:290] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:30.915664 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:30.915060 11533 raft_consensus.cc:740] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
W20260709 12:22:30.917292 11307 leader_election.cc:343] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:30.917793 11307 leader_election.cc:304] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:30.917954 11533 consensus_queue.cc:260] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.920910 11533 ts_tablet_manager.cc:1434] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:30.922132 11533 tablet_bootstrap.cc:492] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:30.923028 11564 raft_consensus.cc:2749] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:30.927623 11531 tablet_bootstrap.cc:492] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.928177 11531 ts_tablet_manager.cc:1403] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.011s
I20260709 12:22:30.928539 11533 tablet_bootstrap.cc:654] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.930882 11531 raft_consensus.cc:359] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.931829 11531 raft_consensus.cc:385] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.932157 11531 raft_consensus.cc:740] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.932814 11531 consensus_queue.cc:260] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.933676 11533 tablet_bootstrap.cc:492] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:30.934149 11533 ts_tablet_manager.cc:1403] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:22:30.934438 11531 ts_tablet_manager.cc:1434] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:30.935431 11531 tablet_bootstrap.cc:492] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:30.936722 11533 raft_consensus.cc:359] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.937417 11533 raft_consensus.cc:385] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.937683 11533 raft_consensus.cc:740] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.938356 11533 consensus_queue.cc:260] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.940227 11533 ts_tablet_manager.cc:1434] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:30.940536 11531 tablet_bootstrap.cc:654] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:30.944958 11531 tablet_bootstrap.cc:492] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:30.945355 11531 ts_tablet_manager.cc:1403] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:22:30.947526 11531 raft_consensus.cc:359] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.948167 11531 raft_consensus.cc:385] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:30.948428 11531 raft_consensus.cc:740] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:30.949083 11531 consensus_queue.cc:260] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:30.951046 11531 ts_tablet_manager.cc:1434] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:22:31.051076 11538 raft_consensus.cc:493] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.051671 11538 raft_consensus.cc:515] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.053392 11538 leader_election.cc:290] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:31.054283 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:31.054448 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:31.054955 11342 raft_consensus.cc:2468] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 0.
I20260709 12:22:31.055099 11417 raft_consensus.cc:2468] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 0.
I20260709 12:22:31.055943 11455 leader_election.cc:304] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:31.056703 11538 raft_consensus.cc:2804] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:31.057014 11538 raft_consensus.cc:493] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.057277 11538 raft_consensus.cc:3060] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.062578 11538 raft_consensus.cc:515] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.064333 11538 leader_election.cc:290] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:31.065186 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:31.065310 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:31.065793 11342 raft_consensus.cc:3060] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.065924 11417 raft_consensus.cc:3060] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.071161 11417 raft_consensus.cc:2468] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 1.
I20260709 12:22:31.071161 11342 raft_consensus.cc:2468] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 1.
I20260709 12:22:31.072374 11456 leader_election.cc:304] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.073088 11538 raft_consensus.cc:2804] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:31.074115 11538 raft_consensus.cc:697] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:31.074970 11538 consensus_queue.cc:237] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.081997 11240 catalog_manager.cc:5697] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:31.098347 11564 raft_consensus.cc:493] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.098934 11564 raft_consensus.cc:515] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:31.100690 11564 leader_election.cc:290] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.101643 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:31.101836 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:31.102787 11490 raft_consensus.cc:2393] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 408b52ec62254d8e99818a554befd6e3 in current term 1: Already voted for candidate 7be56f2c081b41b6939235ccfd24abb1 in this term.
I20260709 12:22:31.103860 11308 leader_election.cc:304] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:31.104588 11564 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.109939 11564 raft_consensus.cc:2749] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:31.127763 11537 raft_consensus.cc:493] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.128419 11537 raft_consensus.cc:515] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.129251 11564 consensus_queue.cc:1048] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.130199 11537 leader_election.cc:290] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.131072 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:31.131235 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:31.131799 11342 raft_consensus.cc:2468] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:31.131870 11490 raft_consensus.cc:2468] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:31.133101 11383 leader_election.cc:304] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.133999 11537 raft_consensus.cc:2804] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:31.134479 11537 raft_consensus.cc:493] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.134819 11537 raft_consensus.cc:3060] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.141743 11417 raft_consensus.cc:3060] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.144440 11537 raft_consensus.cc:515] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.148072 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:31.148743 11342 raft_consensus.cc:3060] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.147270 11537 leader_election.cc:290] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.151340 11489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:31.152056 11489 raft_consensus.cc:3060] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.153924 11555 raft_consensus.cc:493] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.154577 11564 consensus_queue.cc:1048] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:31.154563 11555 raft_consensus.cc:515] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.156934 11555 leader_election.cc:290] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.157822 11342 raft_consensus.cc:2468] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:31.158365 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af2adb5e0745088a20cdf76ead9d5c" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:31.159083 11416 raft_consensus.cc:2468] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:31.159917 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af2adb5e0745088a20cdf76ead9d5c" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:31.160336 11381 leader_election.cc:304] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.160588 11489 raft_consensus.cc:2468] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:31.160697 11490 raft_consensus.cc:2468] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:31.161679 11307 leader_election.cc:304] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.162563 11537 raft_consensus.cc:2804] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:31.163637 11537 raft_consensus.cc:697] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:31.163066 11555 raft_consensus.cc:2804] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:31.163749 11564 consensus_queue.cc:1048] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.164208 11555 raft_consensus.cc:493] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.164656 11555 raft_consensus.cc:3060] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.172439 11555 raft_consensus.cc:515] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.174388 11561 raft_consensus.cc:493] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.174968 11561 raft_consensus.cc:515] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.174922 11537 consensus_queue.cc:237] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.176959 11561 leader_election.cc:290] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.178247 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:31.178687 11555 leader_election.cc:290] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.179121 11490 raft_consensus.cc:2468] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:31.180249 11416 raft_consensus.cc:3060] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.180572 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af2adb5e0745088a20cdf76ead9d5c" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:31.180846 11383 leader_election.cc:304] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.181268 11415 raft_consensus.cc:3060] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.182140 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af2adb5e0745088a20cdf76ead9d5c" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:31.182794 11491 raft_consensus.cc:3060] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.184160 11561 raft_consensus.cc:2804] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:31.184588 11561 raft_consensus.cc:493] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.184904 11561 raft_consensus.cc:3060] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.190454 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:31.190866 11564 consensus_queue.cc:1048] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:31.191284 11342 raft_consensus.cc:2468] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:31.192263 11538 raft_consensus.cc:493] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.192751 11561 raft_consensus.cc:515] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.192778 11538 raft_consensus.cc:515] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.194902 11491 raft_consensus.cc:2468] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:31.194941 11415 raft_consensus.cc:2468] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:31.196017 11538 leader_election.cc:290] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:31.197623 11564 consensus_queue.cc:1048] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.197883 11308 leader_election.cc:304] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:31.199013 11539 raft_consensus.cc:2804] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:31.199509 11539 raft_consensus.cc:697] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:31.199951 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:31.200191 11341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:31.200866 11341 raft_consensus.cc:3060] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.200425 11539 consensus_queue.cc:237] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.201452 11561 leader_election.cc:290] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.203279 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:31.204324 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:31.205156 11415 raft_consensus.cc:2393] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4abc425940dd4061af4d588e0cf78ba2 in current term 1: Already voted for candidate 408b52ec62254d8e99818a554befd6e3 in this term.
I20260709 12:22:31.203941 11491 raft_consensus.cc:3060] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.206171 11456 leader_election.cc:304] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:31.207962 11538 raft_consensus.cc:3060] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:31.210929 11341 raft_consensus.cc:2468] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:31.213238 11238 catalog_manager.cc:5697] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.214967 11381 leader_election.cc:304] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.215806 11561 raft_consensus.cc:2804] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:31.216284 11561 raft_consensus.cc:697] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:31.217254 11561 consensus_queue.cc:237] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.240350 11237 catalog_manager.cc:5697] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.231124 11491 raft_consensus.cc:2468] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:31.242903 11538 raft_consensus.cc:2749] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:31.251770 11240 catalog_manager.cc:5697] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.254923 11564 consensus_queue.cc:1048] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.268056 11561 consensus_queue.cc:1048] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.282706 11561 consensus_queue.cc:1048] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.308835 11539 consensus_queue.cc:1048] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.314419 11416 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97de4708d35e47afb683b92846175828"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
mode: GRACEFUL
new_leader_uuid: "408b52ec62254d8e99818a554befd6e3"
 from {username='slave'} at 127.0.0.1:41196
I20260709 12:22:31.315001 11416 raft_consensus.cc:606] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Received request to transfer leadership to TS 408b52ec62254d8e99818a554befd6e3
I20260709 12:22:31.323359 11555 consensus_queue.cc:1048] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.343216 11341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:22:31.343765 11341 raft_consensus.cc:606] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Received request to transfer leadership to TS 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:31.354446 11341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:22:31.354998 11341 raft_consensus.cc:606] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Received request to transfer leadership to TS 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:31.460431 11538 consensus_queue.cc:1048] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.471186 11582 consensus_queue.cc:1048] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.631106 11555 consensus_queue.cc:1048] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.641222 11555 consensus_queue.cc:1048] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.665413 11561 consensus_queue.cc:1048] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.679821 11561 consensus_queue.cc:1048] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.683826 11561 consensus_queue.cc:1048] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.698180 11598 consensus_queue.cc:1048] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:31.749356 11553 raft_consensus.cc:993] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:31.749873 11539 raft_consensus.cc:1081] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Signalling peer 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:31.751462 11416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:55686
I20260709 12:22:31.752071 11416 raft_consensus.cc:493] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.752480 11416 raft_consensus.cc:3060] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.757637 11416 raft_consensus.cc:515] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.759778 11416 leader_election.cc:290] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.761158 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:31.761863 11490 raft_consensus.cc:3060] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.762140 11341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:31.762801 11341 raft_consensus.cc:3055] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:31.763116 11341 raft_consensus.cc:740] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:31.763908 11341 consensus_queue.cc:260] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.765661 11341 raft_consensus.cc:3060] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.769160 11490 raft_consensus.cc:2468] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:31.770485 11383 leader_election.cc:304] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.771279 11598 raft_consensus.cc:2804] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.771687 11598 raft_consensus.cc:697] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:31.772328 11341 raft_consensus.cc:2468] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:31.772449 11598 consensus_queue.cc:237] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.778775 11239 catalog_manager.cc:5697] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 1 to 2, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.818526 11553 raft_consensus.cc:993] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:31.819028 11539 raft_consensus.cc:1081] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Signalling peer 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:31.820443 11416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:55686
I20260709 12:22:31.821027 11416 raft_consensus.cc:493] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.821389 11416 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.826431 11416 raft_consensus.cc:515] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.828238 11416 leader_election.cc:290] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.829002 11341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:31.829222 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:31.829672 11341 raft_consensus.cc:3055] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:31.829744 11490 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.830075 11341 raft_consensus.cc:740] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:31.830924 11341 consensus_queue.cc:260] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.832160 11341 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.836735 11537 raft_consensus.cc:993] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: : Instructing follower 408b52ec62254d8e99818a554befd6e3 to start an election
I20260709 12:22:31.836879 11490 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:31.837226 11598 raft_consensus.cc:1081] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Signalling peer 408b52ec62254d8e99818a554befd6e3 to start an election
I20260709 12:22:31.838073 11383 leader_election.cc:304] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:31.838881 11598 raft_consensus.cc:2804] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.838860 11342 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97de4708d35e47afb683b92846175828"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
 from {username='slave'} at 127.0.0.1:39772
I20260709 12:22:31.839047 11341 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:31.839654 11598 raft_consensus.cc:697] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:31.839748 11342 raft_consensus.cc:493] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.840155 11342 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.840546 11598 consensus_queue.cc:237] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:31.847672 11342 raft_consensus.cc:515] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:31.849711 11342 leader_election.cc:290] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:31.850255 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:31.850782 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:31.850977 11416 raft_consensus.cc:3055] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:31.850636 11239 catalog_manager.cc:5697] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 1 to 2, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.851398 11490 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.851395 11416 raft_consensus.cc:740] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:31.852451 11416 consensus_queue.cc:260] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:31.853387 11416 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.857863 11490 raft_consensus.cc:2468] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 2.
I20260709 12:22:31.858911 11308 leader_election.cc:304] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:31.859194 11416 raft_consensus.cc:2468] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 2.
I20260709 12:22:31.859611 11539 raft_consensus.cc:2804] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.860111 11539 raft_consensus.cc:697] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:31.860932 11539 consensus_queue.cc:237] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:31.867949 11239 catalog_manager.cc:5697] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 1 to 2, leader changed from 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2) to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:32.248505 11342 raft_consensus.cc:1275] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.249974 11561 consensus_queue.cc:1048] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:32.262346 11490 raft_consensus.cc:1275] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.264343 11598 consensus_queue.cc:1048] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:32.329994 11490 raft_consensus.cc:1275] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 408b52ec62254d8e99818a554befd6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.331380 11539 consensus_queue.cc:1048] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.341600 11416 raft_consensus.cc:1275] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Refusing update from remote peer 408b52ec62254d8e99818a554befd6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.343577 11595 consensus_queue.cc:1048] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.360049 11490 raft_consensus.cc:1275] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.361385 11598 consensus_queue.cc:1048] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.374539 11342 raft_consensus.cc:1275] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.376714 11598 consensus_queue.cc:1048] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.395105 11238 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46414:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:34.398176 11238 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:34.454952 11332 tablet_service.cc:1511] Processing CreateTablet for tablet 7dffc97ecfdd4a8db556f6c1f51cefb9 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:34.456959 11332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dffc97ecfdd4a8db556f6c1f51cefb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.456732 11406 tablet_service.cc:1511] Processing CreateTablet for tablet 7dffc97ecfdd4a8db556f6c1f51cefb9 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:34.458460 11406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dffc97ecfdd4a8db556f6c1f51cefb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.460345 11324 tablet_service.cc:1511] Processing CreateTablet for tablet 5edce2f4af3249e3b06474994cd321d0 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:34.461889 11324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5edce2f4af3249e3b06474994cd321d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.462042 11475 tablet_service.cc:1511] Processing CreateTablet for tablet 7dffc97ecfdd4a8db556f6c1f51cefb9 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:34.463778 11475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dffc97ecfdd4a8db556f6c1f51cefb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.464203 11401 tablet_service.cc:1511] Processing CreateTablet for tablet 5edce2f4af3249e3b06474994cd321d0 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:34.465613 11401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5edce2f4af3249e3b06474994cd321d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.471155 11478 tablet_service.cc:1511] Processing CreateTablet for tablet 5edce2f4af3249e3b06474994cd321d0 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:34.472635 11478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5edce2f4af3249e3b06474994cd321d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.474211 11400 tablet_service.cc:1511] Processing CreateTablet for tablet d0d7525a305245759b35f92da7a5dfc7 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:34.475757 11400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d7525a305245759b35f92da7a5dfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.475641 11476 tablet_service.cc:1511] Processing CreateTablet for tablet d0d7525a305245759b35f92da7a5dfc7 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:34.477108 11476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d7525a305245759b35f92da7a5dfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.480468 11473 tablet_service.cc:1511] Processing CreateTablet for tablet dbf243500ca743a3b8600b11b8599177 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:34.482221 11473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf243500ca743a3b8600b11b8599177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.483495 11481 tablet_service.cc:1511] Processing CreateTablet for tablet 0279a578b9e047f0b044e5ca890208f7 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:34.483561 11404 tablet_service.cc:1511] Processing CreateTablet for tablet dbf243500ca743a3b8600b11b8599177 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:34.484457 11405 tablet_service.cc:1511] Processing CreateTablet for tablet df40005c1ebe414fb797a878bae5eaf9 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:34.485185 11481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0279a578b9e047f0b044e5ca890208f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.485242 11404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf243500ca743a3b8600b11b8599177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.486219 11405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df40005c1ebe414fb797a878bae5eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.487674 11402 tablet_service.cc:1511] Processing CreateTablet for tablet 0279a578b9e047f0b044e5ca890208f7 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:34.489077 11402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0279a578b9e047f0b044e5ca890208f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.488672 11330 tablet_service.cc:1511] Processing CreateTablet for tablet d0d7525a305245759b35f92da7a5dfc7 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:34.490190 11330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d7525a305245759b35f92da7a5dfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.498163 11474 tablet_service.cc:1511] Processing CreateTablet for tablet df40005c1ebe414fb797a878bae5eaf9 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:34.498952 11329 tablet_service.cc:1511] Processing CreateTablet for tablet dbf243500ca743a3b8600b11b8599177 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:34.499673 11474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df40005c1ebe414fb797a878bae5eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.500281 11329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf243500ca743a3b8600b11b8599177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.505045 11399 tablet_service.cc:1511] Processing CreateTablet for tablet 96bdbde5464e4ec9b2ffc980a92280ae (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:34.506742 11399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bdbde5464e4ec9b2ffc980a92280ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.508790 11328 tablet_service.cc:1511] Processing CreateTablet for tablet 0279a578b9e047f0b044e5ca890208f7 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:34.510640 11328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0279a578b9e047f0b044e5ca890208f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.521782 11331 tablet_service.cc:1511] Processing CreateTablet for tablet df40005c1ebe414fb797a878bae5eaf9 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:34.523402 11331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df40005c1ebe414fb797a878bae5eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.528393 11480 tablet_service.cc:1511] Processing CreateTablet for tablet 96bdbde5464e4ec9b2ffc980a92280ae (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:34.531322 11403 tablet_service.cc:1511] Processing CreateTablet for tablet a7b992a1a0614882a13890e523e3e1a3 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:34.532872 11403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b992a1a0614882a13890e523e3e1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.533636 11479 tablet_service.cc:1511] Processing CreateTablet for tablet a7b992a1a0614882a13890e523e3e1a3 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:34.534765 11480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bdbde5464e4ec9b2ffc980a92280ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.535902 11479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b992a1a0614882a13890e523e3e1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.538057 11326 tablet_service.cc:1511] Processing CreateTablet for tablet 96bdbde5464e4ec9b2ffc980a92280ae (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:34.539633 11326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bdbde5464e4ec9b2ffc980a92280ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.552577 11325 tablet_service.cc:1511] Processing CreateTablet for tablet a7b992a1a0614882a13890e523e3e1a3 (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:34.554059 11327 tablet_service.cc:1511] Processing CreateTablet for tablet a8afccb6e54a4a17b883a901da0bef6b (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:34.554108 11325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b992a1a0614882a13890e523e3e1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.558668 11327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8afccb6e54a4a17b883a901da0bef6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.565516 11402 tablet_service.cc:1511] Processing CreateTablet for tablet a8afccb6e54a4a17b883a901da0bef6b (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:34.565959 11477 tablet_service.cc:1511] Processing CreateTablet for tablet a8afccb6e54a4a17b883a901da0bef6b (DEFAULT_TABLE table=soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:34.568037 11477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8afccb6e54a4a17b883a901da0bef6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.591221 11402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8afccb6e54a4a17b883a901da0bef6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:34.619470 11638 tablet_bootstrap.cc:492] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:34.623371 11639 tablet_bootstrap.cc:492] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.624173 11640 tablet_bootstrap.cc:492] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.637697 11640 tablet_bootstrap.cc:654] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.651033 11640 tablet_bootstrap.cc:492] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:34.651692 11640 ts_tablet_manager.cc:1403] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.014s
I20260709 12:22:34.658210 11640 raft_consensus.cc:359] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.659113 11640 raft_consensus.cc:385] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.659502 11640 raft_consensus.cc:740] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.659631 11639 tablet_bootstrap.cc:654] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.660002 11638 tablet_bootstrap.cc:654] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.660530 11640 consensus_queue.cc:260] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.663942 11640 ts_tablet_manager.cc:1434] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:22:34.666606 11640 tablet_bootstrap.cc:492] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.675177 11638 tablet_bootstrap.cc:492] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:34.675838 11638 ts_tablet_manager.cc:1403] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.004s
I20260709 12:22:34.676209 11640 tablet_bootstrap.cc:654] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.679137 11638 raft_consensus.cc:359] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.680007 11638 raft_consensus.cc:385] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.680373 11638 raft_consensus.cc:740] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.681362 11638 consensus_queue.cc:260] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.681808 11639 tablet_bootstrap.cc:492] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:34.683557 11639 ts_tablet_manager.cc:1403] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.013s
I20260709 12:22:34.684563 11638 ts_tablet_manager.cc:1434] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:34.686138 11638 tablet_bootstrap.cc:492] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:34.686827 11639 raft_consensus.cc:359] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.688863 11640 tablet_bootstrap.cc:492] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:34.689150 11639 raft_consensus.cc:385] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.689481 11640 ts_tablet_manager.cc:1403] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:22:34.689570 11639 raft_consensus.cc:740] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.690732 11639 consensus_queue.cc:260] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.693074 11640 raft_consensus.cc:359] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.693887 11640 raft_consensus.cc:385] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.694202 11640 raft_consensus.cc:740] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.694829 11638 tablet_bootstrap.cc:654] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.695096 11640 consensus_queue.cc:260] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.696949 11639 ts_tablet_manager.cc:1434] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:22:34.698004 11640 ts_tablet_manager.cc:1434] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:34.698173 11639 tablet_bootstrap.cc:492] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.699146 11640 tablet_bootstrap.cc:492] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.706012 11639 tablet_bootstrap.cc:654] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.706012 11640 tablet_bootstrap.cc:654] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.727429 11638 tablet_bootstrap.cc:492] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:34.728034 11638 ts_tablet_manager.cc:1403] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.021s
I20260709 12:22:34.729403 11639 tablet_bootstrap.cc:492] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:34.729760 11640 tablet_bootstrap.cc:492] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:34.729918 11639 ts_tablet_manager.cc:1403] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.006s
I20260709 12:22:34.730404 11640 ts_tablet_manager.cc:1403] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.004s
I20260709 12:22:34.731236 11638 raft_consensus.cc:359] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.732038 11638 raft_consensus.cc:385] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.732329 11638 raft_consensus.cc:740] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.732537 11639 raft_consensus.cc:359] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.733295 11639 raft_consensus.cc:385] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.733160 11638 consensus_queue.cc:260] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.733641 11639 raft_consensus.cc:740] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.733522 11640 raft_consensus.cc:359] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.734455 11640 raft_consensus.cc:385] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.734815 11640 raft_consensus.cc:740] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.734596 11639 consensus_queue.cc:260] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.736129 11638 ts_tablet_manager.cc:1434] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:34.735672 11640 consensus_queue.cc:260] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.737222 11638 tablet_bootstrap.cc:492] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:34.737574 11639 ts_tablet_manager.cc:1434] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:22:34.738649 11639 tablet_bootstrap.cc:492] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.738704 11640 ts_tablet_manager.cc:1434] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:22:34.739903 11640 tablet_bootstrap.cc:492] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.745004 11638 tablet_bootstrap.cc:654] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.746647 11640 tablet_bootstrap.cc:654] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:34.746831 11288 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:34.747380 11639 tablet_bootstrap.cc:654] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.749662 11538 raft_consensus.cc:493] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.750264 11538 raft_consensus.cc:515] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.754501 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf243500ca743a3b8600b11b8599177" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
W20260709 12:22:34.756707 11455 leader_election.cc:343] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.759284 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf243500ca743a3b8600b11b8599177" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:34.759963 11639 tablet_bootstrap.cc:492] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:34.760519 11639 ts_tablet_manager.cc:1403] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.009s
I20260709 12:22:34.760908 11640 tablet_bootstrap.cc:492] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
W20260709 12:22:34.761000 11456 leader_election.cc:343] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.761541 11640 ts_tablet_manager.cc:1403] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.004s
I20260709 12:22:34.761593 11456 leader_election.cc:304] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:34.764062 11538 leader_election.cc:290] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:34.764714 11538 raft_consensus.cc:2749] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:34.764782 11639 raft_consensus.cc:359] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.765650 11639 raft_consensus.cc:385] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.766036 11639 raft_consensus.cc:740] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.766513 11640 raft_consensus.cc:359] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.767251 11639 consensus_queue.cc:260] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.769860 11639 ts_tablet_manager.cc:1434] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:34.767323 11640 raft_consensus.cc:385] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.770399 11640 raft_consensus.cc:740] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.771247 11640 consensus_queue.cc:260] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.773836 11640 ts_tablet_manager.cc:1434] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:22:34.774921 11639 tablet_bootstrap.cc:492] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.776059 11638 tablet_bootstrap.cc:492] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:34.776588 11638 ts_tablet_manager.cc:1403] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.040s	user 0.005s	sys 0.012s
I20260709 12:22:34.779808 11638 raft_consensus.cc:359] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.780560 11638 raft_consensus.cc:385] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.780887 11638 raft_consensus.cc:740] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.781716 11638 consensus_queue.cc:260] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.782516 11639 tablet_bootstrap.cc:654] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.783542 11640 tablet_bootstrap.cc:492] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.790104 11640 tablet_bootstrap.cc:654] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.792605 11638 ts_tablet_manager.cc:1434] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:22:34.793438 11537 raft_consensus.cc:493] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.793999 11537 raft_consensus.cc:515] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.794727 11639 tablet_bootstrap.cc:492] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:34.795264 11639 ts_tablet_manager.cc:1403] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.014s
I20260709 12:22:34.795537 11640 tablet_bootstrap.cc:492] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:34.796075 11640 ts_tablet_manager.cc:1403] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 12:22:34.798530 11638 tablet_bootstrap.cc:492] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:34.798480 11639 raft_consensus.cc:359] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.799397 11639 raft_consensus.cc:385] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.799283 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:34.799640 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:34.799882 11639 raft_consensus.cc:740] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.800110 11490 raft_consensus.cc:2468] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:34.800546 11537 leader_election.cc:290] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.800745 11639 consensus_queue.cc:260] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.799234 11640 raft_consensus.cc:359] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.802057 11640 raft_consensus.cc:385] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.802551 11640 raft_consensus.cc:740] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.802459 11383 leader_election.cc:304] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:34.803709 11640 consensus_queue.cc:260] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.805015 11342 raft_consensus.cc:2468] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:34.814186 11658 raft_consensus.cc:2804] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:34.814824 11658 raft_consensus.cc:493] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.815223 11658 raft_consensus.cc:3060] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.817541 11638 tablet_bootstrap.cc:654] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.819001 11640 ts_tablet_manager.cc:1434] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:22:34.819932 11653 raft_consensus.cc:493] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.820428 11653 raft_consensus.cc:515] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.823289 11640 tablet_bootstrap.cc:492] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.823657 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
W20260709 12:22:34.825752 11455 leader_election.cc:343] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.826810 11653 leader_election.cc:290] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:34.836364 11638 tablet_bootstrap.cc:492] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:34.836992 11638 ts_tablet_manager.cc:1403] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.006s
I20260709 12:22:34.837904 11640 tablet_bootstrap.cc:654] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.840934 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
W20260709 12:22:34.842397 11456 leader_election.cc:343] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.842892 11456 leader_election.cc:304] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:34.840795 11658 raft_consensus.cc:515] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.841111 11638 raft_consensus.cc:359] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.843966 11638 raft_consensus.cc:385] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.844079 11653 raft_consensus.cc:2749] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:34.845734 11641 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.845808 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:34.846621 11342 raft_consensus.cc:3060] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.846436 11641 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.847162 11639 ts_tablet_manager.cc:1434] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.052s	user 0.015s	sys 0.033s
I20260709 12:22:34.848701 11641 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.849174 11639 tablet_bootstrap.cc:492] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.849515 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:34.850013 11658 leader_election.cc:290] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.850121 11415 raft_consensus.cc:2468] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:34.851706 11307 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:34.851750 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:34.852685 11491 raft_consensus.cc:3060] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.857461 11595 raft_consensus.cc:2804] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:34.857901 11595 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.858223 11595 raft_consensus.cc:3060] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.858377 11342 raft_consensus.cc:2468] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:34.850235 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:34.859496 11638 raft_consensus.cc:740] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.860492 11639 tablet_bootstrap.cc:654] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:34.860759 11308 leader_election.cc:343] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.860482 11638 consensus_queue.cc:260] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.862025 11381 leader_election.cc:304] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:34.863178 11658 raft_consensus.cc:2804] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:34.864040 11658 raft_consensus.cc:697] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:34.865250 11658 consensus_queue.cc:237] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.867749 11595 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.858726 11641 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.868990 11641 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.870941 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:34.871562 11415 raft_consensus.cc:3060] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.871924 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:34.872596 11416 raft_consensus.cc:2468] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:34.873893 11307 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:34.874897 11642 raft_consensus.cc:2804] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:34.874994 11638 ts_tablet_manager.cc:1434] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 12:22:34.875298 11642 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.875696 11642 raft_consensus.cc:3060] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.881497 11415 raft_consensus.cc:2468] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:34.882973 11307 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:34.886149 11638 tablet_bootstrap.cc:492] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:34.888898 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:34.889880 11640 tablet_bootstrap.cc:492] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
W20260709 12:22:34.890563 11308 leader_election.cc:343] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.891393 11639 tablet_bootstrap.cc:492] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:34.892700 11640 ts_tablet_manager.cc:1403] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.007s
I20260709 12:22:34.892927 11639 ts_tablet_manager.cc:1403] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.015s
I20260709 12:22:34.893687 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:34.894127 11663 raft_consensus.cc:2804] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:22:34.895700 11308 leader_election.cc:343] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.891551 11595 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.897847 11639 raft_consensus.cc:359] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.898670 11639 raft_consensus.cc:385] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.898990 11639 raft_consensus.cc:740] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.899717 11639 consensus_queue.cc:260] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.900924 11640 raft_consensus.cc:359] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.901702 11640 raft_consensus.cc:385] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.902010 11640 raft_consensus.cc:740] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.902189 11639 ts_tablet_manager.cc:1434] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:34.903424 11639 tablet_bootstrap.cc:492] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.903038 11640 consensus_queue.cc:260] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.903285 11642 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.905570 11640 ts_tablet_manager.cc:1434] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:22:34.907958 11640 tablet_bootstrap.cc:492] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.898456 11641 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.909803 11491 raft_consensus.cc:2468] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:34.911706 11642 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.911150 11639 tablet_bootstrap.cc:654] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.912976 11638 tablet_bootstrap.cc:654] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.914731 11640 tablet_bootstrap.cc:654] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.919335 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:34.920166 11415 raft_consensus.cc:3060] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.913304 11239 catalog_manager.cc:5697] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.926506 11640 tablet_bootstrap.cc:492] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:34.927110 11640 ts_tablet_manager.cc:1403] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.013s
I20260709 12:22:34.931614 11640 raft_consensus.cc:359] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.932379 11640 raft_consensus.cc:385] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.936064 11640 raft_consensus.cc:740] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.936864 11640 consensus_queue.cc:260] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.946142 11640 ts_tablet_manager.cc:1434] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:22:34.946632 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:34.947255 11640 tablet_bootstrap.cc:492] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.952306 11663 raft_consensus.cc:697] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:34.953392 11663 consensus_queue.cc:237] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
W20260709 12:22:34.954491 11308 leader_election.cc:343] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.954471 11640 tablet_bootstrap.cc:654] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.955986 11415 raft_consensus.cc:2468] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:34.959281 11307 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:34.960899 11642 raft_consensus.cc:2804] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:34.961532 11642 raft_consensus.cc:697] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:34.963084 11642 consensus_queue.cc:237] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.967718 11595 raft_consensus.cc:493] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.968253 11595 raft_consensus.cc:515] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.970022 11640 tablet_bootstrap.cc:492] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:34.970556 11640 ts_tablet_manager.cc:1403] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:22:34.970603 11639 tablet_bootstrap.cc:492] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:34.970846 11595 leader_election.cc:290] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:34.971225 11639 ts_tablet_manager.cc:1403] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.068s	user 0.027s	sys 0.023s
I20260709 12:22:34.971982 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:34.973152 11491 raft_consensus.cc:2468] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:34.973269 11640 raft_consensus.cc:359] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.973992 11640 raft_consensus.cc:385] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.974315 11640 raft_consensus.cc:740] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.974511 11308 leader_election.cc:304] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:34.972236 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:34.975154 11640 consensus_queue.cc:260] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.976042 11595 raft_consensus.cc:2804] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:34.976440 11307 leader_election.cc:343] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:34.977380 11640 ts_tablet_manager.cc:1434] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:34.976437 11595 raft_consensus.cc:493] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.978037 11595 raft_consensus.cc:3060] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.977787 11639 raft_consensus.cc:359] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.978492 11638 tablet_bootstrap.cc:492] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:34.978591 11640 tablet_bootstrap.cc:492] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: Bootstrap starting.
I20260709 12:22:34.978693 11639 raft_consensus.cc:385] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.979317 11638 ts_tablet_manager.cc:1403] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.094s	user 0.040s	sys 0.015s
I20260709 12:22:34.979377 11639 raft_consensus.cc:740] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.980403 11639 consensus_queue.cc:260] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.982676 11638 raft_consensus.cc:359] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.983230 11639 ts_tablet_manager.cc:1434] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:22:34.985433 11638 raft_consensus.cc:385] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:34.986075 11638 raft_consensus.cc:740] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:34.986251 11639 tablet_bootstrap.cc:492] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:34.985178 11640 tablet_bootstrap.cc:654] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.987192 11638 consensus_queue.cc:260] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.990204 11638 ts_tablet_manager.cc:1434] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:34.991360 11638 tablet_bootstrap.cc:492] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:34.994946 11595 raft_consensus.cc:515] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:34.997975 11639 tablet_bootstrap.cc:654] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:34.997975 11638 tablet_bootstrap.cc:654] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.007649 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.008385 11491 raft_consensus.cc:3060] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.010229 11240 catalog_manager.cc:5697] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.011088 11638 tablet_bootstrap.cc:492] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:35.011667 11638 ts_tablet_manager.cc:1403] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 12:22:35.012528 11639 tablet_bootstrap.cc:492] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:35.011960 11240 catalog_manager.cc:5697] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.007707 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.013156 11639 ts_tablet_manager.cc:1403] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.007s
I20260709 12:22:35.013470 11415 raft_consensus.cc:3060] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.015033 11638 raft_consensus.cc:359] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.015872 11638 raft_consensus.cc:385] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.016211 11638 raft_consensus.cc:740] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.019254 11638 consensus_queue.cc:260] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.020553 11639 raft_consensus.cc:359] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.021347 11639 raft_consensus.cc:385] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.021687 11639 raft_consensus.cc:740] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.022521 11639 consensus_queue.cc:260] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.023692 11638 ts_tablet_manager.cc:1434] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:22:35.024865 11638 tablet_bootstrap.cc:492] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:35.025504 11639 ts_tablet_manager.cc:1434] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:22:35.026656 11639 tablet_bootstrap.cc:492] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:35.027108 11491 raft_consensus.cc:2468] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:35.029484 11663 raft_consensus.cc:493] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.032408 11663 raft_consensus.cc:515] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.033006 11308 leader_election.cc:304] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:35.033286 11639 tablet_bootstrap.cc:654] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.033916 11642 raft_consensus.cc:2804] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.034427 11642 raft_consensus.cc:697] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.035277 11642 consensus_queue.cc:237] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.038190 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:35.038728 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
W20260709 12:22:35.040539 11308 leader_election.cc:343] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:35.041029 11307 leader_election.cc:343] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:35.041639 11307 leader_election.cc:304] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:35.035260 11663 leader_election.cc:290] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.044553 11664 raft_consensus.cc:2749] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:35.030416 11595 leader_election.cc:290] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.031723 11415 raft_consensus.cc:2468] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:35.046078 11638 tablet_bootstrap.cc:654] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.050020 11640 tablet_bootstrap.cc:492] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: No bootstrap required, opened a new log
I20260709 12:22:35.050534 11640 ts_tablet_manager.cc:1403] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: Time spent bootstrapping tablet: real 0.072s	user 0.005s	sys 0.009s
I20260709 12:22:35.059283 11639 tablet_bootstrap.cc:492] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:35.059923 11639 ts_tablet_manager.cc:1403] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.020s
I20260709 12:22:35.060653 11638 tablet_bootstrap.cc:492] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:35.061224 11638 ts_tablet_manager.cc:1403] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.002s
I20260709 12:22:35.062716 11657 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.062793 11639 raft_consensus.cc:359] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.063619 11639 raft_consensus.cc:385] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.063377 11657 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.064003 11639 raft_consensus.cc:740] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.063422 11240 catalog_manager.cc:5697] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.064704 11639 consensus_queue.cc:260] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.059005 11640 raft_consensus.cc:359] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.065552 11657 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.065657 11640 raft_consensus.cc:385] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.066071 11640 raft_consensus.cc:740] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.066663 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:35.067342 11342 raft_consensus.cc:2468] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:35.067943 11638 raft_consensus.cc:359] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.066857 11640 consensus_queue.cc:260] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.068782 11638 raft_consensus.cc:385] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.069168 11638 raft_consensus.cc:740] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.069069 11381 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.069638 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:35.070073 11638 consensus_queue.cc:260] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.071035 11491 raft_consensus.cc:2468] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:35.071365 11656 raft_consensus.cc:2804] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:35.074606 11656 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.074982 11656 raft_consensus.cc:3060] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.072510 11639 ts_tablet_manager.cc:1434] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:22:35.070641 11640 ts_tablet_manager.cc:1434] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:22:35.077294 11639 tablet_bootstrap.cc:492] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: Bootstrap starting.
I20260709 12:22:35.080508 11657 raft_consensus.cc:493] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.081112 11657 raft_consensus.cc:515] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.082590 11639 tablet_bootstrap.cc:654] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.083250 11657 leader_election.cc:290] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.083456 11656 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.084741 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:35.073129 11638 ts_tablet_manager.cc:1434] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:22:35.085412 11491 raft_consensus.cc:2468] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:35.086459 11638 tablet_bootstrap.cc:492] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:35.086589 11383 leader_election.cc:304] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.086645 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:35.088099 11342 raft_consensus.cc:2468] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 0.
I20260709 12:22:35.093181 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.093614 11638 tablet_bootstrap.cc:654] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.093824 11491 raft_consensus.cc:3060] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.098426 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:35.099236 11342 raft_consensus.cc:3060] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.102674 11657 raft_consensus.cc:2804] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:35.103322 11657 raft_consensus.cc:493] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.103534 11656 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.103832 11657 raft_consensus.cc:3060] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.105536 11491 raft_consensus.cc:2468] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:35.106851 11383 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.107224 11639 tablet_bootstrap.cc:492] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: No bootstrap required, opened a new log
I20260709 12:22:35.107705 11639 ts_tablet_manager.cc:1403] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.003s
I20260709 12:22:35.107802 11656 raft_consensus.cc:2804] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.108322 11656 raft_consensus.cc:697] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:35.109139 11656 consensus_queue.cc:237] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.110687 11639 raft_consensus.cc:359] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.111446 11639 raft_consensus.cc:385] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.111754 11639 raft_consensus.cc:740] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.112466 11639 consensus_queue.cc:260] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.115480 11639 ts_tablet_manager.cc:1434] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:35.124080 11342 raft_consensus.cc:2468] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:35.126922 11240 catalog_manager.cc:5697] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.127614 11638 tablet_bootstrap.cc:492] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:35.128162 11638 ts_tablet_manager.cc:1403] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.017s
I20260709 12:22:35.129649 11657 raft_consensus.cc:515] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.131040 11638 raft_consensus.cc:359] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.132014 11657 leader_election.cc:290] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.131795 11638 raft_consensus.cc:385] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.132755 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:35.133054 11638 raft_consensus.cc:740] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.133131 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.133450 11342 raft_consensus.cc:3060] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.133759 11491 raft_consensus.cc:3060] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.133917 11638 consensus_queue.cc:260] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.140111 11342 raft_consensus.cc:2468] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:35.139142 11638 ts_tablet_manager.cc:1434] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:22:35.141273 11381 leader_election.cc:304] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.141501 11491 raft_consensus.cc:2468] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 1.
I20260709 12:22:35.142186 11657 raft_consensus.cc:2804] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.142330 11638 tablet_bootstrap.cc:492] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: Bootstrap starting.
I20260709 12:22:35.142789 11657 raft_consensus.cc:697] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:35.143712 11657 consensus_queue.cc:237] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.149222 11638 tablet_bootstrap.cc:654] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.154289 11239 catalog_manager.cc:5697] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.158074 11638 tablet_bootstrap.cc:492] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: No bootstrap required, opened a new log
I20260709 12:22:35.158946 11638 ts_tablet_manager.cc:1403] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.011s
I20260709 12:22:35.160841 11638 raft_consensus.cc:359] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.161351 11638 raft_consensus.cc:385] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.161576 11638 raft_consensus.cc:740] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.162173 11638 consensus_queue.cc:260] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.164196 11638 ts_tablet_manager.cc:1434] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:35.182509 11653 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.183037 11653 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.184830 11653 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:35.185484 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:35.185738 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:35.186545 11415 raft_consensus.cc:2393] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4abc425940dd4061af4d588e0cf78ba2 in current term 1: Already voted for candidate 408b52ec62254d8e99818a554befd6e3 in this term.
I20260709 12:22:35.187480 11456 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:35.188067 11653 raft_consensus.cc:3060] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.193461 11653 raft_consensus.cc:2749] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:35.250556 11538 raft_consensus.cc:493] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.251025 11538 raft_consensus.cc:515] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.252811 11538 leader_election.cc:290] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:35.253407 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:35.253798 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:35.254012 11342 raft_consensus.cc:2468] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 0.
I20260709 12:22:35.254539 11415 raft_consensus.cc:2468] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 0.
I20260709 12:22:35.255103 11455 leader_election.cc:304] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:35.255944 11538 raft_consensus.cc:2804] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:35.256235 11538 raft_consensus.cc:493] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.256515 11538 raft_consensus.cc:3060] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.262158 11538 raft_consensus.cc:515] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.263821 11538 leader_election.cc:290] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:35.264524 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:35.264806 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.265075 11342 raft_consensus.cc:3060] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.265395 11415 raft_consensus.cc:3060] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.270042 11342 raft_consensus.cc:2468] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 1.
I20260709 12:22:35.271134 11455 leader_election.cc:304] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:35.271286 11415 raft_consensus.cc:2468] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 1.
I20260709 12:22:35.271858 11538 raft_consensus.cc:2804] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.272368 11538 raft_consensus.cc:697] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:35.273097 11538 consensus_queue.cc:237] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.275676 11642 raft_consensus.cc:493] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.276250 11642 raft_consensus.cc:515] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.279206 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf243500ca743a3b8600b11b8599177" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:35.279402 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf243500ca743a3b8600b11b8599177" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:35.279770 11415 raft_consensus.cc:2468] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:35.280107 11491 raft_consensus.cc:2468] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:35.278868 11642 leader_election.cc:290] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.280944 11307 leader_election.cc:304] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.281126 11239 catalog_manager.cc:5697] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 1 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:35.281682 11642 raft_consensus.cc:2804] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:35.282142 11642 raft_consensus.cc:493] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.282451 11642 raft_consensus.cc:3060] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.288381 11642 raft_consensus.cc:515] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.290408 11642 leader_election.cc:290] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.291152 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf243500ca743a3b8600b11b8599177" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.291673 11415 raft_consensus.cc:3060] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.291630 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf243500ca743a3b8600b11b8599177" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.292516 11491 raft_consensus.cc:3060] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.297649 11415 raft_consensus.cc:2468] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:35.298163 11491 raft_consensus.cc:2468] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:35.298802 11307 leader_election.cc:304] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.299527 11642 raft_consensus.cc:2804] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.299947 11642 raft_consensus.cc:697] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.300666 11642 consensus_queue.cc:237] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.308202 11239 catalog_manager.cc:5697] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.314492 11657 consensus_queue.cc:1048] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.324815 11656 consensus_queue.cc:1048] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.345912 11642 consensus_queue.cc:1048] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.355804 11642 consensus_queue.cc:1048] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.415149 11642 raft_consensus.cc:493] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.415583 11642 raft_consensus.cc:515] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.417274 11642 leader_election.cc:290] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.418015 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:35.418362 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:35.418692 11415 raft_consensus.cc:2468] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:35.418991 11491 raft_consensus.cc:2468] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 0.
I20260709 12:22:35.419739 11307 leader_election.cc:304] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.420425 11642 raft_consensus.cc:2804] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:35.420783 11642 raft_consensus.cc:493] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.421053 11642 raft_consensus.cc:3060] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.426764 11642 raft_consensus.cc:515] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.428499 11642 leader_election.cc:290] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.429219 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.429477 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.429842 11415 raft_consensus.cc:3060] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.430099 11491 raft_consensus.cc:3060] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.435491 11415 raft_consensus.cc:2468] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:35.435576 11491 raft_consensus.cc:2468] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 1.
I20260709 12:22:35.436622 11307 leader_election.cc:304] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.437350 11642 raft_consensus.cc:2804] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.437824 11642 raft_consensus.cc:697] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.438650 11664 consensus_queue.cc:1048] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.438678 11642 consensus_queue.cc:237] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.448269 11240 catalog_manager.cc:5697] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.450268 11538 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.452342 11538 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.453313 11491 raft_consensus.cc:3060] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.454531 11538 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:35.456176 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408b52ec62254d8e99818a554befd6e3" is_pre_election: true
I20260709 12:22:35.456804 11416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:35.461074 11456 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:35.462421 11538 raft_consensus.cc:2749] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:35.466780 11642 consensus_queue.cc:1048] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.526469 11656 consensus_queue.cc:1048] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.537273 11658 consensus_queue.cc:1048] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.600142 11642 consensus_queue.cc:1048] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.612558 11642 consensus_queue.cc:1048] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.686192 11415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
mode: GRACEFUL
new_leader_uuid: "408b52ec62254d8e99818a554befd6e3"
 from {username='slave'} at 127.0.0.1:41196
I20260709 12:22:35.686949 11415 raft_consensus.cc:606] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Received request to transfer leadership to TS 408b52ec62254d8e99818a554befd6e3
I20260709 12:22:35.701434 11490 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0"
dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
mode: GRACEFUL
new_leader_uuid: "408b52ec62254d8e99818a554befd6e3"
 from {username='slave'} at 127.0.0.1:51368
I20260709 12:22:35.701953 11490 raft_consensus.cc:606] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Received request to transfer leadership to TS 408b52ec62254d8e99818a554befd6e3
I20260709 12:22:35.712617 11342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:22:35.713285 11342 raft_consensus.cc:606] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Received request to transfer leadership to TS 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:35.719802 11342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:22:35.720338 11342 raft_consensus.cc:606] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Received request to transfer leadership to TS 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:35.730170 11658 consensus_queue.cc:1048] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.731041 11342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:22:35.731724 11342 raft_consensus.cc:606] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Received request to transfer leadership to TS 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:35.745042 11658 consensus_queue.cc:1048] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:35.746388 11656 raft_consensus.cc:993] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: : Instructing follower 408b52ec62254d8e99818a554befd6e3 to start an election
I20260709 12:22:35.747076 11703 raft_consensus.cc:1081] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Signalling peer 408b52ec62254d8e99818a554befd6e3 to start an election
I20260709 12:22:35.754510 11342 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
 from {username='slave'} at 127.0.0.1:39772
I20260709 12:22:35.755174 11342 raft_consensus.cc:493] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:35.755578 11342 raft_consensus.cc:3060] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.764175 11342 raft_consensus.cc:515] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.766414 11342 leader_election.cc:290] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.767422 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.768043 11490 raft_consensus.cc:3060] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.769429 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.770110 11415 raft_consensus.cc:3055] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:35.770443 11415 raft_consensus.cc:740] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:35.771298 11415 consensus_queue.cc:260] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.772699 11415 raft_consensus.cc:3060] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.775259 11490 raft_consensus.cc:2468] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 2.
I20260709 12:22:35.776495 11308 leader_election.cc:304] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:35.777328 11642 raft_consensus.cc:2804] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:35.777848 11642 raft_consensus.cc:697] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.778712 11642 consensus_queue.cc:237] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.780627 11415 raft_consensus.cc:2468] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 2.
I20260709 12:22:35.783878 11538 consensus_queue.cc:1048] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.788057 11239 catalog_manager.cc:5697] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 1 to 2, leader changed from 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2) to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.800302 11664 raft_consensus.cc:993] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:35.800884 11664 raft_consensus.cc:1081] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Signalling peer 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:35.802515 11707 consensus_queue.cc:1048] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:35.808169 11642 consensus_queue.cc:1048] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.811594 11416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:55686
I20260709 12:22:35.812206 11416 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:35.812690 11416 raft_consensus.cc:3060] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.824138 11707 raft_consensus.cc:993] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: : Instructing follower 408b52ec62254d8e99818a554befd6e3 to start an election
I20260709 12:22:35.824183 11416 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.824909 11706 raft_consensus.cc:1081] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Signalling peer 408b52ec62254d8e99818a554befd6e3 to start an election
I20260709 12:22:35.827121 11341 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
 from {username='slave'} at 127.0.0.1:39768
I20260709 12:22:35.827760 11342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:35.827850 11341 raft_consensus.cc:493] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:35.828389 11342 raft_consensus.cc:3055] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:35.828487 11341 raft_consensus.cc:3060] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.828768 11342 raft_consensus.cc:740] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.829347 11642 consensus_queue.cc:1048] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:35.829720 11342 consensus_queue.cc:260] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.830894 11342 raft_consensus.cc:3060] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.832187 11416 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.834331 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.834942 11490 raft_consensus.cc:3060] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.837814 11342 raft_consensus.cc:2468] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:35.838913 11381 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.839749 11658 raft_consensus.cc:2804] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:35.840219 11658 raft_consensus.cc:697] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:35.841001 11658 consensus_queue.cc:237] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.844511 11490 raft_consensus.cc:2468] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:35.850023 11341 raft_consensus.cc:515] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.852742 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.853056 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edce2f4af3249e3b06474994cd321d0" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.853333 11415 raft_consensus.cc:3060] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.853689 11490 raft_consensus.cc:3055] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:35.853994 11490 raft_consensus.cc:740] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:35.854867 11490 consensus_queue.cc:260] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.855700 11490 raft_consensus.cc:3060] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.858479 11341 leader_election.cc:290] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.860935 11490 raft_consensus.cc:2468] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 2.
I20260709 12:22:35.861012 11415 raft_consensus.cc:2468] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 408b52ec62254d8e99818a554befd6e3 in term 2.
I20260709 12:22:35.862015 11308 leader_election.cc:304] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:35.862854 11642 raft_consensus.cc:2804] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:35.863387 11642 raft_consensus.cc:697] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.864167 11642 consensus_queue.cc:237] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.865403 11239 catalog_manager.cc:5697] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 1 to 2, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.871131 11240 catalog_manager.cc:5697] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 reported cstate change: term changed from 1 to 2, leader changed from 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3) to 408b52ec62254d8e99818a554befd6e3 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "408b52ec62254d8e99818a554befd6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.898212 11595 raft_consensus.cc:993] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:35.898695 11642 raft_consensus.cc:1081] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Signalling peer 7be56f2c081b41b6939235ccfd24abb1 to start an election
I20260709 12:22:35.900203 11415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
 from {username='slave'} at 127.0.0.1:55686
I20260709 12:22:35.900795 11415 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:35.901114 11415 raft_consensus.cc:3060] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.906370 11415 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.908138 11415 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:35.908928 11341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:35.909602 11341 raft_consensus.cc:3055] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:35.909756 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "7be56f2c081b41b6939235ccfd24abb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
I20260709 12:22:35.910506 11490 raft_consensus.cc:3060] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.909978 11341 raft_consensus.cc:740] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.912916 11642 consensus_queue.cc:1048] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.913695 11341 consensus_queue.cc:260] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.915148 11341 raft_consensus.cc:3060] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.917444 11490 raft_consensus.cc:2468] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:35.919028 11383 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:35.919817 11658 raft_consensus.cc:2804] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:35.920284 11658 raft_consensus.cc:697] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Becoming Leader. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:35.921104 11658 consensus_queue.cc:237] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.923672 11664 raft_consensus.cc:993] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:35.924177 11664 raft_consensus.cc:1081] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Signalling peer 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:35.924991 11341 raft_consensus.cc:2468] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be56f2c081b41b6939235ccfd24abb1 in term 2.
I20260709 12:22:35.926399 11491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b"
dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:49520
I20260709 12:22:35.927325 11491 raft_consensus.cc:493] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:35.927763 11491 raft_consensus.cc:3060] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.930457 11712 consensus_queue.cc:1048] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.934705 11239 catalog_manager.cc:5697] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 reported cstate change: term changed from 1 to 2, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "7be56f2c081b41b6939235ccfd24abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:35.939042 11491 raft_consensus.cc:515] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.939913 11490 raft_consensus.cc:1240] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Rejecting Update request from peer 408b52ec62254d8e99818a554befd6e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:35.941761 11491 leader_election.cc:290] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:35.941841 11341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:35.942970 11341 raft_consensus.cc:3055] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:35.943260 11341 raft_consensus.cc:740] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:35.943379 11595 consensus_queue.cc:1059] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:35.944480 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:35.944479 11341 consensus_queue.cc:260] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.947052 11415 raft_consensus.cc:3060] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.947520 11341 raft_consensus.cc:3060] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.952925 11341 raft_consensus.cc:2468] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 2.
I20260709 12:22:35.953181 11415 raft_consensus.cc:2468] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 2.
I20260709 12:22:35.954015 11455 leader_election.cc:304] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:35.954782 11707 raft_consensus.cc:2804] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:35.955147 11707 raft_consensus.cc:697] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 2 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:35.955771 11707 consensus_queue.cc:237] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:35.961851 11239 catalog_manager.cc:5697] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 1 to 2, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:36.273034 11341 raft_consensus.cc:1275] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.274434 11658 consensus_queue.cc:1048] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.282052 11491 raft_consensus.cc:1275] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.283716 11658 consensus_queue.cc:1048] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.294185 11415 raft_consensus.cc:1275] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Refusing update from remote peer 408b52ec62254d8e99818a554befd6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.295411 11712 consensus_queue.cc:1048] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.305408 11491 raft_consensus.cc:1275] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 408b52ec62254d8e99818a554befd6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.307416 11712 consensus_queue.cc:1048] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.352095 11491 raft_consensus.cc:1275] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 408b52ec62254d8e99818a554befd6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.353272 11712 consensus_queue.cc:1048] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.362357 11415 raft_consensus.cc:1275] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Refusing update from remote peer 408b52ec62254d8e99818a554befd6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.364070 11663 consensus_queue.cc:1048] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.424643 11491 raft_consensus.cc:1275] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.426047 11658 consensus_queue.cc:1048] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.433136 11341 raft_consensus.cc:1275] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 7be56f2c081b41b6939235ccfd24abb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.435060 11701 consensus_queue.cc:1048] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.468410 11415 raft_consensus.cc:1275] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.469906 11706 consensus_queue.cc:1048] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:36.474871 11341 raft_consensus.cc:1275] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:36.476878 11716 consensus_queue.cc:1048] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:38.735284 11237 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46414:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:22:38.756457 11237 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46414:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:22:38.795725 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:22:39.004032 11308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:34613, user_credentials={real_user=slave}} blocked reactor thread for 49258.8us
I20260709 12:22:39.201675 11777 tablet.cc:1724] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.203637 11772 tablet.cc:1724] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.205984 11773 tablet.cc:1724] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.207778 11776 tablet.cc:1724] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.212311 11775 tablet.cc:1724] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.205938 11771 tablet.cc:1724] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.164184 11763 tablet.cc:1724] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:39.235039 11443 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:0279a578b9e047f0b044e5ca890208f7)
I20260709 12:22:39.247076 11417 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
mode: GRACEFUL
new_leader_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:41212
I20260709 12:22:39.247803 11417 raft_consensus.cc:606] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Received request to transfer leadership to TS 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:39.226853 11780 tablet.cc:1724] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:39.247143 11443 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:0279a578b9e047f0b044e5ca890208f7)
W20260709 12:22:39.242633 11752 log.cc:927] Time spent T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.002s
I20260709 12:22:39.212483 11778 tablet.cc:1724] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:39.314694 11368 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:7dffc97ecfdd4a8db556f6c1f51cefb9)
I20260709 12:22:39.314976 11784 tablet.cc:1724] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.341477 11789 tablet.cc:1724] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.347925 11771 tablet.cc:1724] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.351727 11775 tablet.cc:1724] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.357327 11776 tablet.cc:1724] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.359165 11763 tablet.cc:1724] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.352118 11786 tablet.cc:1724] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.361591 11789 tablet.cc:1724] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.365996 11778 tablet.cc:1724] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.367522 11780 tablet.cc:1724] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.368381 11777 tablet.cc:1724] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.377288 11772 tablet.cc:1724] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.401505 11747 raft_consensus.cc:993] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: : Instructing follower 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:39.401997 11747 raft_consensus.cc:1081] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Signalling peer 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:39.404721 11482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3"
dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:49536
I20260709 12:22:39.405380 11482 raft_consensus.cc:493] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:39.405782 11482 raft_consensus.cc:3060] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:39.415180 11773 tablet.cc:1724] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.421396 11775 tablet.cc:1724] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.425806 11339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:59738
I20260709 12:22:39.426759 11339 raft_consensus.cc:606] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Received request to transfer leadership to TS 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:39.428568 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:22:39.441385 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:39.447803 11789 tablet.cc:1724] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.470489 11339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97de4708d35e47afb683b92846175828"
dest_uuid: "408b52ec62254d8e99818a554befd6e3"
mode: GRACEFUL
new_leader_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:59738
I20260709 12:22:39.471166 11339 raft_consensus.cc:606] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Received request to transfer leadership to TS 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:39.474905 11417 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161"
dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
mode: GRACEFUL
new_leader_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:41212
I20260709 12:22:39.475425 11417 raft_consensus.cc:606] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Received request to transfer leadership to TS 4abc425940dd4061af4d588e0cf78ba2
I20260709 12:22:39.476794 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:39.477442 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:39.478847 11777 tablet.cc:1724] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.470081 11778 tablet.cc:1724] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:39.484527 11482 raft_consensus.cc:515] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.485668 11484 raft_consensus.cc:1240] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Rejecting Update request from peer 7be56f2c081b41b6939235ccfd24abb1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:39.487399 11339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:39.488106 11339 raft_consensus.cc:3060] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:39.488900 11482 leader_election.cc:290] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:39.492740 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:39.493439 11417 raft_consensus.cc:3055] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:39.493791 11417 raft_consensus.cc:740] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:39.494688 11417 consensus_queue.cc:260] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.496070 11417 raft_consensus.cc:3060] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:39.501545 11339 raft_consensus.cc:2468] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 2.
I20260709 12:22:39.504388 11455 leader_election.cc:304] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:39.505213 11706 raft_consensus.cc:2804] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:39.505889 11706 raft_consensus.cc:697] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:39.506973 11706 consensus_queue.cc:237] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.505167 11417 raft_consensus.cc:2468] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 2.
I20260709 12:22:39.520056 11237 catalog_manager.cc:5697] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 1 to 2, leader changed from 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2) to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:39.521049 11237 catalog_manager.cc:5789] TS 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613) does not have the latest schema for tablet a7b992a1a0614882a13890e523e3e1a3 (table soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef]). Expected version 1 got 0
I20260709 12:22:39.530085 11238 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8: soft_deleted_table [id=3ba262927ef44dcfb609c5c8376312ef] alter complete (version 1)
I20260709 12:22:39.575731 11748 raft_consensus.cc:993] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: : Instructing follower 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:39.576349 11718 raft_consensus.cc:1081] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Signalling peer 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:39.577943 11484 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161"
dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:49536
I20260709 12:22:39.578554 11484 raft_consensus.cc:493] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:39.578915 11484 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:39.584736 11484 raft_consensus.cc:515] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.587393 11339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:39.587602 11484 leader_election.cc:290] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:39.588014 11339 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:39.588004 11411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:39.592311 11411 raft_consensus.cc:3055] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:39.592590 11411 raft_consensus.cc:740] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7be56f2c081b41b6939235ccfd24abb1, State: Running, Role: LEADER
I20260709 12:22:39.593241 11411 consensus_queue.cc:260] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.593533 11339 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 3.
I20260709 12:22:39.594274 11411 raft_consensus.cc:3060] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:39.594796 11455 leader_election.cc:304] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3, 4abc425940dd4061af4d588e0cf78ba2; no voters: 
I20260709 12:22:39.595713 11706 raft_consensus.cc:2804] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:39.596172 11706 raft_consensus.cc:697] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:39.598858 11706 consensus_queue.cc:237] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.602669 11411 raft_consensus.cc:2468] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 3.
I20260709 12:22:39.608496 11237 catalog_manager.cc:5697] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 2 to 3, leader changed from 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2) to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 3 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:39.717993 11756 raft_consensus.cc:993] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:39.718706 11743 raft_consensus.cc:1081] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Signalling peer 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:39.720296 11484 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7"
dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:49520
I20260709 12:22:39.720927 11484 raft_consensus.cc:493] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:39.721231 11484 raft_consensus.cc:3060] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:39.726719 11484 raft_consensus.cc:515] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.728310 11484 leader_election.cc:290] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 election: Requested vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:39.729398 11411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:39.729550 11339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:39.730127 11411 raft_consensus.cc:3060] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:39.730175 11339 raft_consensus.cc:3055] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:39.730657 11339 raft_consensus.cc:740] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:39.731491 11339 consensus_queue.cc:260] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.732761 11339 raft_consensus.cc:3060] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:39.739506 11411 raft_consensus.cc:2468] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 2.
I20260709 12:22:39.739969 11339 raft_consensus.cc:2468] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 2.
I20260709 12:22:39.740904 11456 leader_election.cc:304] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:39.741848 11706 raft_consensus.cc:2804] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:39.742439 11706 raft_consensus.cc:697] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:39.743218 11706 consensus_queue.cc:237] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:39.750789 11237 catalog_manager.cc:5697] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 1 to 2, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 2 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:40.003508 11411 raft_consensus.cc:1275] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:40.004813 11706 consensus_queue.cc:1048] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.011708 11796 tablet.cc:1724] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:40.030558 11743 raft_consensus.cc:993] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: : Instructing follower 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:40.030645 11339 raft_consensus.cc:1275] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:40.031255 11756 raft_consensus.cc:1081] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Signalling peer 4abc425940dd4061af4d588e0cf78ba2 to start an election
I20260709 12:22:40.033234 11484 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97de4708d35e47afb683b92846175828"
dest_uuid: "4abc425940dd4061af4d588e0cf78ba2"
 from {username='slave'} at 127.0.0.1:49520
I20260709 12:22:40.033793 11484 raft_consensus.cc:493] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.034178 11484 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:40.037267 11781 consensus_queue.cc:1048] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.045161 11484 raft_consensus.cc:515] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:40.047513 11484 leader_election.cc:290] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 election: Requested vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947)
I20260709 12:22:40.048238 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7be56f2c081b41b6939235ccfd24abb1"
I20260709 12:22:40.048892 11417 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:40.050925 11339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "408b52ec62254d8e99818a554befd6e3"
I20260709 12:22:40.051551 11339 raft_consensus.cc:3055] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:40.051862 11339 raft_consensus.cc:740] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 408b52ec62254d8e99818a554befd6e3, State: Running, Role: LEADER
I20260709 12:22:40.052675 11339 consensus_queue.cc:260] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:40.054706 11796 tablet.cc:1719] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: Skipping requested alter to schema version 1, tablet already version 1
I20260709 12:22:40.058663 11339 raft_consensus.cc:3060] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:40.069545 11339 raft_consensus.cc:2468] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 3.
I20260709 12:22:40.073011 11417 raft_consensus.cc:2468] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abc425940dd4061af4d588e0cf78ba2 in term 3.
I20260709 12:22:40.074265 11456 leader_election.cc:304] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1; no voters: 
I20260709 12:22:40.075451 11706 raft_consensus.cc:2804] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:40.077219 11706 raft_consensus.cc:697] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 LEADER]: Becoming Leader. State: Replica: 4abc425940dd4061af4d588e0cf78ba2, State: Running, Role: LEADER
I20260709 12:22:40.078138 11706 consensus_queue.cc:237] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:40.076447 11800 tablet.cc:1719] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: Skipping requested alter to schema version 1, tablet already version 1
I20260709 12:22:40.081879 11802 tablet.cc:1719] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: Skipping requested alter to schema version 1, tablet already version 1
I20260709 12:22:40.087843 11237 catalog_manager.cc:5697] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 reported cstate change: term changed from 2 to 3, leader changed from 408b52ec62254d8e99818a554befd6e3 (127.30.25.1) to 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3). New cstate: current_term: 3 leader_uuid: "4abc425940dd4061af4d588e0cf78ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.135299 11417 raft_consensus.cc:1275] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:40.136449 11706 consensus_queue.cc:1048] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.144147 11339 raft_consensus.cc:1275] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:40.145586 11737 consensus_queue.cc:1048] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.146181 11339 raft_consensus.cc:1275] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:40.147729 11737 consensus_queue.cc:1048] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.161739 11417 raft_consensus.cc:1275] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:40.163084 11797 consensus_queue.cc:1048] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.478570 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:40.484256 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:40.484683 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:40.489310 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:40.489705 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:40.509260 11417 raft_consensus.cc:1275] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:40.510684 11706 consensus_queue.cc:1048] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:40.519515 11341 raft_consensus.cc:1275] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Refusing update from remote peer 4abc425940dd4061af4d588e0cf78ba2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:40.521018 11797 consensus_queue.cc:1048] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:41.490850 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:41.496752 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:41.497324 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:41.502506 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:41.502941 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:42.504458 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:42.515444 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:42.516130 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:42.524658 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:42.525022 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:43.526292 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:43.531852 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:43.532337 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:43.536932 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:43.537252 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:44.538141 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:44.543004 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:44.543416 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:44.548274 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:44.548698 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:45.549719 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:45.556459 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:45.556986 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:45.562410 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:45.562948 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:46.563887 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:46.569518 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:46.570006 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:46.574506 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:46.574911 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:47.575897 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:47.580739 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:47.581158 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:47.587260 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:47.587642 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:48.588574 30820 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:48.593778 30820 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:48.594223 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:48.599387 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:48.599829 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:50.235031 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
W20260709 12:22:50.441351 11456 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.008s	sys 0.008s
W20260709 12:22:50.442070 11456 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.008s	sys 0.008s
W20260709 12:22:50.458621 11381 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 12:22:50.537634 11456 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.2:46599, user_credentials={real_user=slave}} blocked reactor thread for 51160us
W20260709 12:22:51.207634 11455 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.001s	sys 0.010s
W20260709 12:22:51.208294 11455 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.001s	sys 0.010s
I20260709 12:22:51.272953 11841 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:51.292979 11841 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.612021 11841 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:51.612799 11454 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.000s	sys 0.022s
W20260709 12:22:51.613238 11454 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.000s	sys 0.023s
I20260709 12:22:51.632915 11844 raft_consensus.cc:493] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4abc425940dd4061af4d588e0cf78ba2)
I20260709 12:22:51.633719 11844 raft_consensus.cc:515] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.634605 11845 raft_consensus.cc:493] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:51.635366 11845 raft_consensus.cc:515] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.636976 11844 leader_election.cc:290] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.637674 11845 leader_election.cc:290] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
W20260709 12:22:51.642642 11271 debug-util.cc:398] Leaking SignalData structure 0x7b0800409ee0 after lost signal to thread 30823
I20260709 12:22:51.643435 11846 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
W20260709 12:22:51.643919 11271 debug-util.cc:398] Leaking SignalData structure 0x7b080014be40 after lost signal to thread 11274
I20260709 12:22:51.644158 11846 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.645126 11844 raft_consensus.cc:493] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4abc425940dd4061af4d588e0cf78ba2)
W20260709 12:22:51.645414 11271 debug-util.cc:398] Leaking SignalData structure 0x7b080036a5c0 after lost signal to thread 11366
I20260709 12:22:51.645754 11844 raft_consensus.cc:515] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.647833 11844 leader_election.cc:290] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.647981 11846 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.649744 11846 raft_consensus.cc:493] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4abc425940dd4061af4d588e0cf78ba2)
I20260709 12:22:51.650406 11846 raft_consensus.cc:515] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
W20260709 12:22:51.651281 30820 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 11354)
I20260709 12:22:51.652628 11846 leader_election.cc:290] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
W20260709 12:22:51.669355 11271 debug-util.cc:398] Leaking SignalData structure 0x7b0800301b00 after lost signal to thread 11441
W20260709 12:22:51.670745 11271 debug-util.cc:398] Leaking SignalData structure 0x7b08002350e0 after lost signal to thread 11515
I20260709 12:22:51.672569 11484 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49536 (request call id 255) took 1240 ms. Trace:
I20260709 12:22:51.673020 11484 rpcz_store.cc:276] 0709 12:22:50.432269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.575061 (+1142792us) service_pool.cc:224] Handling call
0709 12:22:51.575492 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:51.612819 (+ 37327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:51.612852 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:51.612898 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:51.612925 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.672166 (+ 59241us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b5400a1e854. stack: 00007f22890b0343 00007f22890b0240 00007f2288a3f1d4 00007f229277d304 00007f229277d299 00007f228c138571 00007f228c1cfe01 00007f228c1ce83e 00007f2291193419 00007f228a70abd4 00007f228a70ab62 00007f228a70aa91 00007f228a70aa0c 00007f228a709cc2 00007f228a50a48c 00007f228a5098c6
0709 12:22:51.672215 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.672378 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85276544}
I20260709 12:22:51.705001 11413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.719159 11846 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:51.720522 11486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
W20260709 12:22:51.720953 11381 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.259s	user 0.010s	sys 0.031s
W20260709 12:22:51.721589 11381 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.010s	sys 0.031s
W20260709 12:22:51.721909 11381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.1:43947, user_credentials={real_user=slave}} blocked reactor thread for 1.26332e+06us
I20260709 12:22:51.719810 11845 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:51.722649 11845 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.722841 11846 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.724494 11413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.724814 11846 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.730336 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0279a578b9e047f0b044e5ca890208f7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.732946 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.735809 11490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b992a1a0614882a13890e523e3e1a3" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.742933 11845 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.744009 11487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d7525a305245759b35f92da7a5dfc7" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.744396 11845 raft_consensus.cc:493] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4abc425940dd4061af4d588e0cf78ba2)
I20260709 12:22:51.744922 11845 raft_consensus.cc:515] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } } peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } }
I20260709 12:22:51.746203 11482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.747990 11483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.750931 11845 leader_election.cc:290] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.753355 11844 raft_consensus.cc:493] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:51.753105 11846 raft_consensus.cc:493] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:51.754000 11844 raft_consensus.cc:515] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.754236 11846 raft_consensus.cc:515] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.760134 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.761405 11487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.761473 11846 leader_election.cc:290] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.763748 11844 leader_election.cc:290] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.766126 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.766198 11847 raft_consensus.cc:493] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4abc425940dd4061af4d588e0cf78ba2)
I20260709 12:22:51.767141 11847 raft_consensus.cc:515] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.770587 11849 raft_consensus.cc:493] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4abc425940dd4061af4d588e0cf78ba2)
I20260709 12:22:51.771406 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.771477 11849 raft_consensus.cc:515] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.773123 11413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df40005c1ebe414fb797a878bae5eaf9" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.773944 11409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.775424 11849 leader_election.cc:290] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.776058 11409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.776338 11414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1eda1818ba4ecdb846e13977678111" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.777050 11491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.781066 11409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8315b03aa974b2b8cc5c155f18f2248" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.782496 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b2319fba4945e2a673d0c96fe90161" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.770033 11847 leader_election.cc:290] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599), 4abc425940dd4061af4d588e0cf78ba2 (127.30.25.3:34613)
I20260709 12:22:51.785075 11415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8afccb6e54a4a17b883a901da0bef6b" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.787086 11411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ec255b57d0c4527aae93cdfaa757e51" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.799839 11410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bdbde5464e4ec9b2ffc980a92280ae" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
W20260709 12:22:51.805459 11381 consensus_peers.cc:597] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.821208 11381 consensus_peers.cc:597] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.822005 11381 consensus_peers.cc:597] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.822687 11381 consensus_peers.cc:597] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.823297 11381 consensus_peers.cc:597] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.823886 11381 consensus_peers.cc:597] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.835636 11455 consensus_peers.cc:597] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.836398 11455 consensus_peers.cc:597] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.836932 11455 consensus_peers.cc:597] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.837476 11455 consensus_peers.cc:597] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:51.837980 11455 consensus_peers.cc:597] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:51.847558 11482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97de4708d35e47afb683b92846175828" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
W20260709 12:22:51.851013 11455 consensus_peers.cc:597] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:51.852816 11308 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.854121 11308 leader_election.cc:304] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.863256 11845 raft_consensus.cc:2749] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.863771 11849 raft_consensus.cc:2749] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.859629 11487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd96fbe8d2944b5859796e118975195" candidate_uuid: "408b52ec62254d8e99818a554befd6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4abc425940dd4061af4d588e0cf78ba2" is_pre_election: true
I20260709 12:22:51.864163 11308 leader_election.cc:304] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.869524 11849 raft_consensus.cc:2749] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.870821 11308 leader_election.cc:304] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.871626 11849 raft_consensus.cc:2749] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.873028 11308 leader_election.cc:304] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.875627 11849 raft_consensus.cc:2749] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:51.878407 11308 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.890837 11849 raft_consensus.cc:2749] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:51.892247 11308 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.895499 11849 raft_consensus.cc:2749] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.896834 11308 leader_election.cc:304] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.897799 11849 raft_consensus.cc:2749] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:51.899190 11308 leader_election.cc:304] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.901760 11849 raft_consensus.cc:2749] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:51.903550 11308 leader_election.cc:304] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.905049 11308 leader_election.cc:304] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.906761 11852 raft_consensus.cc:2749] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:51.906862 11308 leader_election.cc:304] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 408b52ec62254d8e99818a554befd6e3; no voters: 4abc425940dd4061af4d588e0cf78ba2, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.904392 11849 raft_consensus.cc:2749] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.908572 11852 raft_consensus.cc:2749] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:51.911553 11455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.30.25.1:43947, user_credentials={real_user=slave}} blocked reactor thread for 59943.3us
W20260709 12:22:51.920164 11455 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:51.920630 11455 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.921672 11841 raft_consensus.cc:2749] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:51.936040 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:51.937357 30820 tablet_replica.cc:333] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.939157 30820 raft_consensus.cc:2243] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:51.940621 30820 raft_consensus.cc:2272] T a7b992a1a0614882a13890e523e3e1a3 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.945508 30820 tablet_replica.cc:333] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.946738 30820 raft_consensus.cc:2243] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:51.947546 30820 raft_consensus.cc:2272] T d0d7525a305245759b35f92da7a5dfc7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.951447 30820 tablet_replica.cc:333] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.952534 30820 raft_consensus.cc:2243] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:51.953946 30820 raft_consensus.cc:2272] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.958092 30820 tablet_replica.cc:333] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.959229 30820 raft_consensus.cc:2243] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:51.960391 11841 raft_consensus.cc:493] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:51.961055 11841 raft_consensus.cc:515] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.962616 30820 raft_consensus.cc:2272] T 5edce2f4af3249e3b06474994cd321d0 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.966683 30820 tablet_replica.cc:333] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
W20260709 12:22:51.967429 11455 leader_election.cc:336] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:51.967723 30820 raft_consensus.cc:2243] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:51.968802 30820 raft_consensus.cc:2272] T a8afccb6e54a4a17b883a901da0bef6b P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.969468 11417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a35d4770d87426aab884733ee2c3d59" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.970876 11830 raft_consensus.cc:493] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:51.971506 11830 raft_consensus.cc:515] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:51.972779 30820 tablet_replica.cc:333] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.973855 30820 raft_consensus.cc:2243] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:51.974802 30820 raft_consensus.cc:2272] T df40005c1ebe414fb797a878bae5eaf9 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.975905 11412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dffc97ecfdd4a8db556f6c1f51cefb9" candidate_uuid: "4abc425940dd4061af4d588e0cf78ba2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7be56f2c081b41b6939235ccfd24abb1" is_pre_election: true
I20260709 12:22:51.978845 30820 tablet_replica.cc:333] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.978672 11456 leader_election.cc:304] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.979983 30820 raft_consensus.cc:2243] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:51.979979 11854 raft_consensus.cc:2749] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:51.981459 30820 raft_consensus.cc:2272] T 07af2adb5e0745088a20cdf76ead9d5c P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.984544 11841 leader_election.cc:290] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:51.985246 11455 leader_election.cc:336] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:51.973922 11830 leader_election.cc:290] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:51.985488 30820 tablet_replica.cc:333] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.986392 11455 leader_election.cc:304] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:51.987288 11830 raft_consensus.cc:2749] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:51.988282 30820 raft_consensus.cc:2243] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:51.989027 30820 raft_consensus.cc:2272] T 7ec255b57d0c4527aae93cdfaa757e51 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.993073 30820 tablet_replica.cc:333] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:51.994073 30820 raft_consensus.cc:2243] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:51.995505 30820 raft_consensus.cc:2272] T 9a35d4770d87426aab884733ee2c3d59 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:51.999393 30820 tablet_replica.cc:333] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.000324 30820 raft_consensus.cc:2243] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.001111 30820 raft_consensus.cc:2272] T 97de4708d35e47afb683b92846175828 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.004793 30820 tablet_replica.cc:333] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.005636 30820 raft_consensus.cc:2243] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.006443 30820 raft_consensus.cc:2272] T b0b2319fba4945e2a673d0c96fe90161 P 408b52ec62254d8e99818a554befd6e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.010879 30820 tablet_replica.cc:333] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.011895 30820 raft_consensus.cc:2243] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.012617 30820 raft_consensus.cc:2272] T 5d1eda1818ba4ecdb846e13977678111 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.016860 30820 tablet_replica.cc:333] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.017794 30820 raft_consensus.cc:2243] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.019145 30820 raft_consensus.cc:2272] T 1998e87dc4b14e578102b92e9bfa9f03 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.022840 30820 tablet_replica.cc:333] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.023659 30820 raft_consensus.cc:2243] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.024351 30820 raft_consensus.cc:2272] T 0bd96fbe8d2944b5859796e118975195 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.028561 30820 tablet_replica.cc:333] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.029392 30820 raft_consensus.cc:2243] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.031035 30820 raft_consensus.cc:2272] T dbf243500ca743a3b8600b11b8599177 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.035106 30820 tablet_replica.cc:333] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.035943 30820 raft_consensus.cc:2243] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.036705 30820 raft_consensus.cc:2272] T 96bdbde5464e4ec9b2ffc980a92280ae P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.040722 30820 tablet_replica.cc:333] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.041472 30820 raft_consensus.cc:2243] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.042178 30820 raft_consensus.cc:2272] T 0279a578b9e047f0b044e5ca890208f7 P 408b52ec62254d8e99818a554befd6e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.046164 30820 tablet_replica.cc:333] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3: stopping tablet replica
I20260709 12:22:52.046962 30820 raft_consensus.cc:2243] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.047650 30820 raft_consensus.cc:2272] T e8315b03aa974b2b8cc5c155f18f2248 P 408b52ec62254d8e99818a554befd6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:52.141921 11381 connection.cc:573] client connection to 127.30.25.1:43947 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:22:52.169651 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
I20260709 12:22:52.301261 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
W20260709 12:22:52.320904 11456 consensus_peers.cc:597] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:52.338264 11456 consensus_peers.cc:597] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:52.367962 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:52.369063 30820 tablet_replica.cc:333] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.370704 30820 raft_consensus.cc:2243] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.371965 30820 raft_consensus.cc:2272] T 5edce2f4af3249e3b06474994cd321d0 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.375339 30820 tablet_replica.cc:333] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.376268 30820 raft_consensus.cc:2243] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.377148 30820 raft_consensus.cc:2272] T d0d7525a305245759b35f92da7a5dfc7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.381047 30820 tablet_replica.cc:333] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.382090 30820 raft_consensus.cc:2243] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.383711 30820 raft_consensus.cc:2272] T a7b992a1a0614882a13890e523e3e1a3 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.388393 30820 tablet_replica.cc:333] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.389609 30820 raft_consensus.cc:2243] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.390689 30820 raft_consensus.cc:2272] T dbf243500ca743a3b8600b11b8599177 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.394405 30820 tablet_replica.cc:333] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.395392 30820 raft_consensus.cc:2243] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.396744 30820 raft_consensus.cc:2272] T 0279a578b9e047f0b044e5ca890208f7 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.400493 30820 tablet_replica.cc:333] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.401355 30820 raft_consensus.cc:2243] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.402724 30820 raft_consensus.cc:2272] T 96bdbde5464e4ec9b2ffc980a92280ae P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.406353 30820 tablet_replica.cc:333] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.407217 30820 raft_consensus.cc:2243] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.408582 30820 raft_consensus.cc:2272] T 0bd96fbe8d2944b5859796e118975195 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.412456 30820 tablet_replica.cc:333] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.413323 30820 raft_consensus.cc:2243] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.414132 30820 raft_consensus.cc:2272] T 1998e87dc4b14e578102b92e9bfa9f03 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.418076 30820 tablet_replica.cc:333] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.419051 30820 raft_consensus.cc:2243] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.420007 30820 raft_consensus.cc:2272] T a8afccb6e54a4a17b883a901da0bef6b P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.423925 30820 tablet_replica.cc:333] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.424801 30820 raft_consensus.cc:2243] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.425719 30820 raft_consensus.cc:2272] T 07af2adb5e0745088a20cdf76ead9d5c P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.429612 30820 tablet_replica.cc:333] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.430573 30820 raft_consensus.cc:2243] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.432104 30820 raft_consensus.cc:2272] T df40005c1ebe414fb797a878bae5eaf9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.435776 30820 tablet_replica.cc:333] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.436648 30820 raft_consensus.cc:2243] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.438005 30820 raft_consensus.cc:2272] T e8315b03aa974b2b8cc5c155f18f2248 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.441901 30820 tablet_replica.cc:333] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.442895 30820 raft_consensus.cc:2243] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.444038 30820 raft_consensus.cc:2272] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.447904 30820 tablet_replica.cc:333] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.448781 30820 raft_consensus.cc:2243] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.449570 30820 raft_consensus.cc:2272] T 9a35d4770d87426aab884733ee2c3d59 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.453558 30820 tablet_replica.cc:333] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.454484 30820 raft_consensus.cc:2243] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.455235 30820 raft_consensus.cc:2272] T 7ec255b57d0c4527aae93cdfaa757e51 P 7be56f2c081b41b6939235ccfd24abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.458971 30820 tablet_replica.cc:333] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.459843 30820 raft_consensus.cc:2243] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:52.461199 30820 raft_consensus.cc:2272] T 5d1eda1818ba4ecdb846e13977678111 P 7be56f2c081b41b6939235ccfd24abb1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.464663 30820 tablet_replica.cc:333] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.465448 30820 raft_consensus.cc:2243] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.466251 30820 raft_consensus.cc:2272] T b0b2319fba4945e2a673d0c96fe90161 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:52.470026 30820 tablet_replica.cc:333] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1: stopping tablet replica
I20260709 12:22:52.470824 30820 raft_consensus.cc:2243] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:52.471745 30820 raft_consensus.cc:2272] T 97de4708d35e47afb683b92846175828 P 7be56f2c081b41b6939235ccfd24abb1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:52.575610 11456 consensus_peers.cc:597] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:52.577661 11456 consensus_peers.cc:597] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:52.599819 11456 consensus_peers.cc:597] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:52.635373 11456 consensus_peers.cc:597] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:52.683202 11438 debug-util.cc:398] Leaking SignalData structure 0x7b08001dca80 after lost signal to thread 30823
W20260709 12:22:52.685061 11438 debug-util.cc:398] Leaking SignalData structure 0x7b080024a880 after lost signal to thread 11274
W20260709 12:22:52.687077 11438 debug-util.cc:398] Leaking SignalData structure 0x7b08002a0e20 after lost signal to thread 11366
W20260709 12:22:52.689744 11438 debug-util.cc:398] Leaking SignalData structure 0x7b0800268800 after lost signal to thread 11441
W20260709 12:22:52.691672 11438 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2f00 after lost signal to thread 11515
I20260709 12:22:53.241027 11830 raft_consensus.cc:493] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:53.241987 11830 raft_consensus.cc:515] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.243175 11842 raft_consensus.cc:493] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:53.243884 11842 raft_consensus.cc:515] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.244969 11830 leader_election.cc:290] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:53.246222 11842 leader_election.cc:290] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.253250 11456 leader_election.cc:336] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.253461 11842 raft_consensus.cc:493] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
W20260709 12:22:53.253852 11456 leader_election.cc:336] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.254182 11842 raft_consensus.cc:515] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.256740 11842 leader_election.cc:290] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:53.260767 11842 raft_consensus.cc:493] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
W20260709 12:22:53.261731 11455 leader_election.cc:336] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:53.261492 11842 raft_consensus.cc:515] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.262252 11455 leader_election.cc:304] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.264485 11842 leader_election.cc:290] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
I20260709 12:22:53.265161 11830 raft_consensus.cc:2749] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:53.266233 11455 leader_election.cc:336] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:53.266726 11455 leader_election.cc:304] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
W20260709 12:22:53.267134 11456 leader_election.cc:336] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.267729 11842 raft_consensus.cc:2749] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:53.270488 11456 leader_election.cc:336] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:53.271207 11455 leader_election.cc:336] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:53.271677 11455 leader_election.cc:304] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.272387 11842 raft_consensus.cc:2749] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:53.275216 11455 leader_election.cc:336] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:53.275671 11455 leader_election.cc:304] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.276393 11842 raft_consensus.cc:2749] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:53.520516 11842 raft_consensus.cc:493] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:53.521267 11842 raft_consensus.cc:515] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.523958 11842 leader_election.cc:290] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.530109 11456 leader_election.cc:336] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:53.530740 11455 leader_election.cc:336] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:53.531302 11455 leader_election.cc:304] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.532099 11842 raft_consensus.cc:2749] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:53.537099 30820 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11438)
I20260709 12:22:53.660552 11842 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:53.661269 11842 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.663877 11842 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.669641 11455 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:53.671380 11456 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.671802 11456 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.672580 11842 raft_consensus.cc:2749] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:53.746209 11842 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:53.746809 11842 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.749236 11842 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.753897 11455 leader_election.cc:336] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:53.754923 11456 leader_election.cc:336] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.755368 11456 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.756115 11842 raft_consensus.cc:2749] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:53.772053 11842 raft_consensus.cc:493] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:53.772732 11842 raft_consensus.cc:515] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.775326 11842 leader_election.cc:290] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.780999 11456 leader_election.cc:336] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:53.781526 11455 leader_election.cc:336] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:53.781972 11455 leader_election.cc:304] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.782744 11842 raft_consensus.cc:2749] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:53.792553 11842 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:53.793272 11842 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.795761 11842 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.801142 11455 leader_election.cc:336] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:53.801657 11456 leader_election.cc:336] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.802115 11456 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.802923 11842 raft_consensus.cc:2749] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:53.803722 11842 raft_consensus.cc:493] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:53.804332 11842 raft_consensus.cc:515] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.806582 11842 leader_election.cc:290] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.812290 11455 leader_election.cc:336] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:53.816208 11456 leader_election.cc:336] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.816659 11456 leader_election.cc:304] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.817467 11861 raft_consensus.cc:2749] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:53.853211 11455 consensus_peers.cc:597] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:53.887914 11861 raft_consensus.cc:493] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:53.888645 11861 raft_consensus.cc:515] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:53.891202 11861 leader_election.cc:290] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:53.895833 11455 leader_election.cc:336] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:53.896862 11456 leader_election.cc:336] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:53.897300 11456 leader_election.cc:304] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:53.898058 11842 raft_consensus.cc:2749] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:54.000396 11842 raft_consensus.cc:493] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:54.001170 11842 raft_consensus.cc:515] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:54.003862 11842 leader_election.cc:290] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:54.008843 11456 leader_election.cc:336] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:54.010006 11455 leader_election.cc:336] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:54.010460 11455 leader_election.cc:304] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:54.011281 11842 raft_consensus.cc:2749] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:54.053483 11455 consensus_peers.cc:597] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:54.058439 11455 consensus_peers.cc:597] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:54.151439 11455 consensus_peers.cc:597] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:54.367365 11455 consensus_peers.cc:597] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:54.378454 11455 consensus_peers.cc:597] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:54.537719 30820 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11438)
W20260709 12:22:54.857649 11456 consensus_peers.cc:597] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:54.865198 11842 raft_consensus.cc:493] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:54.865886 11842 raft_consensus.cc:515] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:54.869166 11842 leader_election.cc:290] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:54.875351 11455 leader_election.cc:336] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:54.879801 11456 leader_election.cc:336] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:54.880241 11456 leader_election.cc:304] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:54.880998 11842 raft_consensus.cc:2749] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:54.938349 11456 consensus_peers.cc:597] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:54.977087 11864 raft_consensus.cc:493] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:54.977715 11864 raft_consensus.cc:515] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:54.982822 11864 leader_election.cc:290] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:54.985520 11455 leader_election.cc:336] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:54.986136 11456 leader_election.cc:336] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:54.986604 11456 leader_election.cc:304] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:54.987370 11864 raft_consensus.cc:2749] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:55.003553 11456 consensus_peers.cc:597] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:55.023615 11456 consensus_peers.cc:597] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:55.124306 11456 consensus_peers.cc:597] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:55.139151 11456 consensus_peers.cc:597] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:55.188668 11864 raft_consensus.cc:493] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:55.189389 11864 raft_consensus.cc:515] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.191851 11864 leader_election.cc:290] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.196834 11455 leader_election.cc:336] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.197734 11456 leader_election.cc:336] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.198158 11456 leader_election.cc:304] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.198908 11864 raft_consensus.cc:2749] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:55.235946 11864 raft_consensus.cc:493] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:55.236634 11864 raft_consensus.cc:515] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.239017 11864 leader_election.cc:290] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.244343 11455 leader_election.cc:336] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.245676 11456 leader_election.cc:336] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.246116 11456 leader_election.cc:304] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.246889 11864 raft_consensus.cc:2749] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:55.459909 11455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111) [suppressed 132 similar messages]
I20260709 12:22:55.488072 11864 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:55.488641 11864 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.491729 11864 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.494653 11455 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.496003 11456 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.496407 11456 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.496977 11864 raft_consensus.cc:2749] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:55.538369 30820 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 11438)
I20260709 12:22:55.539153 11864 raft_consensus.cc:493] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:55.539690 11864 raft_consensus.cc:515] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.541549 11864 leader_election.cc:290] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.545579 11456 leader_election.cc:336] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:55.546157 11455 leader_election.cc:336] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:55.546615 11455 leader_election.cc:304] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.547338 11864 raft_consensus.cc:2749] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:55.575145 11864 raft_consensus.cc:493] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:55.575646 11864 raft_consensus.cc:515] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.577404 11864 leader_election.cc:290] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.581425 11456 leader_election.cc:336] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:55.581920 11455 leader_election.cc:336] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:55.582288 11455 leader_election.cc:304] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.582914 11864 raft_consensus.cc:2749] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:55.678644 11864 raft_consensus.cc:493] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:55.679275 11864 raft_consensus.cc:515] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.681659 11864 leader_election.cc:290] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.685359 11455 leader_election.cc:336] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.686865 11456 leader_election.cc:336] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.687218 11456 leader_election.cc:304] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.687793 11864 raft_consensus.cc:2749] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:55.723142 11864 raft_consensus.cc:493] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:55.723798 11864 raft_consensus.cc:515] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.726441 11864 leader_election.cc:290] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.730011 11455 leader_election.cc:336] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.731036 11456 leader_election.cc:336] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.731474 11456 leader_election.cc:304] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.732218 11864 raft_consensus.cc:2749] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:55.781154 11864 raft_consensus.cc:493] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:55.781698 11864 raft_consensus.cc:515] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.783722 11864 leader_election.cc:290] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.787894 11455 leader_election.cc:336] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.788385 11456 leader_election.cc:336] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.788847 11456 leader_election.cc:304] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.789624 11864 raft_consensus.cc:2749] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:55.913832 11864 raft_consensus.cc:493] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:55.914412 11864 raft_consensus.cc:515] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:55.917222 11864 leader_election.cc:290] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:55.920081 11455 leader_election.cc:336] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:55.920616 11456 leader_election.cc:336] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:55.921119 11456 leader_election.cc:304] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:55.921787 11864 raft_consensus.cc:2749] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:56.089057 11864 raft_consensus.cc:493] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 408b52ec62254d8e99818a554befd6e3)
I20260709 12:22:56.089650 11864 raft_consensus.cc:515] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:56.091497 11864 leader_election.cc:290] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:56.095289 11455 leader_election.cc:336] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
W20260709 12:22:56.095746 11456 leader_election.cc:336] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
I20260709 12:22:56.096102 11456 leader_election.cc:304] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:56.096673 11864 raft_consensus.cc:2749] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:56.140108 11455 consensus_peers.cc:597] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:56.317678 11455 consensus_peers.cc:597] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:56.432643 11455 consensus_peers.cc:597] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:56.538985 30820 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 11438)
W20260709 12:22:56.570072 11455 consensus_peers.cc:597] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:56.944128 11455 consensus_peers.cc:597] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:56.955144 11455 consensus_peers.cc:597] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Couldn't send request to peer 408b52ec62254d8e99818a554befd6e3. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:57.074589 11865 raft_consensus.cc:493] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7be56f2c081b41b6939235ccfd24abb1)
I20260709 12:22:57.075456 11865 raft_consensus.cc:515] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408b52ec62254d8e99818a554befd6e3" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 43947 } } peers { permanent_uuid: "7be56f2c081b41b6939235ccfd24abb1" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 46599 } } peers { permanent_uuid: "4abc425940dd4061af4d588e0cf78ba2" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 34613 } }
I20260709 12:22:57.079221 11865 leader_election.cc:290] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947), 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599)
W20260709 12:22:57.085261 11456 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111)
W20260709 12:22:57.088600 11456 consensus_peers.cc:597] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 -> Peer 7be56f2c081b41b6939235ccfd24abb1 (127.30.25.2:46599): Couldn't send request to peer 7be56f2c081b41b6939235ccfd24abb1. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:46599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:57.089815 11455 leader_election.cc:336] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 408b52ec62254d8e99818a554befd6e3 (127.30.25.1:43947): Network error: Client connection negotiation failed: client connection to 127.30.25.1:43947: connect: Connection refused (error 111)
I20260709 12:22:57.090423 11455 leader_election.cc:304] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abc425940dd4061af4d588e0cf78ba2; no voters: 408b52ec62254d8e99818a554befd6e3, 7be56f2c081b41b6939235ccfd24abb1
I20260709 12:22:57.091221 11866 raft_consensus.cc:2749] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:57.180622 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
I20260709 12:22:57.224282 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
I20260709 12:22:57.261600 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:57.262398 30820 tablet_replica.cc:333] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.263239 30820 raft_consensus.cc:2243] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.263878 30820 raft_consensus.cc:2272] T 5edce2f4af3249e3b06474994cd321d0 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.266412 30820 tablet_replica.cc:333] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.267078 30820 raft_consensus.cc:2243] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.268267 30820 raft_consensus.cc:2272] T a7b992a1a0614882a13890e523e3e1a3 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.271270 30820 tablet_replica.cc:333] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.271853 30820 raft_consensus.cc:2243] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.272686 30820 raft_consensus.cc:2272] T d0d7525a305245759b35f92da7a5dfc7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.274993 30820 tablet_replica.cc:333] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.275550 30820 raft_consensus.cc:2243] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.276057 30820 raft_consensus.cc:2272] T 0279a578b9e047f0b044e5ca890208f7 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.278496 30820 tablet_replica.cc:333] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.279073 30820 raft_consensus.cc:2243] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.279628 30820 raft_consensus.cc:2272] T 96bdbde5464e4ec9b2ffc980a92280ae P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.281774 30820 tablet_replica.cc:333] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.282403 30820 raft_consensus.cc:2243] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.282913 30820 raft_consensus.cc:2272] T dbf243500ca743a3b8600b11b8599177 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.285197 30820 tablet_replica.cc:333] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.285760 30820 raft_consensus.cc:2243] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.286252 30820 raft_consensus.cc:2272] T e8315b03aa974b2b8cc5c155f18f2248 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.288635 30820 tablet_replica.cc:333] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.289206 30820 raft_consensus.cc:2243] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.289685 30820 raft_consensus.cc:2272] T 9a35d4770d87426aab884733ee2c3d59 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.291939 30820 tablet_replica.cc:333] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.292486 30820 raft_consensus.cc:2243] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.293347 30820 raft_consensus.cc:2272] T 7ec255b57d0c4527aae93cdfaa757e51 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.295630 30820 tablet_replica.cc:333] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.296180 30820 raft_consensus.cc:2243] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.296689 30820 raft_consensus.cc:2272] T 7dffc97ecfdd4a8db556f6c1f51cefb9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.298864 30820 tablet_replica.cc:333] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.299444 30820 raft_consensus.cc:2243] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.299898 30820 raft_consensus.cc:2272] T 1998e87dc4b14e578102b92e9bfa9f03 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.302047 30820 tablet_replica.cc:333] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.302640 30820 raft_consensus.cc:2243] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.303113 30820 raft_consensus.cc:2272] T 0bd96fbe8d2944b5859796e118975195 P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.305344 30820 tablet_replica.cc:333] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.305831 30820 raft_consensus.cc:2243] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.306695 30820 raft_consensus.cc:2272] T b0b2319fba4945e2a673d0c96fe90161 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.308781 30820 tablet_replica.cc:333] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.309273 30820 raft_consensus.cc:2243] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.309685 30820 raft_consensus.cc:2272] T 5d1eda1818ba4ecdb846e13977678111 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.311749 30820 tablet_replica.cc:333] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.312209 30820 raft_consensus.cc:2243] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.312937 30820 raft_consensus.cc:2272] T 97de4708d35e47afb683b92846175828 P 4abc425940dd4061af4d588e0cf78ba2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.315104 30820 tablet_replica.cc:333] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.315593 30820 raft_consensus.cc:2243] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.316046 30820 raft_consensus.cc:2272] T df40005c1ebe414fb797a878bae5eaf9 P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.317970 30820 tablet_replica.cc:333] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.318477 30820 raft_consensus.cc:2243] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.319254 30820 raft_consensus.cc:2272] T a8afccb6e54a4a17b883a901da0bef6b P 4abc425940dd4061af4d588e0cf78ba2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.321332 30820 tablet_replica.cc:333] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2: stopping tablet replica
I20260709 12:22:57.321754 30820 raft_consensus.cc:2243] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:57.322134 30820 raft_consensus.cc:2272] T 07af2adb5e0745088a20cdf76ead9d5c P 4abc425940dd4061af4d588e0cf78ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.366607 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
I20260709 12:22:57.418707 30820 master.cc:562] Master@127.30.25.62:43419 shutting down...
I20260709 12:22:57.447160 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:57.447805 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:57.448187 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4c3b84b09ea14fff9561745b1d7e3ac8: stopping tablet replica
I20260709 12:22:57.487449 30820 master.cc:584] Master@127.30.25.62:43419 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (27979 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:22:57.543392 30820 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.30.25.62:40149
I20260709 12:22:57.544636 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:57.551100 11868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:57.551440 11871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:57.551878 11869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:57.552711 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:57.553715 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:57.553913 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:57.554064 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599777554031 us; error 0 us; skew 500 ppm
I20260709 12:22:57.554630 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:57.556852 30820 webserver.cc:533] Webserver started at http://127.30.25.62:41853/ using document root <none> and password file <none>
I20260709 12:22:57.557336 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:57.557521 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:57.557788 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:57.558955 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/master-0-root/instance:
uuid: "20b8427267024bf089c30070b05a671b"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.571561 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.012s	user 0.005s	sys 0.008s
I20260709 12:22:57.576462 11876 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.577168 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:22:57.577499 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/master-0-root
uuid: "20b8427267024bf089c30070b05a671b"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.577766 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:57.602995 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:57.604259 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:57.642448 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.62:40149
I20260709 12:22:57.642518 11927 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.62:40149 every 8 connection(s)
I20260709 12:22:57.646543 11928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:57.658535 11928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b: Bootstrap starting.
I20260709 12:22:57.663409 11928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:57.667587 11928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b: No bootstrap required, opened a new log
I20260709 12:22:57.669708 11928 raft_consensus.cc:359] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b8427267024bf089c30070b05a671b" member_type: VOTER }
I20260709 12:22:57.670079 11928 raft_consensus.cc:385] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:57.670257 11928 raft_consensus.cc:740] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b8427267024bf089c30070b05a671b, State: Initialized, Role: FOLLOWER
I20260709 12:22:57.670873 11928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b8427267024bf089c30070b05a671b" member_type: VOTER }
I20260709 12:22:57.671303 11928 raft_consensus.cc:399] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:57.671494 11928 raft_consensus.cc:493] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:57.671694 11928 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:57.676126 11928 raft_consensus.cc:515] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b8427267024bf089c30070b05a671b" member_type: VOTER }
I20260709 12:22:57.676564 11928 leader_election.cc:304] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20b8427267024bf089c30070b05a671b; no voters: 
I20260709 12:22:57.677690 11928 leader_election.cc:290] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:57.678092 11931 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:57.679507 11931 raft_consensus.cc:697] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 1 LEADER]: Becoming Leader. State: Replica: 20b8427267024bf089c30070b05a671b, State: Running, Role: LEADER
I20260709 12:22:57.680272 11931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b8427267024bf089c30070b05a671b" member_type: VOTER }
I20260709 12:22:57.681180 11928 sys_catalog.cc:565] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:57.687080 11932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20b8427267024bf089c30070b05a671b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b8427267024bf089c30070b05a671b" member_type: VOTER } }
I20260709 12:22:57.687326 11933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20b8427267024bf089c30070b05a671b. Latest consensus state: current_term: 1 leader_uuid: "20b8427267024bf089c30070b05a671b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b8427267024bf089c30070b05a671b" member_type: VOTER } }
I20260709 12:22:57.688093 11932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:57.688270 11933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:57.690577 11937 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:57.698720 11937 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:57.700241 30820 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:57.708034 11937 catalog_manager.cc:1383] Generated new cluster ID: d27280b3e9ce4b70aceaea03697e37c9
I20260709 12:22:57.708338 11937 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:57.741513 11937 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:57.742872 11937 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:57.755327 11937 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b: Generated new TSK 0
I20260709 12:22:57.755954 11937 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:57.767280 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:57.773353 11949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:57.774431 11950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:57.777011 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:57.777158 11952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:57.778466 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:57.778661 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:57.778793 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599777778782 us; error 0 us; skew 500 ppm
I20260709 12:22:57.779312 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:57.781594 30820 webserver.cc:533] Webserver started at http://127.30.25.1:41109/ using document root <none> and password file <none>
I20260709 12:22:57.782054 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:57.782207 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:57.782508 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:57.783720 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-0-root/instance:
uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.788245 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:22:57.791726 11957 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.792524 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:57.792829 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-0-root
uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.793118 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:57.815454 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:57.816676 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:57.818727 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:57.821115 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:57.821336 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.821583 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:57.821779 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.862654 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.1:36969
I20260709 12:22:57.862783 12019 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.1:36969 every 8 connection(s)
I20260709 12:22:57.867365 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:57.875398 12024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:57.875756 12025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:57.879079 12027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:57.879997 30820 server_base.cc:1061] running on GCE node
I20260709 12:22:57.881448 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:57.881672 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:57.881886 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599777881860 us; error 0 us; skew 500 ppm
I20260709 12:22:57.882628 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:57.882544 12020 heartbeater.cc:344] Connected to a master server at 127.30.25.62:40149
I20260709 12:22:57.883215 12020 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:57.884001 12020 heartbeater.cc:507] Master 127.30.25.62:40149 requested a full tablet report, sending...
I20260709 12:22:57.885846 30820 webserver.cc:533] Webserver started at http://127.30.25.2:33545/ using document root <none> and password file <none>
I20260709 12:22:57.886372 11893 ts_manager.cc:194] Registered new tserver with Master: 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969)
I20260709 12:22:57.886617 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:57.886950 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:57.887288 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:57.888876 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-1-root/instance:
uuid: "a958846336274bb386baea0d38600e01"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.888962 11893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60606
I20260709 12:22:57.893810 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:22:57.897284 12032 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.898021 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:57.898290 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-1-root
uuid: "a958846336274bb386baea0d38600e01"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.898617 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:57.911630 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:57.912696 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:57.914103 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:57.916391 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:57.916594 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.916872 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:57.917054 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.957839 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.2:44995
I20260709 12:22:57.957944 12094 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.2:44995 every 8 connection(s)
I20260709 12:22:57.963073 30820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:57.970216 12098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:57.971649 12099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:57.974763 30820 server_base.cc:1061] running on GCE node
W20260709 12:22:57.975410 12101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:57.975591 12095 heartbeater.cc:344] Connected to a master server at 127.30.25.62:40149
I20260709 12:22:57.976150 12095 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:57.976665 30820 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:57.976984 30820 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:57.977072 12095 heartbeater.cc:507] Master 127.30.25.62:40149 requested a full tablet report, sending...
I20260709 12:22:57.977222 30820 hybrid_clock.cc:648] HybridClock initialized: now 1783599777977200 us; error 0 us; skew 500 ppm
I20260709 12:22:57.978083 30820 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:57.979493 11893 ts_manager.cc:194] Registered new tserver with Master: a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:57.980867 30820 webserver.cc:533] Webserver started at http://127.30.25.3:35799/ using document root <none> and password file <none>
I20260709 12:22:57.981189 11893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60618
I20260709 12:22:57.981437 30820 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:57.981760 30820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:57.982061 30820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:57.983693 30820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-2-root/instance:
uuid: "91885dd1fdd54c928ae934cbe377559a"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.988255 30820 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:57.991672 12106 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:57.992465 30820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:22:57.992776 30820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-2-root
uuid: "91885dd1fdd54c928ae934cbe377559a"
format_stamp: "Formatted at 2026-07-09 12:22:57 on dist-test-slave-rldg"
I20260709 12:22:57.993104 30820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskg6CpM7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599338705961-30820-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:58.020547 30820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:58.021814 30820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:58.023419 30820 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:58.025676 30820 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:58.025918 30820 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:58.026165 30820 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:58.026356 30820 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:58.066145 30820 rpc_server.cc:307] RPC server started. Bound to: 127.30.25.3:44897
I20260709 12:22:58.066334 12168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.30.25.3:44897 every 8 connection(s)
I20260709 12:22:58.080188 12169 heartbeater.cc:344] Connected to a master server at 127.30.25.62:40149
I20260709 12:22:58.080698 12169 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:58.081425 12169 heartbeater.cc:507] Master 127.30.25.62:40149 requested a full tablet report, sending...
I20260709 12:22:58.083571 11893 ts_manager.cc:194] Registered new tserver with Master: 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.084338 30820 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014869381s
I20260709 12:22:58.085268 11893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60632
I20260709 12:22:58.110183 11893 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60640:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:58.112950 11893 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:58.202791 12060 tablet_service.cc:1511] Processing CreateTablet for tablet 516cbcf53f6f4a288926bbfb38ee34b0 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:58.203164 12059 tablet_service.cc:1511] Processing CreateTablet for tablet 9795bed32fab40ff939e0c11afb1f0ff (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:58.204336 12060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516cbcf53f6f4a288926bbfb38ee34b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.205386 12058 tablet_service.cc:1511] Processing CreateTablet for tablet ede410d0ce4646889e6f17c8c742cc46 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:58.205893 12054 tablet_service.cc:1511] Processing CreateTablet for tablet 215c4dd47fc64dd4ae6dfad696c471da (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:58.206852 12058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ede410d0ce4646889e6f17c8c742cc46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.207072 12056 tablet_service.cc:1511] Processing CreateTablet for tablet ae0d804930c3497da9c71c272591e6b8 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:58.208461 12056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae0d804930c3497da9c71c272591e6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.208366 12055 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0995365e4a4b878cb2b16803b7f0c5 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:58.209829 12055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0995365e4a4b878cb2b16803b7f0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.212105 12059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9795bed32fab40ff939e0c11afb1f0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.205893 12057 tablet_service.cc:1511] Processing CreateTablet for tablet a8d39c2cef1e4d8dbe8b750de8c7f9e2 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:58.215793 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8d39c2cef1e4d8dbe8b750de8c7f9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.218719 12054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215c4dd47fc64dd4ae6dfad696c471da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.229518 11985 tablet_service.cc:1511] Processing CreateTablet for tablet 516cbcf53f6f4a288926bbfb38ee34b0 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:58.230994 11981 tablet_service.cc:1511] Processing CreateTablet for tablet ae0d804930c3497da9c71c272591e6b8 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:58.231532 12053 tablet_service.cc:1511] Processing CreateTablet for tablet 543a0c698b8644dd945607edb988a1d1 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:58.232244 11984 tablet_service.cc:1511] Processing CreateTablet for tablet 9795bed32fab40ff939e0c11afb1f0ff (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:58.232982 12053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543a0c698b8644dd945607edb988a1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.233450 11980 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0995365e4a4b878cb2b16803b7f0c5 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:58.234656 11983 tablet_service.cc:1511] Processing CreateTablet for tablet ede410d0ce4646889e6f17c8c742cc46 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:58.235092 11977 tablet_service.cc:1511] Processing CreateTablet for tablet c5ee1ff107b34cae9b04c12178909492 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:58.235788 11979 tablet_service.cc:1511] Processing CreateTablet for tablet 215c4dd47fc64dd4ae6dfad696c471da (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:58.237008 11982 tablet_service.cc:1511] Processing CreateTablet for tablet a8d39c2cef1e4d8dbe8b750de8c7f9e2 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:58.231071 11985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516cbcf53f6f4a288926bbfb38ee34b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.238149 11978 tablet_service.cc:1511] Processing CreateTablet for tablet 543a0c698b8644dd945607edb988a1d1 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:58.238823 12052 tablet_service.cc:1511] Processing CreateTablet for tablet c5ee1ff107b34cae9b04c12178909492 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:58.239508 11978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543a0c698b8644dd945607edb988a1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.240141 12052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ee1ff107b34cae9b04c12178909492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.240213 11984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9795bed32fab40ff939e0c11afb1f0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.242705 11982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8d39c2cef1e4d8dbe8b750de8c7f9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.246652 11980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0995365e4a4b878cb2b16803b7f0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.250674 11979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215c4dd47fc64dd4ae6dfad696c471da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.251889 11981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae0d804930c3497da9c71c272591e6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.260582 11977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ee1ff107b34cae9b04c12178909492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.261426 12129 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0995365e4a4b878cb2b16803b7f0c5 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:58.262912 12129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0995365e4a4b878cb2b16803b7f0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.263216 12128 tablet_service.cc:1511] Processing CreateTablet for tablet 215c4dd47fc64dd4ae6dfad696c471da (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:58.264595 12128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215c4dd47fc64dd4ae6dfad696c471da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.267290 12133 tablet_service.cc:1511] Processing CreateTablet for tablet 9795bed32fab40ff939e0c11afb1f0ff (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:58.268671 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9795bed32fab40ff939e0c11afb1f0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.273747 12132 tablet_service.cc:1511] Processing CreateTablet for tablet ede410d0ce4646889e6f17c8c742cc46 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:58.275218 12132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ede410d0ce4646889e6f17c8c742cc46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.279476 12130 tablet_service.cc:1511] Processing CreateTablet for tablet ae0d804930c3497da9c71c272591e6b8 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:58.261294 12127 tablet_service.cc:1511] Processing CreateTablet for tablet 543a0c698b8644dd945607edb988a1d1 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:58.284066 12131 tablet_service.cc:1511] Processing CreateTablet for tablet a8d39c2cef1e4d8dbe8b750de8c7f9e2 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:58.285027 12134 tablet_service.cc:1511] Processing CreateTablet for tablet 516cbcf53f6f4a288926bbfb38ee34b0 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:58.282277 12130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae0d804930c3497da9c71c272591e6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.286108 11983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ede410d0ce4646889e6f17c8c742cc46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.294484 12134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516cbcf53f6f4a288926bbfb38ee34b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.298972 12127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543a0c698b8644dd945607edb988a1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.302996 12131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8d39c2cef1e4d8dbe8b750de8c7f9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.364717 12126 tablet_service.cc:1511] Processing CreateTablet for tablet c5ee1ff107b34cae9b04c12178909492 (DEFAULT_TABLE table=test-workload [id=e34377fd341f45cb84345b05a043c367]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:58.367913 12126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ee1ff107b34cae9b04c12178909492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:58.407644 12184 tablet_bootstrap.cc:492] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.412806 12186 tablet_bootstrap.cc:492] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.422202 12184 tablet_bootstrap.cc:654] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.423264 12185 tablet_bootstrap.cc:492] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.433187 12186 tablet_bootstrap.cc:654] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.438810 12184 tablet_bootstrap.cc:492] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.439316 12184 ts_tablet_manager.cc:1403] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.015s
I20260709 12:22:58.440904 12185 tablet_bootstrap.cc:654] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.442252 12184 raft_consensus.cc:359] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.443106 12184 raft_consensus.cc:385] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.443413 12184 raft_consensus.cc:740] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.444360 12184 consensus_queue.cc:260] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.445214 12186 tablet_bootstrap.cc:492] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:58.445837 12186 ts_tablet_manager.cc:1403] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.009s
I20260709 12:22:58.445864 12185 tablet_bootstrap.cc:492] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.446663 12185 ts_tablet_manager.cc:1403] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 12:22:58.446946 12095 heartbeater.cc:499] Master 127.30.25.62:40149 was elected leader, sending a full tablet report...
I20260709 12:22:58.447435 12184 ts_tablet_manager.cc:1434] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 12:22:58.448518 12184 tablet_bootstrap.cc:492] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.449136 12186 raft_consensus.cc:359] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.449908 12186 raft_consensus.cc:385] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.449558 12185 raft_consensus.cc:359] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.450225 12186 raft_consensus.cc:740] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.450403 12185 raft_consensus.cc:385] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.450770 12185 raft_consensus.cc:740] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.451128 12186 consensus_queue.cc:260] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.451792 12185 consensus_queue.cc:260] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.456147 12184 tablet_bootstrap.cc:654] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.461365 12169 heartbeater.cc:499] Master 127.30.25.62:40149 was elected leader, sending a full tablet report...
I20260709 12:22:58.461903 12186 ts_tablet_manager.cc:1434] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.005s
W20260709 12:22:58.462198 12096 tablet.cc:2406] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:58.462935 12186 tablet_bootstrap.cc:492] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.468349 12190 raft_consensus.cc:493] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.469538 12186 tablet_bootstrap.cc:654] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.469074 12190 raft_consensus.cc:515] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.492400 12184 tablet_bootstrap.cc:492] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.493188 12184 ts_tablet_manager.cc:1403] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.006s
I20260709 12:22:58.498133 12184 raft_consensus.cc:359] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.499245 12184 raft_consensus.cc:385] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.499655 12184 raft_consensus.cc:740] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.500656 12184 consensus_queue.cc:260] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.503063 12190 leader_election.cc:290] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.503757 12184 ts_tablet_manager.cc:1434] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 12:22:58.508157 12184 tablet_bootstrap.cc:492] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.516933 12184 tablet_bootstrap.cc:654] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.556629 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516cbcf53f6f4a288926bbfb38ee34b0" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
W20260709 12:22:58.558996 12035 leader_election.cc:343] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.565835 12186 tablet_bootstrap.cc:492] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:58.566308 12186 ts_tablet_manager.cc:1403] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.104s	user 0.032s	sys 0.067s
I20260709 12:22:58.566880 12184 tablet_bootstrap.cc:492] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.567443 12184 ts_tablet_manager.cc:1403] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.060s	user 0.026s	sys 0.032s
I20260709 12:22:58.568588 12186 raft_consensus.cc:359] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.569388 12186 raft_consensus.cc:385] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.569674 12186 raft_consensus.cc:740] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.570271 12185 ts_tablet_manager.cc:1434] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.123s	user 0.049s	sys 0.064s
I20260709 12:22:58.570647 12186 consensus_queue.cc:260] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.571249 12020 heartbeater.cc:499] Master 127.30.25.62:40149 was elected leader, sending a full tablet report...
I20260709 12:22:58.571485 12185 tablet_bootstrap.cc:492] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.575186 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516cbcf53f6f4a288926bbfb38ee34b0" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
W20260709 12:22:58.576936 12035 leader_election.cc:343] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.577394 12035 leader_election.cc:304] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:58.578598 12190 raft_consensus.cc:2749] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:58.578962 12185 tablet_bootstrap.cc:654] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.580297 12186 ts_tablet_manager.cc:1434] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 12:22:58.580245 12184 raft_consensus.cc:359] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.581295 12184 raft_consensus.cc:385] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.581660 12184 raft_consensus.cc:740] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.582612 12184 consensus_queue.cc:260] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.586053 12191 raft_consensus.cc:493] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.586838 12191 raft_consensus.cc:515] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.589722 12190 raft_consensus.cc:493] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.590271 12190 raft_consensus.cc:515] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.592402 12190 leader_election.cc:290] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.594281 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:58.596088 12192 raft_consensus.cc:493] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.596774 12192 raft_consensus.cc:515] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.597535 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
W20260709 12:22:58.599148 12035 leader_election.cc:343] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:58.600226 12035 leader_election.cc:343] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.600653 12035 leader_election.cc:304] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:58.603394 12190 raft_consensus.cc:2749] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:58.603596 12185 tablet_bootstrap.cc:492] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.604317 12192 leader_election.cc:290] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.604540 12185 ts_tablet_manager.cc:1403] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.009s
I20260709 12:22:58.595197 12191 leader_election.cc:290] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:58.607921 12185 raft_consensus.cc:359] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.608614 12185 raft_consensus.cc:385] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.608901 12185 raft_consensus.cc:740] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.610220 12186 tablet_bootstrap.cc:492] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.609637 12185 consensus_queue.cc:260] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.613073 12185 ts_tablet_manager.cc:1434] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:22:58.614593 12185 tablet_bootstrap.cc:492] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.613546 12184 ts_tablet_manager.cc:1434] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.003s
I20260709 12:22:58.621157 12185 tablet_bootstrap.cc:654] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.624037 12184 tablet_bootstrap.cc:492] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.627565 12186 tablet_bootstrap.cc:654] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.633448 12184 tablet_bootstrap.cc:654] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.636027 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.636837 12070 raft_consensus.cc:2468] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:22:58.638234 11960 leader_election.cc:304] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:58.639259 12192 raft_consensus.cc:2804] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.639650 12192 raft_consensus.cc:493] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.640003 12192 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.643404 12185 tablet_bootstrap.cc:492] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.644268 12185 ts_tablet_manager.cc:1403] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.019s
I20260709 12:22:58.648015 12192 raft_consensus.cc:515] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.648594 12185 raft_consensus.cc:359] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.649813 12185 raft_consensus.cc:385] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.650296 12185 raft_consensus.cc:740] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.651139 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:58.651786 12070 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.651634 12185 consensus_queue.cc:260] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.659711 12070 raft_consensus.cc:2468] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:22:58.660944 11960 leader_election.cc:304] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:58.661227 12192 leader_election.cc:290] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.663332 12185 ts_tablet_manager.cc:1434] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:22:58.663919 12192 raft_consensus.cc:2804] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.664557 12192 raft_consensus.cc:697] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:58.668732 12185 tablet_bootstrap.cc:492] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.669747 12186 tablet_bootstrap.cc:492] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:58.684065 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215c4dd47fc64dd4ae6dfad696c471da" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.679082 12185 tablet_bootstrap.cc:654] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.684345 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215c4dd47fc64dd4ae6dfad696c471da" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
W20260709 12:22:58.687095 12109 leader_election.cc:343] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.665740 12192 consensus_queue.cc:237] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
W20260709 12:22:58.688179 12109 leader_election.cc:343] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.683532 12186 ts_tablet_manager.cc:1403] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.000s
I20260709 12:22:58.692207 12109 leader_election.cc:304] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01
I20260709 12:22:58.693363 12191 raft_consensus.cc:2749] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:22:58.695597 11941 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:58.699223 12186 raft_consensus.cc:359] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.701473 12186 raft_consensus.cc:385] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.701833 12186 raft_consensus.cc:740] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.702646 12186 consensus_queue.cc:260] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.703126 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:58.704607 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:58.705549 12208 raft_consensus.cc:493] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:58.707337 11960 leader_election.cc:343] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.707103 12208 raft_consensus.cc:515] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
W20260709 12:22:58.708563 11960 leader_election.cc:343] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.710379 12208 leader_election.cc:290] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.704524 12184 tablet_bootstrap.cc:492] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.711499 12184 ts_tablet_manager.cc:1403] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.088s	user 0.037s	sys 0.037s
I20260709 12:22:58.712797 11892 catalog_manager.cc:5697] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:58.717036 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:58.717036 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.718242 12184 raft_consensus.cc:359] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.719038 12185 tablet_bootstrap.cc:492] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.719048 12186 ts_tablet_manager.cc:1434] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
W20260709 12:22:58.719640 11960 leader_election.cc:343] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.719820 12185 ts_tablet_manager.cc:1403] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.024s
I20260709 12:22:58.719075 12184 raft_consensus.cc:385] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:22:58.720583 11960 leader_election.cc:343] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.721081 11960 leader_election.cc:304] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05; no voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01
I20260709 12:22:58.720608 12184 raft_consensus.cc:740] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.722357 12208 raft_consensus.cc:2749] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:58.723073 12184 consensus_queue.cc:260] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.725020 12185 raft_consensus.cc:359] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.725615 12184 ts_tablet_manager.cc:1434] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.005s
I20260709 12:22:58.725868 12185 raft_consensus.cc:385] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.726366 12185 raft_consensus.cc:740] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.726773 12184 tablet_bootstrap.cc:492] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.727311 12186 tablet_bootstrap.cc:492] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.727236 12185 consensus_queue.cc:260] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.730544 12185 ts_tablet_manager.cc:1434] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:22:58.731537 12185 tablet_bootstrap.cc:492] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.734046 12184 tablet_bootstrap.cc:654] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.734938 12186 tablet_bootstrap.cc:654] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.737978 12185 tablet_bootstrap.cc:654] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.741200 12184 tablet_bootstrap.cc:492] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.741847 12184 ts_tablet_manager.cc:1403] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.005s
I20260709 12:22:58.741968 12186 tablet_bootstrap.cc:492] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:58.742640 12186 ts_tablet_manager.cc:1403] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 12:22:58.745993 12184 raft_consensus.cc:359] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.746804 12186 raft_consensus.cc:359] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.747269 12184 raft_consensus.cc:385] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.747835 12186 raft_consensus.cc:385] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.747980 12184 raft_consensus.cc:740] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.748518 12186 raft_consensus.cc:740] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.749485 12184 consensus_queue.cc:260] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.749974 12186 consensus_queue.cc:260] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.756654 12186 ts_tablet_manager.cc:1434] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:22:58.757817 12186 tablet_bootstrap.cc:492] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.757973 12184 ts_tablet_manager.cc:1434] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:22:58.759595 12184 tablet_bootstrap.cc:492] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.765261 12186 tablet_bootstrap.cc:654] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.765286 12185 tablet_bootstrap.cc:492] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.767025 12184 tablet_bootstrap.cc:654] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.767305 12185 ts_tablet_manager.cc:1403] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.019s
I20260709 12:22:58.771498 12185 raft_consensus.cc:359] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.772563 12185 raft_consensus.cc:385] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.772976 12185 raft_consensus.cc:740] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.773980 12185 consensus_queue.cc:260] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.777576 12185 ts_tablet_manager.cc:1434] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:22:58.779115 12185 tablet_bootstrap.cc:492] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.786919 12191 raft_consensus.cc:493] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.787554 12191 raft_consensus.cc:515] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.788173 12185 tablet_bootstrap.cc:654] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.789811 12191 leader_election.cc:290] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:58.792275 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.793242 12070 raft_consensus.cc:2468] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 0.
I20260709 12:22:58.795239 12208 raft_consensus.cc:493] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.795961 12208 raft_consensus.cc:515] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.798890 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:58.800875 12208 leader_election.cc:290] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
W20260709 12:22:58.804126 12109 leader_election.cc:343] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.805137 12109 leader_election.cc:304] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:22:58.806529 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d39c2cef1e4d8dbe8b750de8c7f9e2" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
W20260709 12:22:58.808588 11960 leader_election.cc:343] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.810627 12191 raft_consensus.cc:2804] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.811669 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d39c2cef1e4d8dbe8b750de8c7f9e2" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:58.812755 12143 raft_consensus.cc:2468] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:22:58.814639 11960 leader_election.cc:304] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: a958846336274bb386baea0d38600e01
I20260709 12:22:58.815913 12185 tablet_bootstrap.cc:492] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.816387 12191 raft_consensus.cc:493] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.816599 12185 ts_tablet_manager.cc:1403] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.017s
I20260709 12:22:58.816941 12208 raft_consensus.cc:2804] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.817538 12191 raft_consensus.cc:3060] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.817780 12208 raft_consensus.cc:493] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.818190 12208 raft_consensus.cc:3060] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.821130 12184 tablet_bootstrap.cc:492] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.821739 12184 ts_tablet_manager.cc:1403] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.062s	user 0.033s	sys 0.020s
I20260709 12:22:58.824213 12185 raft_consensus.cc:359] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.825083 12185 raft_consensus.cc:385] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.824751 12184 raft_consensus.cc:359] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.821134 12186 tablet_bootstrap.cc:492] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:58.825435 12185 raft_consensus.cc:740] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.825834 12208 raft_consensus.cc:515] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.826333 12186 ts_tablet_manager.cc:1403] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.020s
I20260709 12:22:58.826607 12184 raft_consensus.cc:385] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.827072 12184 raft_consensus.cc:740] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.826637 12185 consensus_queue.cc:260] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.827883 12184 consensus_queue.cc:260] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.828298 12191 raft_consensus.cc:515] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.829883 12186 raft_consensus.cc:359] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.830739 12186 raft_consensus.cc:385] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.831073 12186 raft_consensus.cc:740] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.833739 12186 consensus_queue.cc:260] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.834779 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d39c2cef1e4d8dbe8b750de8c7f9e2" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:58.835078 12185 ts_tablet_manager.cc:1434] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:22:58.831365 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:58.836210 12185 tablet_bootstrap.cc:492] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.839242 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:58.840215 12069 raft_consensus.cc:3060] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.841128 12184 ts_tablet_manager.cc:1434] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 12:22:58.842675 12184 tablet_bootstrap.cc:492] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.842808 12185 tablet_bootstrap.cc:654] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:58.843092 11960 leader_election.cc:343] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:58.840960 12109 leader_election.cc:343] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.845091 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d39c2cef1e4d8dbe8b750de8c7f9e2" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:58.845887 12143 raft_consensus.cc:3060] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.832814 12208 leader_election.cc:290] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.850728 12191 leader_election.cc:290] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:58.851349 12184 tablet_bootstrap.cc:654] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.856326 12185 tablet_bootstrap.cc:492] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.856842 12185 ts_tablet_manager.cc:1403] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.005s
I20260709 12:22:58.859458 12190 raft_consensus.cc:493] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.859941 12208 raft_consensus.cc:493] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.860323 12186 ts_tablet_manager.cc:1434] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.018s
I20260709 12:22:58.859990 12190 raft_consensus.cc:515] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.860540 12208 raft_consensus.cc:515] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.863029 12208 leader_election.cc:290] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.863222 12186 tablet_bootstrap.cc:492] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.864825 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516cbcf53f6f4a288926bbfb38ee34b0" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.865352 12190 leader_election.cc:290] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.865535 12070 raft_consensus.cc:2468] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:22:58.859995 12185 raft_consensus.cc:359] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.866704 12185 raft_consensus.cc:385] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.867000 12185 raft_consensus.cc:740] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.867460 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:58.867811 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:58.868125 11995 raft_consensus.cc:2468] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:22:58.868889 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516cbcf53f6f4a288926bbfb38ee34b0" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
W20260709 12:22:58.869535 12035 leader_election.cc:343] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.870244 12035 leader_election.cc:304] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:58.871124 12190 raft_consensus.cc:2804] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.871500 12190 raft_consensus.cc:493] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.871748 12220 raft_consensus.cc:493] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.871928 12190 raft_consensus.cc:3060] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.872300 12220 raft_consensus.cc:515] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.874615 12220 leader_election.cc:290] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:58.875310 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:58.877991 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.878789 12070 raft_consensus.cc:2393] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91885dd1fdd54c928ae934cbe377559a in current term 1: Already voted for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in this term.
I20260709 12:22:58.870172 11960 leader_election.cc:304] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:58.879889 12192 raft_consensus.cc:2804] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.879920 12184 tablet_bootstrap.cc:492] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.880447 12192 raft_consensus.cc:493] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.880767 12184 ts_tablet_manager.cc:1403] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.005s
W20260709 12:22:58.880012 11960 leader_election.cc:343] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.881054 12192 raft_consensus.cc:3060] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.884055 12184 raft_consensus.cc:359] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.884743 12184 raft_consensus.cc:385] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.885026 12184 raft_consensus.cc:740] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.885238 12143 raft_consensus.cc:2468] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:22:58.885849 12069 raft_consensus.cc:2468] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 1.
I20260709 12:22:58.880358 12109 leader_election.cc:304] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01
I20260709 12:22:58.878911 12186 tablet_bootstrap.cc:654] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.885919 12184 consensus_queue.cc:260] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.887454 11960 leader_election.cc:304] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: a958846336274bb386baea0d38600e01
I20260709 12:22:58.867794 12185 consensus_queue.cc:260] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.888520 12208 raft_consensus.cc:2804] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.888356 12192 raft_consensus.cc:515] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.887467 12220 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.889921 12208 raft_consensus.cc:697] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:58.890849 12208 consensus_queue.cc:237] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.891271 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516cbcf53f6f4a288926bbfb38ee34b0" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:58.891906 12070 raft_consensus.cc:3060] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.891862 12184 ts_tablet_manager.cc:1434] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:22:58.893806 12184 tablet_bootstrap.cc:492] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.897728 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516cbcf53f6f4a288926bbfb38ee34b0" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:58.898444 12185 ts_tablet_manager.cc:1434] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.001s
W20260709 12:22:58.899053 11960 leader_election.cc:343] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.899524 12185 tablet_bootstrap.cc:492] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:58.899739 12220 raft_consensus.cc:2749] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:58.902587 12109 leader_election.cc:304] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:22:58.904211 12191 raft_consensus.cc:2804] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.906569 12186 tablet_bootstrap.cc:492] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:58.907004 12070 raft_consensus.cc:2468] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:22:58.907344 12186 ts_tablet_manager.cc:1403] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.000s
I20260709 12:22:58.908120 12184 tablet_bootstrap.cc:654] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.908241 11960 leader_election.cc:304] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:58.909245 12207 raft_consensus.cc:2804] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:58.909654 12192 leader_election.cc:290] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.911616 12186 raft_consensus.cc:359] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.912633 12186 raft_consensus.cc:385] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.913059 12186 raft_consensus.cc:740] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.917303 12186 consensus_queue.cc:260] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.909440 12190 raft_consensus.cc:515] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.925068 12184 tablet_bootstrap.cc:492] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:58.925482 12190 leader_election.cc:290] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.920468 12185 tablet_bootstrap.cc:654] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.925892 12184 ts_tablet_manager.cc:1403] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.032s	user 0.022s	sys 0.001s
I20260709 12:22:58.928218 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:58.929087 11995 raft_consensus.cc:3060] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.929466 12207 raft_consensus.cc:697] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:58.930584 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:58.931411 12143 raft_consensus.cc:3060] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.931173 12207 consensus_queue.cc:237] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.920405 12191 raft_consensus.cc:697] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Becoming Leader. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:22:58.929440 12208 raft_consensus.cc:493] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.940666 12184 raft_consensus.cc:359] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.941627 12184 raft_consensus.cc:385] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.941901 12184 raft_consensus.cc:740] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.938484 12208 raft_consensus.cc:515] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.942615 12225 raft_consensus.cc:493] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.942618 12184 consensus_queue.cc:260] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.943507 12225 raft_consensus.cc:515] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.944007 12186 ts_tablet_manager.cc:1434] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.036s	user 0.013s	sys 0.011s
I20260709 12:22:58.945096 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.945986 12225 leader_election.cc:290] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
W20260709 12:22:58.946540 11960 leader_election.cc:343] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.946925 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:58.947655 11994 raft_consensus.cc:2468] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 0.
I20260709 12:22:58.947963 12208 leader_election.cc:290] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:58.948825 12184 ts_tablet_manager.cc:1434] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 12:22:58.938645 11891 catalog_manager.cc:5697] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:58.949791 12184 tablet_bootstrap.cc:492] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:22:58.950925 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:58.951027 12191 consensus_queue.cc:237] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.951611 12142 raft_consensus.cc:2468] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:22:58.957947 11995 raft_consensus.cc:2468] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:22:58.958640 12109 leader_election.cc:304] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:22:58.958771 11960 leader_election.cc:304] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: a958846336274bb386baea0d38600e01
I20260709 12:22:58.959563 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:58.961716 12035 leader_election.cc:304] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:58.962190 12225 raft_consensus.cc:2804] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.962628 12190 raft_consensus.cc:2804] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:22:58.962903 12109 leader_election.cc:343] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:58.963027 12225 raft_consensus.cc:493] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.963826 12225 raft_consensus.cc:3060] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.960237 12186 tablet_bootstrap.cc:492] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:58.963191 12190 raft_consensus.cc:697] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:58.964792 12184 tablet_bootstrap.cc:654] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.964269 12143 raft_consensus.cc:2468] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:22:58.965161 12208 raft_consensus.cc:2804] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:58.968665 12208 raft_consensus.cc:493] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.965866 12190 consensus_queue.cc:237] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.972942 12185 tablet_bootstrap.cc:492] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:58.973438 12185 ts_tablet_manager.cc:1403] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.011s
I20260709 12:22:58.971024 12208 raft_consensus.cc:3060] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:58.989324 12186 tablet_bootstrap.cc:654] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:58.991827 12185 raft_consensus.cc:359] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:58.992874 12185 raft_consensus.cc:385] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:58.993216 12185 raft_consensus.cc:740] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:58.996860 12185 consensus_queue.cc:260] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.002717 11891 catalog_manager.cc:5697] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a reported cstate change: term changed from 0 to 1, leader changed from <none> to 91885dd1fdd54c928ae934cbe377559a (127.30.25.3). New cstate: current_term: 1 leader_uuid: "91885dd1fdd54c928ae934cbe377559a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:59.007444 12185 ts_tablet_manager.cc:1434] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.002s
I20260709 12:22:59.007416 11892 catalog_manager.cc:5697] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.008692 12185 tablet_bootstrap.cc:492] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:22:59.003545 12225 raft_consensus.cc:515] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.013288 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.009617 12208 raft_consensus.cc:515] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.016847 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:59.017287 12184 tablet_bootstrap.cc:492] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:22:59.017910 12184 ts_tablet_manager.cc:1403] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.024s
I20260709 12:22:59.016881 12225 leader_election.cc:290] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:59.020469 11995 raft_consensus.cc:2393] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91885dd1fdd54c928ae934cbe377559a in current term 1: Already voted for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in this term.
I20260709 12:22:59.021596 12184 raft_consensus.cc:359] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.022451 12184 raft_consensus.cc:385] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.022753 12184 raft_consensus.cc:740] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.023814 12207 raft_consensus.cc:493] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.023557 12184 consensus_queue.cc:260] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.024425 12207 raft_consensus.cc:515] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
W20260709 12:22:59.025097 12109 leader_election.cc:343] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.024380 11890 catalog_manager.cc:5697] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.025684 12109 leader_election.cc:304] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01
I20260709 12:22:59.025969 12184 ts_tablet_manager.cc:1434] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 12:22:59.026866 12207 leader_election.cc:290] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.027787 12185 tablet_bootstrap.cc:654] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.030442 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:59.031028 12186 tablet_bootstrap.cc:492] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:59.031878 12186 ts_tablet_manager.cc:1403] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.003s
I20260709 12:22:59.032764 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:59.033592 12070 raft_consensus.cc:2393] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in current term 1: Already voted for candidate 91885dd1fdd54c928ae934cbe377559a in this term.
I20260709 12:22:59.034523 12225 raft_consensus.cc:2749] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.035506 11960 leader_election.cc:304] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05; no voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01
I20260709 12:22:59.036350 12207 raft_consensus.cc:3060] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.038599 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:59.039572 12070 raft_consensus.cc:3060] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.043880 12207 raft_consensus.cc:2749] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.044389 12190 raft_consensus.cc:493] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.045060 12190 raft_consensus.cc:515] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.047919 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:59.048538 11995 raft_consensus.cc:2468] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:22:59.049963 12035 leader_election.cc:304] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.050827 12194 raft_consensus.cc:2804] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.051047 12208 leader_election.cc:290] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.051218 12194 raft_consensus.cc:493] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.052490 12194 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.052583 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.053771 12143 raft_consensus.cc:2393] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in current term 1: Already voted for candidate 91885dd1fdd54c928ae934cbe377559a in this term.
I20260709 12:22:59.054234 12236 raft_consensus.cc:493] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.054764 12236 raft_consensus.cc:515] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.055765 12185 tablet_bootstrap.cc:492] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:22:59.056419 12185 ts_tablet_manager.cc:1403] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.016s
I20260709 12:22:59.056655 12186 raft_consensus.cc:359] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.057619 12186 raft_consensus.cc:385] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.058108 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215c4dd47fc64dd4ae6dfad696c471da" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:59.058671 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215c4dd47fc64dd4ae6dfad696c471da" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:59.059407 12143 raft_consensus.cc:2468] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
W20260709 12:22:59.059526 12035 leader_election.cc:343] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.056948 12236 leader_election.cc:290] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.057988 12186 raft_consensus.cc:740] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.060206 12185 raft_consensus.cc:359] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.060648 12035 leader_election.cc:304] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:22:59.061280 12185 raft_consensus.cc:385] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.061697 12185 raft_consensus.cc:740] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.061847 12236 raft_consensus.cc:2804] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.062496 12190 leader_election.cc:290] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.062966 12236 raft_consensus.cc:493] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.063329 12194 raft_consensus.cc:515] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.063012 12185 consensus_queue.cc:260] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.063627 12236 raft_consensus.cc:3060] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.063556 12208 raft_consensus.cc:493] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.064561 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:59.064977 12208 raft_consensus.cc:515] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.066231 12185 ts_tablet_manager.cc:1434] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:22:59.067116 12208 leader_election.cc:290] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.068034 12194 leader_election.cc:290] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.068775 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:59.069587 12069 raft_consensus.cc:2468] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:22:59.069947 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:59.071058 11960 leader_election.cc:304] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.071983 12208 raft_consensus.cc:2804] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:59.072395 12208 raft_consensus.cc:493] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.072202 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.072867 12208 raft_consensus.cc:3060] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.073177 11994 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.076262 12236 raft_consensus.cc:515] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.078464 12236 leader_election.cc:290] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
W20260709 12:22:59.079725 12035 leader_election.cc:343] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.080338 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.080260 12208 raft_consensus.cc:515] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.061628 12186 consensus_queue.cc:260] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.082510 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215c4dd47fc64dd4ae6dfad696c471da" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.083266 12208 leader_election.cc:290] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.083236 11995 raft_consensus.cc:3060] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.084825 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:59.085516 12069 raft_consensus.cc:3060] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.086086 12186 ts_tablet_manager.cc:1434] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 12:22:59.086980 12186 tablet_bootstrap.cc:492] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:59.090790 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215c4dd47fc64dd4ae6dfad696c471da" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.091617 12186 tablet_bootstrap.cc:654] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.091419 12144 raft_consensus.cc:3060] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.091874 12143 raft_consensus.cc:2468] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
W20260709 12:22:59.094095 12035 leader_election.cc:343] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:59.094786 12070 raft_consensus.cc:2468] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:22:59.091347 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.099442 12142 raft_consensus.cc:3060] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.101485 12069 raft_consensus.cc:2468] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:22:59.105058 11960 leader_election.cc:304] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:59.106185 12208 raft_consensus.cc:2804] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.106868 12208 raft_consensus.cc:697] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.106726 11960 leader_election.cc:304] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.108111 12142 raft_consensus.cc:2468] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:22:59.108654 12207 raft_consensus.cc:2804] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.108069 12208 consensus_queue.cc:237] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.109364 12207 raft_consensus.cc:697] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.110574 12207 consensus_queue.cc:237] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.123309 11994 raft_consensus.cc:2468] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:22:59.125480 12035 leader_election.cc:304] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:59.126935 12236 raft_consensus.cc:2804] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.127702 12236 raft_consensus.cc:697] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.125651 12144 raft_consensus.cc:2468] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:22:59.129022 12236 consensus_queue.cc:237] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.132578 12035 leader_election.cc:304] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.138531 12194 raft_consensus.cc:2804] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.139417 12194 raft_consensus.cc:697] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.141008 12194 consensus_queue.cc:237] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.142707 12190 raft_consensus.cc:493] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.143323 12190 raft_consensus.cc:515] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.146255 12190 leader_election.cc:290] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.148515 11993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d39c2cef1e4d8dbe8b750de8c7f9e2" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:59.152758 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8d39c2cef1e4d8dbe8b750de8c7f9e2" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:22:59.153844 12144 raft_consensus.cc:2393] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a958846336274bb386baea0d38600e01 in current term 1: Already voted for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in this term.
I20260709 12:22:59.155728 12035 leader_election.cc:304] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:59.157099 12190 raft_consensus.cc:3060] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.162011 11890 catalog_manager.cc:5697] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.167501 12190 raft_consensus.cc:2749] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.176909 11995 raft_consensus.cc:2468] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:22:59.189678 11890 catalog_manager.cc:5697] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.202178 12186 tablet_bootstrap.cc:492] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:59.202836 12186 ts_tablet_manager.cc:1403] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.116s	user 0.032s	sys 0.064s
I20260709 12:22:59.205510 12186 raft_consensus.cc:359] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.206192 12186 raft_consensus.cc:385] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.206475 12186 raft_consensus.cc:740] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.207170 12186 consensus_queue.cc:260] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.207327 11891 catalog_manager.cc:5697] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.208889 12186 ts_tablet_manager.cc:1434] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:59.209841 12186 tablet_bootstrap.cc:492] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:22:59.214416 12186 tablet_bootstrap.cc:654] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.218941 12186 tablet_bootstrap.cc:492] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:22:59.219339 12186 ts_tablet_manager.cc:1403] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:22:59.221129 12186 raft_consensus.cc:359] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.221645 12186 raft_consensus.cc:385] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.221900 12186 raft_consensus.cc:740] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.222576 12186 consensus_queue.cc:260] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.224306 12186 ts_tablet_manager.cc:1434] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
I20260709 12:22:59.224937 11890 catalog_manager.cc:5697] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.286391 12208 consensus_queue.cc:1048] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.293771 12192 consensus_queue.cc:1048] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.305697 12208 consensus_queue.cc:1048] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.310415 12226 consensus_queue.cc:1048] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.363379 12225 raft_consensus.cc:493] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.363732 12069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff"
dest_uuid: "a958846336274bb386baea0d38600e01"
mode: GRACEFUL
new_leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:52278
I20260709 12:22:59.363920 12225 raft_consensus.cc:515] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.364557 12069 raft_consensus.cc:606] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Received request to transfer leadership to TS 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:22:59.366401 12225 leader_election.cc:290] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:59.367152 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:22:59.367543 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:22:59.367888 11995 raft_consensus.cc:2393] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91885dd1fdd54c928ae934cbe377559a in current term 1: Already voted for candidate a958846336274bb386baea0d38600e01 in this term.
I20260709 12:22:59.369223 12109 leader_election.cc:304] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01
I20260709 12:22:59.370007 12225 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.375694 12225 raft_consensus.cc:2749] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:59.380213 12144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46"
dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
mode: GRACEFUL
new_leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:55686
I20260709 12:22:59.380908 12144 raft_consensus.cc:606] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Received request to transfer leadership to TS 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:22:59.396938 11995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
mode: GRACEFUL
new_leader_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:60128
I20260709 12:22:59.397403 11995 raft_consensus.cc:606] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Received request to transfer leadership to TS a958846336274bb386baea0d38600e01
I20260709 12:22:59.402387 11995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
mode: GRACEFUL
new_leader_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:60128
I20260709 12:22:59.402844 11995 raft_consensus.cc:606] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Received request to transfer leadership to TS a958846336274bb386baea0d38600e01
I20260709 12:22:59.409996 11995 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
mode: GRACEFUL
new_leader_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:60128
I20260709 12:22:59.410533 11995 raft_consensus.cc:606] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Received request to transfer leadership to TS a958846336274bb386baea0d38600e01
I20260709 12:22:59.415303 12069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492"
dest_uuid: "a958846336274bb386baea0d38600e01"
mode: GRACEFUL
new_leader_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:52278
I20260709 12:22:59.415797 12069 raft_consensus.cc:606] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Received request to transfer leadership to TS 91885dd1fdd54c928ae934cbe377559a
I20260709 12:22:59.433494 12225 consensus_queue.cc:1048] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.439801 12228 raft_consensus.cc:993] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: : Instructing follower 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:22:59.440344 12191 raft_consensus.cc:1081] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Signalling peer 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:22:59.442236 11994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:60126
I20260709 12:22:59.443804 11994 raft_consensus.cc:493] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.444218 11994 raft_consensus.cc:3060] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.446782 12228 consensus_queue.cc:1048] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.451391 11994 raft_consensus.cc:515] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.453896 11994 leader_election.cc:290] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.454769 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:59.455420 12070 raft_consensus.cc:3060] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.456156 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ede410d0ce4646889e6f17c8c742cc46" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.456746 12144 raft_consensus.cc:3055] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.457063 12144 raft_consensus.cc:740] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:22:59.457870 12144 consensus_queue.cc:260] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.459172 12144 raft_consensus.cc:3060] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.459009 12194 consensus_queue.cc:1048] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.462610 12070 raft_consensus.cc:2468] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:22:59.464644 11960 leader_election.cc:304] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.465771 12226 raft_consensus.cc:2804] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.466442 12226 raft_consensus.cc:697] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.467970 12144 raft_consensus.cc:2468] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:22:59.467541 12226 consensus_queue.cc:237] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.472260 12144 raft_consensus.cc:3060] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.478848 12236 raft_consensus.cc:993] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: : Instructing follower 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:22:59.479375 12236 raft_consensus.cc:1081] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Signalling peer 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:22:59.480058 12226 consensus_queue.cc:1048] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.481565 12236 consensus_queue.cc:1048] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.481567 11891 catalog_manager.cc:5697] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 1 to 2, leader changed from 91885dd1fdd54c928ae934cbe377559a (127.30.25.3) to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.483958 12144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492"
dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:55666
I20260709 12:22:59.487887 12144 raft_consensus.cc:493] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.488300 12144 raft_consensus.cc:3060] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.500691 12207 consensus_queue.cc:1048] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.500277 12144 raft_consensus.cc:515] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.504142 12144 leader_election.cc:290] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:22:59.505084 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:59.505767 12069 raft_consensus.cc:3055] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.506081 12069 raft_consensus.cc:740] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.506966 12069 consensus_queue.cc:260] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.508315 12069 raft_consensus.cc:3060] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.511118 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ee1ff107b34cae9b04c12178909492" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.511737 11994 raft_consensus.cc:3060] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.518702 11994 raft_consensus.cc:2468] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 2.
I20260709 12:22:59.519814 12109 leader_election.cc:304] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:22:59.520135 12069 raft_consensus.cc:2468] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 2.
I20260709 12:22:59.520710 12191 raft_consensus.cc:2804] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.521239 12191 raft_consensus.cc:697] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 2 LEADER]: Becoming Leader. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:22:59.522066 12191 consensus_queue.cc:237] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.528623 11891 catalog_manager.cc:5697] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a reported cstate change: term changed from 1 to 2, leader changed from a958846336274bb386baea0d38600e01 (127.30.25.2) to 91885dd1fdd54c928ae934cbe377559a (127.30.25.3). New cstate: current_term: 2 leader_uuid: "91885dd1fdd54c928ae934cbe377559a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:59.582206 12194 consensus_queue.cc:1048] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.592912 12236 consensus_queue.cc:1048] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.600816 12190 raft_consensus.cc:993] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: : Instructing follower 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:22:59.601307 12194 raft_consensus.cc:1081] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Signalling peer 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:22:59.602798 11994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:60114
I20260709 12:22:59.603364 11994 raft_consensus.cc:493] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.603744 11994 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.609068 12207 consensus_queue.cc:1048] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.609977 11994 raft_consensus.cc:515] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.612164 11994 leader_election.cc:290] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.612838 12070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:22:59.613188 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.613535 12070 raft_consensus.cc:3055] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.613876 12144 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.614051 12070 raft_consensus.cc:740] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.616010 12070 consensus_queue.cc:260] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.617836 12070 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.620481 12144 raft_consensus.cc:2468] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:22:59.621920 11960 leader_election.cc:304] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:22:59.622879 12207 raft_consensus.cc:2804] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.623353 12207 raft_consensus.cc:697] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.624203 12207 consensus_queue.cc:237] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.625401 12208 raft_consensus.cc:993] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: : Instructing follower a958846336274bb386baea0d38600e01 to start an election
I20260709 12:22:59.625885 12208 raft_consensus.cc:1081] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Signalling peer a958846336274bb386baea0d38600e01 to start an election
I20260709 12:22:59.629695 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8"
dest_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:52264
I20260709 12:22:59.632917 12070 raft_consensus.cc:2468] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:22:59.633890 12068 raft_consensus.cc:493] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.633055 11890 catalog_manager.cc:5697] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 1 to 2, leader changed from a958846336274bb386baea0d38600e01 (127.30.25.2) to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.634438 12068 raft_consensus.cc:3060] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.634470 12207 consensus_queue.cc:1048] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.642993 12068 raft_consensus.cc:515] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.646852 12068 leader_election.cc:290] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.648674 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.649390 12144 raft_consensus.cc:3060] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.652441 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae0d804930c3497da9c71c272591e6b8" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.653137 11994 raft_consensus.cc:3055] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.653487 11994 raft_consensus.cc:740] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.654762 11994 consensus_queue.cc:260] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.656100 11994 raft_consensus.cc:3060] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.661827 12144 raft_consensus.cc:2468] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:22:59.663067 12035 leader_election.cc:304] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.663789 12194 raft_consensus.cc:2804] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.664287 12194 raft_consensus.cc:697] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.664436 11994 raft_consensus.cc:2468] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:22:59.665134 12194 consensus_queue.cc:237] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.671928 11892 catalog_manager.cc:5697] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 1 to 2, leader changed from 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1) to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.685312 12207 consensus_queue.cc:1048] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.697301 12192 consensus_queue.cc:1048] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.704952 12190 consensus_queue.cc:1048] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:59.707624 12208 raft_consensus.cc:993] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: : Instructing follower a958846336274bb386baea0d38600e01 to start an election
I20260709 12:22:59.708124 12192 raft_consensus.cc:1081] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Signalling peer a958846336274bb386baea0d38600e01 to start an election
I20260709 12:22:59.715576 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5"
dest_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:52264
I20260709 12:22:59.716179 12068 raft_consensus.cc:493] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.716625 12068 raft_consensus.cc:3060] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.724490 12194 consensus_queue.cc:1048] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:59.726644 12068 raft_consensus.cc:515] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.729501 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.731788 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0995365e4a4b878cb2b16803b7f0c5" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.732450 12143 raft_consensus.cc:3060] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.732980 12068 leader_election.cc:290] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.730091 11995 raft_consensus.cc:3055] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.734076 11995 raft_consensus.cc:740] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.735476 11995 consensus_queue.cc:260] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.737249 11995 raft_consensus.cc:3060] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.740617 12143 raft_consensus.cc:2468] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:22:59.741932 12035 leader_election.cc:304] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.742853 12236 raft_consensus.cc:2804] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.743342 12236 raft_consensus.cc:697] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.744175 12236 consensus_queue.cc:237] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.746130 11995 raft_consensus.cc:2468] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:22:59.751982 11891 catalog_manager.cc:5697] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 1 to 2, leader changed from 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1) to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.840415 12208 raft_consensus.cc:993] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: : Instructing follower a958846336274bb386baea0d38600e01 to start an election
I20260709 12:22:59.840906 12192 raft_consensus.cc:1081] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Signalling peer a958846336274bb386baea0d38600e01 to start an election
I20260709 12:22:59.842442 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1"
dest_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:52264
I20260709 12:22:59.842964 12068 raft_consensus.cc:493] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.843324 12068 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.848470 12068 raft_consensus.cc:515] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.850071 12068 leader_election.cc:290] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:22:59.850791 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:22:59.851047 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:22:59.851504 11995 raft_consensus.cc:3055] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.851603 12144 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.851879 11995 raft_consensus.cc:740] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:22:59.852736 11995 consensus_queue.cc:260] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.853981 11995 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.858745 12144 raft_consensus.cc:2468] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:22:59.860023 12035 leader_election.cc:304] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:22:59.860761 12236 raft_consensus.cc:2804] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.860859 11995 raft_consensus.cc:2468] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:22:59.861330 12236 raft_consensus.cc:697] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:22:59.862248 12236 consensus_queue.cc:237] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:22:59.868372 11891 catalog_manager.cc:5697] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 1 to 2, leader changed from 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1) to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.881985 12068 raft_consensus.cc:1275] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:59.883296 12192 consensus_queue.cc:1048] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:59.890825 12144 raft_consensus.cc:1275] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:59.892338 12207 consensus_queue.cc:1048] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:59.969877 12068 raft_consensus.cc:1275] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:59.971259 12191 consensus_queue.cc:1048] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:59.979734 11995 raft_consensus.cc:1275] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:59.981400 12191 consensus_queue.cc:1048] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.078052 12068 raft_consensus.cc:1275] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.079372 12226 consensus_queue.cc:1048] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.086872 12144 raft_consensus.cc:1275] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.088215 12207 consensus_queue.cc:1048] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.222371 12143 raft_consensus.cc:1275] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.223659 12236 consensus_queue.cc:1048] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.231384 11995 raft_consensus.cc:1275] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.232899 12269 consensus_queue.cc:1048] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.234112 11995 raft_consensus.cc:1275] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.235859 12236 consensus_queue.cc:1048] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.248140 12143 raft_consensus.cc:1275] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.250005 12194 consensus_queue.cc:1048] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.304891 11994 raft_consensus.cc:1275] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.306053 12270 consensus_queue.cc:1048] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.314702 12144 raft_consensus.cc:1275] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.316135 12194 consensus_queue.cc:1048] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.460093 11890 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37566:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:02.463042 11890 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:02.523665 11979 tablet_service.cc:1511] Processing CreateTablet for tablet 8682452fd61349e49782e24ea9c59a77 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:02.524799 12057 tablet_service.cc:1511] Processing CreateTablet for tablet 8682452fd61349e49782e24ea9c59a77 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:02.525736 11979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8682452fd61349e49782e24ea9c59a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.526638 12133 tablet_service.cc:1511] Processing CreateTablet for tablet 8682452fd61349e49782e24ea9c59a77 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:02.526638 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8682452fd61349e49782e24ea9c59a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.528244 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8682452fd61349e49782e24ea9c59a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.546198 11977 tablet_service.cc:1511] Processing CreateTablet for tablet 6e9f2ad0c2c14e26bb0db426ff70af3f (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:02.548110 11977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e9f2ad0c2c14e26bb0db426ff70af3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.553669 12055 tablet_service.cc:1511] Processing CreateTablet for tablet 6e9f2ad0c2c14e26bb0db426ff70af3f (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:02.555366 12055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e9f2ad0c2c14e26bb0db426ff70af3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.556844 12127 tablet_service.cc:1511] Processing CreateTablet for tablet 6e9f2ad0c2c14e26bb0db426ff70af3f (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:02.558300 12127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e9f2ad0c2c14e26bb0db426ff70af3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.573222 11983 tablet_service.cc:1511] Processing CreateTablet for tablet 37e636804216407ab1bcfd35f444d34a (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:02.575567 11983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37e636804216407ab1bcfd35f444d34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.576406 12293 tablet_bootstrap.cc:492] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:02.580787 12291 tablet_bootstrap.cc:492] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:02.582224 12292 tablet_bootstrap.cc:492] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:02.583405 12057 tablet_service.cc:1511] Processing CreateTablet for tablet 37e636804216407ab1bcfd35f444d34a (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:02.585539 11979 tablet_service.cc:1511] Processing CreateTablet for tablet b9967187d4ff4971985685c439a92590 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:02.586930 11979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9967187d4ff4971985685c439a92590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.586617 12133 tablet_service.cc:1511] Processing CreateTablet for tablet 37e636804216407ab1bcfd35f444d34a (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:02.585577 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37e636804216407ab1bcfd35f444d34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.588063 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37e636804216407ab1bcfd35f444d34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.592164 12292 tablet_bootstrap.cc:654] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.594723 12293 tablet_bootstrap.cc:654] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.600780 11977 tablet_service.cc:1511] Processing CreateTablet for tablet d4cfc047f3204be0aee3cc87b04c89f4 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:02.603682 11977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cfc047f3204be0aee3cc87b04c89f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.606156 12291 tablet_bootstrap.cc:654] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.600890 12134 tablet_service.cc:1511] Processing CreateTablet for tablet b9967187d4ff4971985685c439a92590 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:02.607395 12134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9967187d4ff4971985685c439a92590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.600884 12054 tablet_service.cc:1511] Processing CreateTablet for tablet b9967187d4ff4971985685c439a92590 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:02.611223 12054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9967187d4ff4971985685c439a92590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.612550 12133 tablet_service.cc:1511] Processing CreateTablet for tablet d4cfc047f3204be0aee3cc87b04c89f4 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:02.613930 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cfc047f3204be0aee3cc87b04c89f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.615505 12055 tablet_service.cc:1511] Processing CreateTablet for tablet d4cfc047f3204be0aee3cc87b04c89f4 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:02.623850 11979 tablet_service.cc:1511] Processing CreateTablet for tablet 2dfc9ec5c0a24b0d94657880b1bd7fb1 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:02.624630 12055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cfc047f3204be0aee3cc87b04c89f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.625265 11979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfc9ec5c0a24b0d94657880b1bd7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.632640 12059 tablet_service.cc:1511] Processing CreateTablet for tablet 2dfc9ec5c0a24b0d94657880b1bd7fb1 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:02.632794 12130 tablet_service.cc:1511] Processing CreateTablet for tablet 2dfc9ec5c0a24b0d94657880b1bd7fb1 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:02.634397 12130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfc9ec5c0a24b0d94657880b1bd7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.635907 11980 tablet_service.cc:1511] Processing CreateTablet for tablet 024a5510f1da4a24ab6c12a9e78ae4ef (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:02.637339 11980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 024a5510f1da4a24ab6c12a9e78ae4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.638481 12059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfc9ec5c0a24b0d94657880b1bd7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.641547 12292 tablet_bootstrap.cc:492] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:02.641021 12129 tablet_service.cc:1511] Processing CreateTablet for tablet 024a5510f1da4a24ab6c12a9e78ae4ef (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:02.642871 12129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 024a5510f1da4a24ab6c12a9e78ae4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.643934 12052 tablet_service.cc:1511] Processing CreateTablet for tablet 024a5510f1da4a24ab6c12a9e78ae4ef (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:02.645481 12052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 024a5510f1da4a24ab6c12a9e78ae4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.641021 11981 tablet_service.cc:1511] Processing CreateTablet for tablet 719c67c8a50d48b1ae3d8b28c3892a24 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:02.647217 11981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719c67c8a50d48b1ae3d8b28c3892a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.646997 12127 tablet_service.cc:1511] Processing CreateTablet for tablet 719c67c8a50d48b1ae3d8b28c3892a24 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:02.648331 12127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719c67c8a50d48b1ae3d8b28c3892a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.656587 11982 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef9f6c9adea439b83bd628a79f6ba9a (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:02.658094 11982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef9f6c9adea439b83bd628a79f6ba9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.658685 12292 ts_tablet_manager.cc:1403] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.077s	user 0.020s	sys 0.000s
I20260709 12:23:02.668555 12058 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef9f6c9adea439b83bd628a79f6ba9a (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:02.670040 12058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef9f6c9adea439b83bd628a79f6ba9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.672220 12130 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef9f6c9adea439b83bd628a79f6ba9a (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:02.675678 12130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef9f6c9adea439b83bd628a79f6ba9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.684687 12291 tablet_bootstrap.cc:492] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:02.662175 12292 raft_consensus.cc:359] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.687989 12292 raft_consensus.cc:385] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.689265 12292 raft_consensus.cc:740] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.690621 12292 consensus_queue.cc:260] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.693490 12292 ts_tablet_manager.cc:1434] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.004s
I20260709 12:23:02.696327 12293 tablet_bootstrap.cc:492] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:02.697530 12293 ts_tablet_manager.cc:1403] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.122s	user 0.009s	sys 0.005s
I20260709 12:23:02.696935 12292 tablet_bootstrap.cc:492] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:02.719803 12291 ts_tablet_manager.cc:1403] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.139s	user 0.010s	sys 0.005s
I20260709 12:23:02.721616 12293 raft_consensus.cc:359] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.722499 12293 raft_consensus.cc:385] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.722798 12293 raft_consensus.cc:740] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.723424 12291 raft_consensus.cc:359] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.724256 12291 raft_consensus.cc:385] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.724584 12291 raft_consensus.cc:740] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.725414 12291 consensus_queue.cc:260] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.726464 12293 consensus_queue.cc:260] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.728036 12291 ts_tablet_manager.cc:1434] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:02.734472 12291 tablet_bootstrap.cc:492] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:02.739650 12293 ts_tablet_manager.cc:1434] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.042s	user 0.001s	sys 0.006s
I20260709 12:23:02.740681 12293 tablet_bootstrap.cc:492] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:02.733287 12292 tablet_bootstrap.cc:654] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.728507 12057 tablet_service.cc:1511] Processing CreateTablet for tablet 719c67c8a50d48b1ae3d8b28c3892a24 (DEFAULT_TABLE table=soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:02.747421 12057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719c67c8a50d48b1ae3d8b28c3892a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.747622 12293 tablet_bootstrap.cc:654] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.757927 12291 tablet_bootstrap.cc:654] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:02.764411 11941 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:02.771711 12292 tablet_bootstrap.cc:492] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:02.771752 12194 raft_consensus.cc:493] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.772538 12292 ts_tablet_manager.cc:1403] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.012s
I20260709 12:23:02.772511 12293 tablet_bootstrap.cc:492] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:02.772509 12194 raft_consensus.cc:515] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.773519 12293 ts_tablet_manager.cc:1403] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.033s	user 0.026s	sys 0.005s
I20260709 12:23:02.773617 12291 tablet_bootstrap.cc:492] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:02.774155 12291 ts_tablet_manager.cc:1403] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.003s
I20260709 12:23:02.776208 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:02.776101 12292 raft_consensus.cc:359] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.777032 12292 raft_consensus.cc:385] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.776854 12293 raft_consensus.cc:359] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.777544 12292 raft_consensus.cc:740] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.777292 12291 raft_consensus.cc:359] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.778208 12291 raft_consensus.cc:385] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.778586 12291 raft_consensus.cc:740] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.778498 12292 consensus_queue.cc:260] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.779249 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:02.779920 12144 raft_consensus.cc:2468] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:02.779597 12291 consensus_queue.cc:260] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.777081 11995 raft_consensus.cc:2468] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:02.781564 12035 leader_election.cc:304] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:02.781924 12194 leader_election.cc:290] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:02.782598 12279 raft_consensus.cc:2804] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:02.782649 12293 raft_consensus.cc:385] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.783082 12279 raft_consensus.cc:493] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.783133 12293 raft_consensus.cc:740] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.783566 12279 raft_consensus.cc:3060] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.783911 12292 ts_tablet_manager.cc:1434] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:23:02.784193 12293 consensus_queue.cc:260] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.784997 12292 tablet_bootstrap.cc:492] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:02.786938 12293 ts_tablet_manager.cc:1434] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.005s
I20260709 12:23:02.788009 12293 tablet_bootstrap.cc:492] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:02.789410 12291 ts_tablet_manager.cc:1434] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:23:02.793063 12291 tablet_bootstrap.cc:492] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:02.794812 12293 tablet_bootstrap.cc:654] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.821667 12292 tablet_bootstrap.cc:654] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.824244 12291 tablet_bootstrap.cc:654] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.832005 12293 tablet_bootstrap.cc:492] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:02.832679 12293 ts_tablet_manager.cc:1403] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.045s	user 0.023s	sys 0.019s
I20260709 12:23:02.834571 12291 tablet_bootstrap.cc:492] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:02.835235 12291 ts_tablet_manager.cc:1403] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.024s
I20260709 12:23:02.836257 12293 raft_consensus.cc:359] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.837196 12293 raft_consensus.cc:385] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.837541 12293 raft_consensus.cc:740] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.838403 12293 consensus_queue.cc:260] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.838651 12291 raft_consensus.cc:359] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.839756 12291 raft_consensus.cc:385] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.840135 12291 raft_consensus.cc:740] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.841068 12293 ts_tablet_manager.cc:1434] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:23:02.842175 12293 tablet_bootstrap.cc:492] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:02.843420 12279 raft_consensus.cc:515] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.845788 12279 leader_election.cc:290] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:02.847683 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:02.848345 11995 raft_consensus.cc:3060] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.849228 12293 tablet_bootstrap.cc:654] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.849385 12292 tablet_bootstrap.cc:492] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:02.849907 12292 ts_tablet_manager.cc:1403] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.028s
I20260709 12:23:02.841110 12291 consensus_queue.cc:260] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.851431 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:02.852130 12144 raft_consensus.cc:3060] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.852962 12291 ts_tablet_manager.cc:1434] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.004s
I20260709 12:23:02.853207 12292 raft_consensus.cc:359] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.854007 12292 raft_consensus.cc:385] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.854343 12292 raft_consensus.cc:740] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.854430 12291 tablet_bootstrap.cc:492] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:02.863598 12144 raft_consensus.cc:2468] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:02.864346 12291 tablet_bootstrap.cc:654] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.865212 12035 leader_election.cc:304] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:02.866106 12279 raft_consensus.cc:2804] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:02.866698 12279 raft_consensus.cc:697] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:02.867551 12279 consensus_queue.cc:237] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.861593 12292 consensus_queue.cc:260] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.868961 11995 raft_consensus.cc:2468] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:02.871619 12292 ts_tablet_manager.cc:1434] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 12:23:02.873075 12292 tablet_bootstrap.cc:492] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:02.882647 12194 raft_consensus.cc:493] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.883248 12194 raft_consensus.cc:515] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.884343 12292 tablet_bootstrap.cc:654] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.885481 12194 leader_election.cc:290] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:02.889555 12293 tablet_bootstrap.cc:492] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:02.890486 12293 ts_tablet_manager.cc:1403] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.049s	user 0.025s	sys 0.018s
I20260709 12:23:02.895256 12293 raft_consensus.cc:359] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.895864 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:02.896452 12293 raft_consensus.cc:385] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.897084 12293 raft_consensus.cc:740] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.898403 12293 consensus_queue.cc:260] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.899116 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:02.899828 12144 raft_consensus.cc:2468] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
W20260709 12:23:02.901502 12035 leader_election.cc:343] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.902518 12293 ts_tablet_manager.cc:1434] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.002s
I20260709 12:23:02.911916 12293 tablet_bootstrap.cc:492] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:02.903039 12035 leader_election.cc:304] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:23:02.912329 12292 tablet_bootstrap.cc:492] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:02.913031 12292 ts_tablet_manager.cc:1403] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.011s
I20260709 12:23:02.912997 12194 raft_consensus.cc:2804] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:02.913684 12194 raft_consensus.cc:493] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.914063 12194 raft_consensus.cc:3060] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.928969 12293 tablet_bootstrap.cc:654] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.932243 11890 catalog_manager.cc:5697] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:02.933749 12291 tablet_bootstrap.cc:492] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:02.934480 12291 ts_tablet_manager.cc:1403] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.042s
I20260709 12:23:02.934907 12292 raft_consensus.cc:359] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.935701 12292 raft_consensus.cc:385] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.937090 12292 raft_consensus.cc:740] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.937839 12291 raft_consensus.cc:359] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.938046 12292 consensus_queue.cc:260] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.938710 12291 raft_consensus.cc:385] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.938730 12286 raft_consensus.cc:493] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.939129 12291 raft_consensus.cc:740] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.940665 12286 raft_consensus.cc:515] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.941854 12291 consensus_queue.cc:260] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.943102 12286 leader_election.cc:290] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:02.946507 12291 ts_tablet_manager.cc:1434] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:23:02.947969 12291 tablet_bootstrap.cc:492] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:02.948165 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:02.948866 12068 raft_consensus.cc:2468] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 0.
I20260709 12:23:02.958809 12109 leader_election.cc:304] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:02.959712 12286 raft_consensus.cc:2804] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:02.960103 12286 raft_consensus.cc:493] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.960450 12286 raft_consensus.cc:3060] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.960310 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:02.966085 11995 raft_consensus.cc:2468] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 0.
I20260709 12:23:02.976080 12292 ts_tablet_manager.cc:1434] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.053s	user 0.017s	sys 0.013s
I20260709 12:23:02.976016 12194 raft_consensus.cc:515] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.977412 12292 tablet_bootstrap.cc:492] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:02.981233 12291 tablet_bootstrap.cc:654] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.985070 12293 tablet_bootstrap.cc:492] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:02.985577 12293 ts_tablet_manager.cc:1403] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.004s
I20260709 12:23:02.985448 12286 raft_consensus.cc:515] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.989676 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:02.990545 11994 raft_consensus.cc:3060] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.991029 12292 tablet_bootstrap.cc:654] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.995428 12293 raft_consensus.cc:359] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:02.996294 12293 raft_consensus.cc:385] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.996590 12293 raft_consensus.cc:740] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.996984 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:02.997480 12293 consensus_queue.cc:260] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
W20260709 12:23:02.998517 12035 leader_election.cc:343] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.000320 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.001142 12144 raft_consensus.cc:3060] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.001731 12293 ts_tablet_manager.cc:1434] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 12:23:03.003249 12293 tablet_bootstrap.cc:492] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:02.995828 12194 leader_election.cc:290] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.010061 12293 tablet_bootstrap.cc:654] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.016590 12144 raft_consensus.cc:2468] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:03.018026 12286 leader_election.cc:290] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:03.021564 12292 tablet_bootstrap.cc:492] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:03.021224 11994 raft_consensus.cc:2468] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 1.
I20260709 12:23:03.021510 12291 tablet_bootstrap.cc:492] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:03.019166 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.022251 12292 ts_tablet_manager.cc:1403] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.009s
I20260709 12:23:03.022742 12068 raft_consensus.cc:3060] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.023442 12035 leader_election.cc:304] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:23:03.025105 12293 tablet_bootstrap.cc:492] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:03.025736 12293 ts_tablet_manager.cc:1403] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.002s
I20260709 12:23:03.025688 12292 raft_consensus.cc:359] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.025113 12194 raft_consensus.cc:2804] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.026659 12292 raft_consensus.cc:385] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.026969 12194 raft_consensus.cc:697] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.027312 12292 raft_consensus.cc:740] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.028013 12194 consensus_queue.cc:237] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.028381 12292 consensus_queue.cc:260] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.029060 12109 leader_election.cc:304] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:03.030380 12192 raft_consensus.cc:493] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.030206 12308 raft_consensus.cc:2804] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.030197 12293 raft_consensus.cc:359] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.031203 12293 raft_consensus.cc:385] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.030982 12192 raft_consensus.cc:515] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.031597 12293 raft_consensus.cc:740] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.032562 12293 consensus_queue.cc:260] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.033380 12192 leader_election.cc:290] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.035527 12293 ts_tablet_manager.cc:1434] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:03.035411 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9967187d4ff4971985685c439a92590" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.036228 12069 raft_consensus.cc:2468] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:23:03.036850 12293 tablet_bootstrap.cc:492] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:03.038000 11960 leader_election.cc:304] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.039554 12292 ts_tablet_manager.cc:1434] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:23:03.040201 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9967187d4ff4971985685c439a92590" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.040426 12308 raft_consensus.cc:697] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Becoming Leader. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:23:03.041312 12292 tablet_bootstrap.cc:492] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
W20260709 12:23:03.041841 11960 leader_election.cc:343] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.041410 12308 consensus_queue.cc:237] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.042876 12068 raft_consensus.cc:2468] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 1.
I20260709 12:23:03.044433 12291 ts_tablet_manager.cc:1403] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.024s
I20260709 12:23:03.042883 12192 raft_consensus.cc:2804] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.045156 12192 raft_consensus.cc:493] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.045562 12293 tablet_bootstrap.cc:654] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.045511 12192 raft_consensus.cc:3060] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.047796 12291 raft_consensus.cc:359] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.048604 12291 raft_consensus.cc:385] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.048918 12291 raft_consensus.cc:740] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.050161 12291 consensus_queue.cc:260] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.052626 12292 tablet_bootstrap.cc:654] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.056984 11891 catalog_manager.cc:5697] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.062557 12293 tablet_bootstrap.cc:492] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:03.063264 12293 ts_tablet_manager.cc:1403] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.002s
I20260709 12:23:03.066833 12293 raft_consensus.cc:359] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.067632 12293 raft_consensus.cc:385] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.068015 12293 raft_consensus.cc:740] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.068977 12293 consensus_queue.cc:260] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.072196 12293 ts_tablet_manager.cc:1434] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:23:03.073750 12293 tablet_bootstrap.cc:492] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:03.081022 12192 raft_consensus.cc:515] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.085390 12293 tablet_bootstrap.cc:654] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.086300 12292 tablet_bootstrap.cc:492] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:03.086918 12292 ts_tablet_manager.cc:1403] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.000s
I20260709 12:23:03.092728 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9967187d4ff4971985685c439a92590" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.093449 12068 raft_consensus.cc:3060] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.097107 11892 catalog_manager.cc:5697] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a reported cstate change: term changed from 0 to 1, leader changed from <none> to 91885dd1fdd54c928ae934cbe377559a (127.30.25.3). New cstate: current_term: 1 leader_uuid: "91885dd1fdd54c928ae934cbe377559a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.098415 12293 tablet_bootstrap.cc:492] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:03.099000 12293 ts_tablet_manager.cc:1403] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.009s
I20260709 12:23:03.100966 12192 leader_election.cc:290] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.101572 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9967187d4ff4971985685c439a92590" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.102033 12068 raft_consensus.cc:2468] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:23:03.103364 12291 ts_tablet_manager.cc:1434] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.030s
I20260709 12:23:03.103266 11960 leader_election.cc:304] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
W20260709 12:23:03.104462 11960 leader_election.cc:343] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.104672 12291 tablet_bootstrap.cc:492] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:03.105357 12192 raft_consensus.cc:2804] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.105892 12192 raft_consensus.cc:697] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:03.106750 12192 consensus_queue.cc:237] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.116165 12291 tablet_bootstrap.cc:654] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.116034 12293 raft_consensus.cc:359] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.117046 12293 raft_consensus.cc:385] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.117347 12293 raft_consensus.cc:740] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.118045 12293 consensus_queue.cc:260] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.120167 12293 ts_tablet_manager.cc:1434] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 12:23:03.122716 12293 tablet_bootstrap.cc:492] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: Bootstrap starting.
I20260709 12:23:03.138482 12293 tablet_bootstrap.cc:654] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.143146 12317 raft_consensus.cc:493] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.143945 12317 raft_consensus.cc:515] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.147989 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.148973 12068 raft_consensus.cc:2468] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:23:03.150458 12315 raft_consensus.cc:493] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.151165 12315 raft_consensus.cc:515] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.152326 12292 raft_consensus.cc:359] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.153357 12315 leader_election.cc:290] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.153736 12292 raft_consensus.cc:385] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.154225 12292 raft_consensus.cc:740] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.155462 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719c67c8a50d48b1ae3d8b28c3892a24" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.155330 12292 consensus_queue.cc:260] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.156050 11960 leader_election.cc:304] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
W20260709 12:23:03.157079 11960 leader_election.cc:343] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.159379 12292 ts_tablet_manager.cc:1434] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.072s	user 0.010s	sys 0.000s
I20260709 12:23:03.157979 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719c67c8a50d48b1ae3d8b28c3892a24" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.160856 12292 tablet_bootstrap.cc:492] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
W20260709 12:23:03.161327 11960 leader_election.cc:343] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.162083 11960 leader_election.cc:304] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05; no voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01
I20260709 12:23:03.167896 12317 leader_election.cc:290] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.163234 12315 raft_consensus.cc:2804] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.168972 12315 raft_consensus.cc:493] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.169431 12315 raft_consensus.cc:3060] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.166375 11892 catalog_manager.cc:5697] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.173643 12317 raft_consensus.cc:2749] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.174954 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.175658 12144 raft_consensus.cc:2468] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:23:03.176599 12293 tablet_bootstrap.cc:492] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: No bootstrap required, opened a new log
I20260709 12:23:03.177101 12293 ts_tablet_manager.cc:1403] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.008s
I20260709 12:23:03.180202 12293 raft_consensus.cc:359] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.180858 12293 raft_consensus.cc:385] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.181133 12293 raft_consensus.cc:740] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.181151 12292 tablet_bootstrap.cc:654] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.182018 12293 consensus_queue.cc:260] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.184415 12293 ts_tablet_manager.cc:1434] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:23:03.184964 12291 tablet_bootstrap.cc:492] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:03.185468 12291 ts_tablet_manager.cc:1403] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.081s	user 0.040s	sys 0.026s
I20260709 12:23:03.188333 12315 raft_consensus.cc:515] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.188558 12291 raft_consensus.cc:359] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.189363 12291 raft_consensus.cc:385] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.189666 12291 raft_consensus.cc:740] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.190534 12291 consensus_queue.cc:260] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.191563 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.192209 12068 raft_consensus.cc:3060] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.192857 12291 ts_tablet_manager.cc:1434] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:03.197082 12291 tablet_bootstrap.cc:492] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:03.203984 12291 tablet_bootstrap.cc:654] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.212589 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.213533 12144 raft_consensus.cc:3060] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.221277 12291 tablet_bootstrap.cc:492] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:03.221979 12291 ts_tablet_manager.cc:1403] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.007s
I20260709 12:23:03.222468 12315 leader_election.cc:290] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.208478 12068 raft_consensus.cc:2468] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:23:03.223698 12144 raft_consensus.cc:2468] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:23:03.224347 11960 leader_election.cc:304] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.225348 12315 raft_consensus.cc:2804] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.225257 12291 raft_consensus.cc:359] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.225950 12292 tablet_bootstrap.cc:492] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:03.226202 12291 raft_consensus.cc:385] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.226536 12292 ts_tablet_manager.cc:1403] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.022s
I20260709 12:23:03.226584 12291 raft_consensus.cc:740] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.226904 12315 raft_consensus.cc:697] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:03.227682 12291 consensus_queue.cc:260] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.228016 12315 consensus_queue.cc:237] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.230022 12291 ts_tablet_manager.cc:1434] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:23:03.229874 12292 raft_consensus.cc:359] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.230746 12292 raft_consensus.cc:385] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.231057 12292 raft_consensus.cc:740] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.231129 12291 tablet_bootstrap.cc:492] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:03.231927 12292 consensus_queue.cc:260] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.234562 12292 ts_tablet_manager.cc:1434] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:03.235590 12292 tablet_bootstrap.cc:492] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:03.238930 12291 tablet_bootstrap.cc:654] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.238719 11892 catalog_manager.cc:5697] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.242697 12292 tablet_bootstrap.cc:654] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.248600 12291 tablet_bootstrap.cc:492] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:03.249056 12291 ts_tablet_manager.cc:1403] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 12:23:03.251802 12291 raft_consensus.cc:359] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.252418 12291 raft_consensus.cc:385] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.252655 12291 raft_consensus.cc:740] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.253260 12291 consensus_queue.cc:260] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.255115 12291 ts_tablet_manager.cc:1434] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:23:03.256026 12291 tablet_bootstrap.cc:492] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: Bootstrap starting.
I20260709 12:23:03.257493 12292 tablet_bootstrap.cc:492] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:03.257957 12292 ts_tablet_manager.cc:1403] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.006s
I20260709 12:23:03.260793 12292 raft_consensus.cc:359] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.261438 12292 raft_consensus.cc:385] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.261709 12292 raft_consensus.cc:740] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.261834 12291 tablet_bootstrap.cc:654] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.262569 12292 consensus_queue.cc:260] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.261772 12317 raft_consensus.cc:493] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.263747 12307 consensus_queue.cc:1048] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:03.264421 12317 raft_consensus.cc:515] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.266506 12317 leader_election.cc:290] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.268275 12292 ts_tablet_manager.cc:1434] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:03.269358 12292 tablet_bootstrap.cc:492] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: Bootstrap starting.
I20260709 12:23:03.273556 12291 tablet_bootstrap.cc:492] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: No bootstrap required, opened a new log
I20260709 12:23:03.274540 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.275291 12068 raft_consensus.cc:2468] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 0.
I20260709 12:23:03.274128 12291 ts_tablet_manager.cc:1403] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:23:03.276309 12292 tablet_bootstrap.cc:654] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.276583 11960 leader_election.cc:304] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.277484 12317 raft_consensus.cc:2804] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.279228 12291 raft_consensus.cc:359] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.280143 12291 raft_consensus.cc:385] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.280444 12291 raft_consensus.cc:740] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.281272 12291 consensus_queue.cc:260] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.283386 12291 ts_tablet_manager.cc:1434] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:03.277849 12317 raft_consensus.cc:493] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.284111 12317 raft_consensus.cc:3060] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.285530 12292 tablet_bootstrap.cc:492] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: No bootstrap required, opened a new log
I20260709 12:23:03.286134 12292 ts_tablet_manager.cc:1403] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:23:03.289130 12292 raft_consensus.cc:359] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.289947 12292 raft_consensus.cc:385] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.291301 12317 raft_consensus.cc:515] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.292872 12317 leader_election.cc:290] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.293768 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.294405 12068 raft_consensus.cc:3060] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.294531 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.295210 12143 raft_consensus.cc:3060] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.290197 12292 raft_consensus.cc:740] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.297396 12292 consensus_queue.cc:260] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.301756 12292 ts_tablet_manager.cc:1434] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:23:03.307546 12143 raft_consensus.cc:2468] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:23:03.308749 11960 leader_election.cc:304] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:03.309491 12317 raft_consensus.cc:2804] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.309962 12317 raft_consensus.cc:697] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:03.310469 12307 consensus_queue.cc:1048] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.310810 12317 consensus_queue.cc:237] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.295011 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.315549 12144 raft_consensus.cc:2376] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1. Re-sending same reply.
I20260709 12:23:03.316069 12068 raft_consensus.cc:2468] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 1.
I20260709 12:23:03.331537 11892 catalog_manager.cc:5697] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 1 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.383677 12194 raft_consensus.cc:493] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.384181 12194 raft_consensus.cc:515] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.384718 12308 raft_consensus.cc:493] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.385267 12308 raft_consensus.cc:515] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.386508 12194 leader_election.cc:290] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.387161 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:03.387449 12308 leader_election.cc:290] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:03.387552 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.387858 11994 raft_consensus.cc:2468] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:03.388352 12143 raft_consensus.cc:2468] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:03.388549 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:03.389129 12035 leader_election.cc:304] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.389225 11995 raft_consensus.cc:2468] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 0.
I20260709 12:23:03.390128 12194 raft_consensus.cc:2804] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.389272 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.390574 12194 raft_consensus.cc:493] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.391055 12194 raft_consensus.cc:3060] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.390823 12109 leader_election.cc:304] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:03.391768 12308 raft_consensus.cc:2804] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.392136 12308 raft_consensus.cc:493] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.392457 12308 raft_consensus.cc:3060] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.394771 12317 raft_consensus.cc:493] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.395358 12317 raft_consensus.cc:515] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.397583 12317 leader_election.cc:290] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.398787 12194 raft_consensus.cc:515] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.399220 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.399322 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.401000 12194 leader_election.cc:290] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.402184 12143 raft_consensus.cc:2393] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in current term 1: Already voted for candidate a958846336274bb386baea0d38600e01 in this term.
I20260709 12:23:03.402410 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.402578 12308 raft_consensus.cc:515] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.403353 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.400267 12068 raft_consensus.cc:2393] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91885dd1fdd54c928ae934cbe377559a in current term 1: Already voted for candidate a958846336274bb386baea0d38600e01 in this term.
I20260709 12:23:03.403798 12142 raft_consensus.cc:2393] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a958846336274bb386baea0d38600e01 in current term 1: Already voted for candidate 91885dd1fdd54c928ae934cbe377559a in this term.
I20260709 12:23:03.403973 11995 raft_consensus.cc:3060] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.404812 12308 leader_election.cc:290] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:03.405694 11960 leader_election.cc:304] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05; no voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01
I20260709 12:23:03.406795 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.406968 12315 raft_consensus.cc:3060] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.406795 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.409803 12109 leader_election.cc:304] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01
I20260709 12:23:03.410714 12308 raft_consensus.cc:2749] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.411727 11995 raft_consensus.cc:2468] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:03.412806 12035 leader_election.cc:304] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 91885dd1fdd54c928ae934cbe377559a
I20260709 12:23:03.413496 12194 raft_consensus.cc:2804] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.413950 12315 raft_consensus.cc:2749] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.414026 12194 raft_consensus.cc:697] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.415092 12194 consensus_queue.cc:237] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.423135 11893 catalog_manager.cc:5697] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.449505 12194 raft_consensus.cc:493] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.449963 12194 raft_consensus.cc:515] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.451627 12194 leader_election.cc:290] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.452373 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719c67c8a50d48b1ae3d8b28c3892a24" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:03.452706 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719c67c8a50d48b1ae3d8b28c3892a24" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.452980 11995 raft_consensus.cc:2468] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:03.453684 12142 raft_consensus.cc:2468] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:03.454155 12035 leader_election.cc:304] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.454978 12194 raft_consensus.cc:2804] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.455345 12194 raft_consensus.cc:493] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.455632 12194 raft_consensus.cc:3060] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.460593 12194 raft_consensus.cc:515] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.462693 12194 leader_election.cc:290] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.463406 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719c67c8a50d48b1ae3d8b28c3892a24" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.463686 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719c67c8a50d48b1ae3d8b28c3892a24" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.464102 11995 raft_consensus.cc:3060] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.464304 12142 raft_consensus.cc:3060] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.471689 11995 raft_consensus.cc:2468] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:03.471701 12142 raft_consensus.cc:2468] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:03.472968 12035 leader_election.cc:304] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.473616 12194 raft_consensus.cc:2804] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.474069 12194 raft_consensus.cc:697] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.474972 12194 consensus_queue.cc:237] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.483923 11893 catalog_manager.cc:5697] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.510249 12308 consensus_queue.cc:1048] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.522157 12286 consensus_queue.cc:1048] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.535298 12194 raft_consensus.cc:493] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.535750 12194 raft_consensus.cc:515] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.537554 12194 leader_election.cc:290] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.538107 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "024a5510f1da4a24ab6c12a9e78ae4ef" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:03.538925 11994 raft_consensus.cc:2468] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:03.539068 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "024a5510f1da4a24ab6c12a9e78ae4ef" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a" is_pre_election: true
I20260709 12:23:03.539801 12142 raft_consensus.cc:2468] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 0.
I20260709 12:23:03.539963 12035 leader_election.cc:304] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.541096 12194 raft_consensus.cc:2804] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.541471 12194 raft_consensus.cc:493] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.541764 12194 raft_consensus.cc:3060] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.546929 12194 raft_consensus.cc:515] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.548605 12194 leader_election.cc:290] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.549261 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "024a5510f1da4a24ab6c12a9e78ae4ef" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.549584 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "024a5510f1da4a24ab6c12a9e78ae4ef" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.549806 11994 raft_consensus.cc:3060] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.550225 12142 raft_consensus.cc:3060] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.555101 11994 raft_consensus.cc:2468] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:03.555398 12142 raft_consensus.cc:2468] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 1.
I20260709 12:23:03.556293 12035 leader_election.cc:304] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.556982 12194 raft_consensus.cc:2804] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.557480 12194 raft_consensus.cc:697] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.558146 12194 consensus_queue.cc:237] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.564474 12194 consensus_queue.cc:1048] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.568495 12286 raft_consensus.cc:493] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.569051 12286 raft_consensus.cc:515] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.571362 12286 leader_election.cc:290] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:03.567287 11892 catalog_manager.cc:5697] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 1 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.572645 11995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9967187d4ff4971985685c439a92590" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" is_pre_election: true
I20260709 12:23:03.572995 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9967187d4ff4971985685c439a92590" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a958846336274bb386baea0d38600e01" is_pre_election: true
I20260709 12:23:03.573856 12068 raft_consensus.cc:2393] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91885dd1fdd54c928ae934cbe377559a in current term 1: Already voted for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in this term.
I20260709 12:23:03.574775 12109 leader_election.cc:304] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a; no voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01
I20260709 12:23:03.575858 12286 raft_consensus.cc:3060] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.584704 12286 raft_consensus.cc:2749] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.590057 12315 consensus_queue.cc:1048] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:03.592427 12279 consensus_queue.cc:1048] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.608433 12192 consensus_queue.cc:1048] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.648619 12068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77"
dest_uuid: "a958846336274bb386baea0d38600e01"
mode: GRACEFUL
new_leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:52278
I20260709 12:23:03.649370 12068 raft_consensus.cc:606] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Received request to transfer leadership to TS 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:23:03.655206 12142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f"
dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
mode: GRACEFUL
new_leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:55686
I20260709 12:23:03.655689 12142 raft_consensus.cc:606] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Received request to transfer leadership to TS 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:23:03.660665 12068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a"
dest_uuid: "a958846336274bb386baea0d38600e01"
mode: GRACEFUL
new_leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:52278
I20260709 12:23:03.661115 12068 raft_consensus.cc:606] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Received request to transfer leadership to TS 5ff0cd0b3e2b4c159ae5fae75c894a05
I20260709 12:23:03.668655 11994 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
mode: GRACEFUL
new_leader_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:60128
I20260709 12:23:03.669238 11994 raft_consensus.cc:606] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Received request to transfer leadership to TS a958846336274bb386baea0d38600e01
I20260709 12:23:03.669435 12192 consensus_queue.cc:1048] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.675308 11994 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
mode: GRACEFUL
new_leader_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:60128
I20260709 12:23:03.675841 11994 raft_consensus.cc:606] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Received request to transfer leadership to TS a958846336274bb386baea0d38600e01
I20260709 12:23:03.686889 12317 consensus_queue.cc:1048] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.691110 12069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a"
dest_uuid: "a958846336274bb386baea0d38600e01"
mode: GRACEFUL
new_leader_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:52278
I20260709 12:23:03.691746 12069 raft_consensus.cc:606] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Received request to transfer leadership to TS 91885dd1fdd54c928ae934cbe377559a
I20260709 12:23:03.697218 12315 raft_consensus.cc:993] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: : Instructing follower a958846336274bb386baea0d38600e01 to start an election
I20260709 12:23:03.697734 12317 raft_consensus.cc:1081] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Signalling peer a958846336274bb386baea0d38600e01 to start an election
I20260709 12:23:03.699354 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4"
dest_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:52264
I20260709 12:23:03.699901 12068 raft_consensus.cc:493] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.700268 12068 raft_consensus.cc:3060] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.705313 12068 raft_consensus.cc:515] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.706945 12068 leader_election.cc:290] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.707762 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.708052 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cfc047f3204be0aee3cc87b04c89f4" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.708465 11994 raft_consensus.cc:3055] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:03.708606 12142 raft_consensus.cc:3060] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.708861 11994 raft_consensus.cc:740] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:03.709718 11994 consensus_queue.cc:260] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.710752 11994 raft_consensus.cc:3060] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.714674 12142 raft_consensus.cc:2468] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:23:03.715637 12035 leader_election.cc:304] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.716120 11994 raft_consensus.cc:2468] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:23:03.716300 12194 raft_consensus.cc:2804] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:03.716852 12194 raft_consensus.cc:697] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.717490 12194 consensus_queue.cc:237] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.723721 11892 catalog_manager.cc:5697] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 1 to 2, leader changed from 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1) to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.805351 12279 raft_consensus.cc:993] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: : Instructing follower 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:23:03.805851 12194 raft_consensus.cc:1081] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Signalling peer 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:23:03.807307 11994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:60114
I20260709 12:23:03.807852 11994 raft_consensus.cc:493] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.808109 11994 raft_consensus.cc:3060] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.812991 11994 raft_consensus.cc:515] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.814294 11994 leader_election.cc:290] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.815274 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.815601 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8682452fd61349e49782e24ea9c59a77" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.815951 12068 raft_consensus.cc:3055] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:03.816216 12142 raft_consensus.cc:3060] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.816277 12068 raft_consensus.cc:740] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.817224 12068 consensus_queue.cc:260] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.818423 12068 raft_consensus.cc:3060] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.823403 12142 raft_consensus.cc:2468] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:23:03.824808 11960 leader_election.cc:304] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:03.825145 12068 raft_consensus.cc:2468] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:23:03.825565 12317 raft_consensus.cc:2804] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:03.826076 12317 raft_consensus.cc:697] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:03.826880 12317 consensus_queue.cc:237] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.833431 11892 catalog_manager.cc:5697] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 1 to 2, leader changed from a958846336274bb386baea0d38600e01 (127.30.25.2) to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.838961 12317 consensus_queue.cc:1048] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:03.845790 12315 raft_consensus.cc:993] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: : Instructing follower a958846336274bb386baea0d38600e01 to start an election
I20260709 12:23:03.846242 12315 raft_consensus.cc:1081] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Signalling peer a958846336274bb386baea0d38600e01 to start an election
I20260709 12:23:03.849534 12069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1"
dest_uuid: "a958846336274bb386baea0d38600e01"
 from {username='slave'} at 127.0.0.1:52264
I20260709 12:23:03.850647 12069 raft_consensus.cc:493] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.851059 12069 raft_consensus.cc:3060] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.853811 12315 consensus_queue.cc:1048] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.858740 12069 raft_consensus.cc:515] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.862401 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.863117 11994 raft_consensus.cc:3055] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:03.863407 11994 raft_consensus.cc:740] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:03.864122 11994 consensus_queue.cc:260] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.866569 12069 leader_election.cc:290] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:03.867473 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfc9ec5c0a24b0d94657880b1bd7fb1" candidate_uuid: "a958846336274bb386baea0d38600e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:03.868599 12142 raft_consensus.cc:3060] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.867254 11994 raft_consensus.cc:3060] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.874514 12194 consensus_queue.cc:1048] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.875434 11994 raft_consensus.cc:2468] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:23:03.875434 12142 raft_consensus.cc:2468] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a958846336274bb386baea0d38600e01 in term 2.
I20260709 12:23:03.876916 12035 leader_election.cc:304] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:03.877610 12194 raft_consensus.cc:2804] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:03.878065 12194 raft_consensus.cc:697] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Becoming Leader. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.878899 12194 consensus_queue.cc:237] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.887229 11892 catalog_manager.cc:5697] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 reported cstate change: term changed from 1 to 2, leader changed from 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1) to a958846336274bb386baea0d38600e01 (127.30.25.2). New cstate: current_term: 2 leader_uuid: "a958846336274bb386baea0d38600e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.888016 12307 consensus_queue.cc:1048] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:03.890615 12194 consensus_queue.cc:1048] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.917763 12194 raft_consensus.cc:993] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: : Instructing follower 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:23:03.918177 12194 raft_consensus.cc:1081] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Signalling peer 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:23:03.919548 12307 consensus_queue.cc:1048] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.920506 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a"
dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:55666
I20260709 12:23:03.921166 12143 raft_consensus.cc:493] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:03.921538 12143 raft_consensus.cc:3060] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.928232 12143 raft_consensus.cc:515] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.930905 11994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:03.931460 11994 raft_consensus.cc:3060] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.931409 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef9f6c9adea439b83bd628a79f6ba9a" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:03.932449 12069 raft_consensus.cc:3055] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:03.932859 12069 raft_consensus.cc:740] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:03.934207 12069 consensus_queue.cc:260] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.936959 12143 leader_election.cc:290] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 2 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:03.938858 12069 raft_consensus.cc:3060] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:03.938948 11994 raft_consensus.cc:2468] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 2.
I20260709 12:23:03.940871 12109 leader_election.cc:304] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:03.941605 12286 raft_consensus.cc:2804] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:03.942078 12286 raft_consensus.cc:697] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 2 LEADER]: Becoming Leader. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:23:03.942759 12286 consensus_queue.cc:237] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:03.950412 12069 raft_consensus.cc:2468] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 2.
I20260709 12:23:03.953398 11891 catalog_manager.cc:5697] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a reported cstate change: term changed from 1 to 2, leader changed from a958846336274bb386baea0d38600e01 (127.30.25.2) to 91885dd1fdd54c928ae934cbe377559a (127.30.25.3). New cstate: current_term: 2 leader_uuid: "91885dd1fdd54c928ae934cbe377559a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.014130 12307 consensus_queue.cc:1048] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.025501 12357 consensus_queue.cc:1048] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.080917 12355 raft_consensus.cc:993] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: : Instructing follower 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:23:04.081480 12313 raft_consensus.cc:1081] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Signalling peer 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:23:04.082901 11994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:60126
I20260709 12:23:04.083408 11994 raft_consensus.cc:493] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:04.083685 11994 raft_consensus.cc:3060] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:04.088217 11994 raft_consensus.cc:515] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:04.089558 11994 leader_election.cc:290] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:04.090507 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:04.090791 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e9f2ad0c2c14e26bb0db426ff70af3f" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:04.091221 12069 raft_consensus.cc:3060] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:04.091387 12142 raft_consensus.cc:3055] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:04.091787 12142 raft_consensus.cc:740] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:23:04.092621 12142 consensus_queue.cc:260] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:04.093884 12142 raft_consensus.cc:3060] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:04.098393 12069 raft_consensus.cc:2468] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:23:04.099773 11960 leader_election.cc:304] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:04.100626 12192 raft_consensus.cc:2804] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:04.101110 12192 raft_consensus.cc:697] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:04.101230 12142 raft_consensus.cc:2468] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:23:04.101994 12192 consensus_queue.cc:237] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:04.109591 11891 catalog_manager.cc:5697] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 1 to 2, leader changed from 91885dd1fdd54c928ae934cbe377559a (127.30.25.3) to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.158123 11994 raft_consensus.cc:1275] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.159339 12357 consensus_queue.cc:1048] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.166421 12142 raft_consensus.cc:1275] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.168190 12357 consensus_queue.cc:1048] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.173154 12357 raft_consensus.cc:993] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: : Instructing follower 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:23:04.173617 12307 raft_consensus.cc:1081] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Signalling peer 5ff0cd0b3e2b4c159ae5fae75c894a05 to start an election
I20260709 12:23:04.175875 11994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
 from {username='slave'} at 127.0.0.1:60114
I20260709 12:23:04.176440 11994 raft_consensus.cc:493] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:04.176822 11994 raft_consensus.cc:3060] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:04.183959 11994 raft_consensus.cc:515] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:04.185868 11994 leader_election.cc:290] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Requested vote from peers a958846336274bb386baea0d38600e01 (127.30.25.2:44995), 91885dd1fdd54c928ae934cbe377559a (127.30.25.3:44897)
I20260709 12:23:04.187083 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:04.187098 12142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e636804216407ab1bcfd35f444d34a" candidate_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
I20260709 12:23:04.187937 12069 raft_consensus.cc:3055] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:04.187940 12142 raft_consensus.cc:3060] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:04.188333 12069 raft_consensus.cc:740] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:04.189224 12069 consensus_queue.cc:260] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:04.190287 12069 raft_consensus.cc:3060] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:04.196310 12142 raft_consensus.cc:2468] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:23:04.197158 12069 raft_consensus.cc:2468] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff0cd0b3e2b4c159ae5fae75c894a05 in term 2.
I20260709 12:23:04.197430 11960 leader_election.cc:304] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:04.198278 12192 raft_consensus.cc:2804] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:04.198786 12192 raft_consensus.cc:697] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Becoming Leader. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:04.199435 12192 consensus_queue.cc:237] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:04.207311 11892 catalog_manager.cc:5697] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 reported cstate change: term changed from 1 to 2, leader changed from a958846336274bb386baea0d38600e01 (127.30.25.2) to 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1). New cstate: current_term: 2 leader_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.234438 12142 raft_consensus.cc:1275] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.235761 12192 consensus_queue.cc:1048] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.243860 12069 raft_consensus.cc:1275] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.245995 12192 consensus_queue.cc:1048] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.356604 11994 raft_consensus.cc:1275] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.357882 12307 consensus_queue.cc:1048] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.367329 12143 raft_consensus.cc:1275] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer a958846336274bb386baea0d38600e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.368618 12307 consensus_queue.cc:1048] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.490999 12069 raft_consensus.cc:1275] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.492367 12192 consensus_queue.cc:1048] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.500411 12143 raft_consensus.cc:1275] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.502184 12315 consensus_queue.cc:1048] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.512140 11994 raft_consensus.cc:1275] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.513480 12313 consensus_queue.cc:1048] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.519306 12069 raft_consensus.cc:1275] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.520954 12355 consensus_queue.cc:1048] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.671340 12069 raft_consensus.cc:1275] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.672793 12315 consensus_queue.cc:1048] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.681407 12143 raft_consensus.cc:1275] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Refusing update from remote peer 5ff0cd0b3e2b4c159ae5fae75c894a05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:04.683007 12192 consensus_queue.cc:1048] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.695539 11893 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37566:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:23:06.714901 11893 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37566:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:23:06.756325 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:06.862986 12399 tablet.cc:1724] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:06.868156 12398 tablet.cc:1724] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:06.872016 12397 tablet.cc:1724] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:06.929967 12068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1"
dest_uuid: "a958846336274bb386baea0d38600e01"
mode: GRACEFUL
new_leader_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:47146
I20260709 12:23:06.930779 12068 raft_consensus.cc:606] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Received request to transfer leadership to TS 91885dd1fdd54c928ae934cbe377559a
I20260709 12:23:06.975674 12410 tablet.cc:1724] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:23:06.981348 12403 log.cc:927] Time spent T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: Append to log took a long time: real 0.059s	user 0.004s	sys 0.000s
I20260709 12:23:06.957046 12398 tablet.cc:1724] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:23:06.992668 12409 log.cc:927] Time spent T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:07.012549 12422 tablet.cc:1724] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:23:07.027307 12402 log.cc:927] Time spent T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
I20260709 12:23:06.990378 12397 tablet.cc:1724] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.052666 12421 tablet.cc:1724] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.056918 12428 tablet.cc:1724] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:23:07.074748 12390 log.cc:927] Time spent T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
W20260709 12:23:07.076937 12417 log.cc:927] Time spent T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:07.067145 12423 tablet.cc:1724] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:23:07.083900 12412 log.cc:927] Time spent T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:23:07.092761 12397 tablet.cc:1724] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.114163 12399 tablet.cc:1724] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.132644 12410 tablet.cc:1724] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.133889 11994 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff"
dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
mode: GRACEFUL
new_leader_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:33026
I20260709 12:23:07.139562 11994 raft_consensus.cc:606] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Received request to transfer leadership to TS 91885dd1fdd54c928ae934cbe377559a
I20260709 12:23:07.135646 12426 tablet.cc:1724] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.138352 12398 tablet.cc:1724] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.144052 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:07.144737 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:07.146000 12432 tablet.cc:1724] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.149919 12424 raft_consensus.cc:993] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: : Instructing follower 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:23:07.150568 12424 raft_consensus.cc:1081] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Signalling peer 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:23:07.155037 12144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff"
dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:55680
I20260709 12:23:07.155645 12144 raft_consensus.cc:493] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:07.156077 12144 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:07.171456 12421 tablet.cc:1724] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.171676 12410 tablet.cc:1724] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.185806 12423 tablet.cc:1724] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.191866 12434 tablet.cc:1724] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.184839 12397 tablet.cc:1724] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.190204 12428 tablet.cc:1724] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.148387 12422 tablet.cc:1724] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.206864 12399 tablet.cc:1724] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.211772 12398 tablet.cc:1724] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.212680 12434 tablet.cc:1724] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.243556 12144 raft_consensus.cc:515] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:07.245754 12144 leader_election.cc:290] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 3 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:07.247404 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:07.248266 12066 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:07.247382 11993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9795bed32fab40ff939e0c11afb1f0ff" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:07.250455 11993 raft_consensus.cc:3055] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:07.250788 11993 raft_consensus.cc:740] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5ff0cd0b3e2b4c159ae5fae75c894a05, State: Running, Role: LEADER
I20260709 12:23:07.251607 11993 consensus_queue.cc:260] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:07.252929 11993 raft_consensus.cc:3060] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:07.233502 12423 tablet.cc:1724] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:07.256088 12066 raft_consensus.cc:2468] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 3.
I20260709 12:23:07.257419 12109 leader_election.cc:304] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91885dd1fdd54c928ae934cbe377559a, a958846336274bb386baea0d38600e01; no voters: 
I20260709 12:23:07.259703 12413 raft_consensus.cc:2804] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:07.260175 12413 raft_consensus.cc:697] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 3 LEADER]: Becoming Leader. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:23:07.260998 12413 consensus_queue.cc:237] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:07.265375 11993 raft_consensus.cc:2468] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 3.
I20260709 12:23:07.274443 11889 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b: soft_deleted_table [id=a3923d94d7bc4cd5874b03b394e17f5a] alter complete (version 1)
I20260709 12:23:07.282809 11893 catalog_manager.cc:5697] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a reported cstate change: term changed from 2 to 3, leader changed from 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1) to 91885dd1fdd54c928ae934cbe377559a (127.30.25.3). New cstate: current_term: 3 leader_uuid: "91885dd1fdd54c928ae934cbe377559a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:07.314777 12418 raft_consensus.cc:993] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: : Instructing follower 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:23:07.315377 12420 raft_consensus.cc:1081] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Signalling peer 91885dd1fdd54c928ae934cbe377559a to start an election
I20260709 12:23:07.317085 12139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1"
dest_uuid: "91885dd1fdd54c928ae934cbe377559a"
 from {username='slave'} at 127.0.0.1:55666
I20260709 12:23:07.317670 12139 raft_consensus.cc:493] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:07.318050 12139 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:07.323824 12139 raft_consensus.cc:515] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:07.325567 12139 leader_election.cc:290] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 3 election: Requested vote from peers 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969), a958846336274bb386baea0d38600e01 (127.30.25.2:44995)
I20260709 12:23:07.326694 11993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05"
I20260709 12:23:07.326869 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "543a0c698b8644dd945607edb988a1d1" candidate_uuid: "91885dd1fdd54c928ae934cbe377559a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a958846336274bb386baea0d38600e01"
I20260709 12:23:07.327296 11993 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:07.327512 12066 raft_consensus.cc:3055] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:07.327875 12066 raft_consensus.cc:740] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a958846336274bb386baea0d38600e01, State: Running, Role: LEADER
I20260709 12:23:07.328552 12066 consensus_queue.cc:260] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:07.329612 12066 raft_consensus.cc:3060] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:07.332813 11993 raft_consensus.cc:2468] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 3.
I20260709 12:23:07.334196 12109 leader_election.cc:304] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff0cd0b3e2b4c159ae5fae75c894a05, 91885dd1fdd54c928ae934cbe377559a; no voters: 
I20260709 12:23:07.334949 12413 raft_consensus.cc:2804] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:07.335223 12066 raft_consensus.cc:2468] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91885dd1fdd54c928ae934cbe377559a in term 3.
I20260709 12:23:07.335469 12413 raft_consensus.cc:697] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 3 LEADER]: Becoming Leader. State: Replica: 91885dd1fdd54c928ae934cbe377559a, State: Running, Role: LEADER
I20260709 12:23:07.336405 12413 consensus_queue.cc:237] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } }
I20260709 12:23:07.344463 11893 catalog_manager.cc:5697] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a reported cstate change: term changed from 2 to 3, leader changed from a958846336274bb386baea0d38600e01 (127.30.25.2) to 91885dd1fdd54c928ae934cbe377559a (127.30.25.3). New cstate: current_term: 3 leader_uuid: "91885dd1fdd54c928ae934cbe377559a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91885dd1fdd54c928ae934cbe377559a" member_type: VOTER last_known_addr { host: "127.30.25.3" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:07.791706 11993 raft_consensus.cc:1275] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:07.793171 12413 consensus_queue.cc:1048] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:07.806036 12066 raft_consensus.cc:1275] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:07.807459 12414 consensus_queue.cc:1048] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:07.832019 11993 raft_consensus.cc:1275] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:07.833422 12414 consensus_queue.cc:1048] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff0cd0b3e2b4c159ae5fae75c894a05" member_type: VOTER last_known_addr { host: "127.30.25.1" port: 36969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:07.844089 12066 raft_consensus.cc:1275] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Refusing update from remote peer 91885dd1fdd54c928ae934cbe377559a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:07.845463 12443 consensus_queue.cc:1048] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a958846336274bb386baea0d38600e01" member_type: VOTER last_known_addr { host: "127.30.25.2" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:08.145906 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:08.152482 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:08.152884 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:09.153915 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:09.158844 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:09.159344 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:23:10.436352 11960 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.30.25.3:44897, user_credentials={real_user=slave}} blocked reactor thread for 53612.8us
I20260709 12:23:10.639112 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:10.644165 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:10.644512 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:23:10.918179 11924 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9140 after lost signal to thread 30823
W20260709 12:23:10.919234 11924 debug-util.cc:398] Leaking SignalData structure 0x7b08000745a0 after lost signal to thread 11927
W20260709 12:23:10.920333 11924 debug-util.cc:398] Leaking SignalData structure 0x7b080004f2a0 after lost signal to thread 12019
W20260709 12:23:10.921334 11924 debug-util.cc:398] Leaking SignalData structure 0x7b0800354d00 after lost signal to thread 12094
W20260709 12:23:10.922297 11924 debug-util.cc:398] Leaking SignalData structure 0x7b0800063e80 after lost signal to thread 12168
I20260709 12:23:11.645574 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:11.654253 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:11.654925 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:12.656524 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:12.665270 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:12.665858 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:13.667135 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:13.673074 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:13.673530 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:14.674542 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:14.679411 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:14.679847 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:15.680995 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:15.686265 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:15.686759 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:16.687791 30820 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:16.693188 30820 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:16.693630 30820 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:17.751935 30820 tablet_server.cc:179] TabletServer@127.30.25.1:0 shutting down...
W20260709 12:23:17.764560 12035 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 64 similar messages]
W20260709 12:23:17.771169 12035 consensus_peers.cc:597] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:17.778376 12035 consensus_peers.cc:597] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.795420 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:17.796773 30820 tablet_replica.cc:333] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.798470 30820 raft_consensus.cc:2243] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.799607 30820 raft_consensus.cc:2272] T 024a5510f1da4a24ab6c12a9e78ae4ef P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:17.801215 12109 consensus_peers.cc:597] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.803539 30820 tablet_replica.cc:333] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.804430 30820 raft_consensus.cc:2243] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.805071 30820 raft_consensus.cc:2272] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.807688 30820 tablet_replica.cc:333] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.808435 30820 raft_consensus.cc:2243] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.809010 30820 raft_consensus.cc:2272] T 8ef9f6c9adea439b83bd628a79f6ba9a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.811596 30820 tablet_replica.cc:333] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.812284 30820 raft_consensus.cc:2243] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.812924 30820 raft_consensus.cc:2272] T 719c67c8a50d48b1ae3d8b28c3892a24 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.815449 30820 tablet_replica.cc:333] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.816094 30820 raft_consensus.cc:2243] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.816677 30820 raft_consensus.cc:2272] T d4cfc047f3204be0aee3cc87b04c89f4 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.820115 30820 tablet_replica.cc:333] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.821027 30820 raft_consensus.cc:2243] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:17.822068 30820 raft_consensus.cc:2272] T b9967187d4ff4971985685c439a92590 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.824661 30820 tablet_replica.cc:333] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.825269 30820 raft_consensus.cc:2243] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:17.826519 30820 raft_consensus.cc:2272] T 8682452fd61349e49782e24ea9c59a77 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.829608 30820 tablet_replica.cc:333] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
W20260709 12:23:17.829679 12035 consensus_peers.cc:597] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.830271 30820 raft_consensus.cc:2243] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:17.831401 30820 raft_consensus.cc:2272] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.834116 30820 tablet_replica.cc:333] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.834755 30820 raft_consensus.cc:2243] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:17.835719 30820 raft_consensus.cc:2272] T 37e636804216407ab1bcfd35f444d34a P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.838814 30820 tablet_replica.cc:333] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.839404 30820 raft_consensus.cc:2243] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:23:17.840196 12035 consensus_peers.cc:597] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.840422 30820 raft_consensus.cc:2272] T ede410d0ce4646889e6f17c8c742cc46 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.843108 30820 tablet_replica.cc:333] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.843677 30820 raft_consensus.cc:2243] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.844231 30820 raft_consensus.cc:2272] T 1e0995365e4a4b878cb2b16803b7f0c5 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.846724 30820 tablet_replica.cc:333] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.847319 30820 raft_consensus.cc:2243] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:17.848229 30820 raft_consensus.cc:2272] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.850610 30820 tablet_replica.cc:333] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.851173 30820 raft_consensus.cc:2243] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.851661 30820 raft_consensus.cc:2272] T 9795bed32fab40ff939e0c11afb1f0ff P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.854410 30820 tablet_replica.cc:333] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.855247 30820 raft_consensus.cc:2243] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.855943 30820 raft_consensus.cc:2272] T 543a0c698b8644dd945607edb988a1d1 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:17.857733 12035 consensus_peers.cc:597] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.859198 30820 tablet_replica.cc:333] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.859838 30820 raft_consensus.cc:2243] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:17.860730 30820 raft_consensus.cc:2272] T 516cbcf53f6f4a288926bbfb38ee34b0 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.863488 30820 tablet_replica.cc:333] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.864037 30820 raft_consensus.cc:2243] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.864589 30820 raft_consensus.cc:2272] T c5ee1ff107b34cae9b04c12178909492 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.867164 30820 tablet_replica.cc:333] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.867668 30820 raft_consensus.cc:2243] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.868136 30820 raft_consensus.cc:2272] T 215c4dd47fc64dd4ae6dfad696c471da P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.870421 30820 tablet_replica.cc:333] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05: stopping tablet replica
I20260709 12:23:17.870954 30820 raft_consensus.cc:2243] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.871421 30820 raft_consensus.cc:2272] T ae0d804930c3497da9c71c272591e6b8 P 5ff0cd0b3e2b4c159ae5fae75c894a05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.921067 30820 tablet_server.cc:196] TabletServer@127.30.25.1:0 shutdown complete.
I20260709 12:23:17.965416 30820 tablet_server.cc:179] TabletServer@127.30.25.2:0 shutting down...
I20260709 12:23:17.999764 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:18.000452 30820 tablet_replica.cc:333] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.001377 30820 raft_consensus.cc:2243] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.002381 30820 raft_consensus.cc:2272] T 024a5510f1da4a24ab6c12a9e78ae4ef P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.005030 30820 tablet_replica.cc:333] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.005651 30820 raft_consensus.cc:2243] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.006599 30820 raft_consensus.cc:2272] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.008877 30820 tablet_replica.cc:333] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.009473 30820 raft_consensus.cc:2243] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.010366 30820 raft_consensus.cc:2272] T d4cfc047f3204be0aee3cc87b04c89f4 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.012662 30820 tablet_replica.cc:333] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.013360 30820 raft_consensus.cc:2243] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.014147 30820 raft_consensus.cc:2272] T 719c67c8a50d48b1ae3d8b28c3892a24 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.016444 30820 tablet_replica.cc:333] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.017032 30820 raft_consensus.cc:2243] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.017557 30820 raft_consensus.cc:2272] T 8ef9f6c9adea439b83bd628a79f6ba9a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.019847 30820 tablet_replica.cc:333] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.020395 30820 raft_consensus.cc:2243] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.020916 30820 raft_consensus.cc:2272] T b9967187d4ff4971985685c439a92590 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.023198 30820 tablet_replica.cc:333] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.023758 30820 raft_consensus.cc:2243] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.024324 30820 raft_consensus.cc:2272] T 8682452fd61349e49782e24ea9c59a77 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.026579 30820 tablet_replica.cc:333] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.027066 30820 raft_consensus.cc:2243] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.027727 30820 raft_consensus.cc:2272] T 9795bed32fab40ff939e0c11afb1f0ff P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.029814 30820 tablet_replica.cc:333] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.030459 30820 raft_consensus.cc:2243] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.030939 30820 raft_consensus.cc:2272] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.033008 30820 tablet_replica.cc:333] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.033557 30820 raft_consensus.cc:2243] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.034049 30820 raft_consensus.cc:2272] T 6e9f2ad0c2c14e26bb0db426ff70af3f P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.036094 30820 tablet_replica.cc:333] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.036656 30820 raft_consensus.cc:2243] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.037163 30820 raft_consensus.cc:2272] T 37e636804216407ab1bcfd35f444d34a P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.039167 30820 tablet_replica.cc:333] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.039703 30820 raft_consensus.cc:2243] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.040154 30820 raft_consensus.cc:2272] T ede410d0ce4646889e6f17c8c742cc46 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.042451 30820 tablet_replica.cc:333] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.042943 30820 raft_consensus.cc:2243] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.043906 30820 raft_consensus.cc:2272] T 215c4dd47fc64dd4ae6dfad696c471da P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.046052 30820 tablet_replica.cc:333] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.046591 30820 raft_consensus.cc:2243] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.047434 30820 raft_consensus.cc:2272] T ae0d804930c3497da9c71c272591e6b8 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.049650 30820 tablet_replica.cc:333] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.050173 30820 raft_consensus.cc:2243] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.050657 30820 raft_consensus.cc:2272] T 516cbcf53f6f4a288926bbfb38ee34b0 P a958846336274bb386baea0d38600e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.052681 30820 tablet_replica.cc:333] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.053188 30820 raft_consensus.cc:2243] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.053659 30820 raft_consensus.cc:2272] T 543a0c698b8644dd945607edb988a1d1 P a958846336274bb386baea0d38600e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.055754 30820 tablet_replica.cc:333] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.056223 30820 raft_consensus.cc:2243] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.056718 30820 raft_consensus.cc:2272] T c5ee1ff107b34cae9b04c12178909492 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.058727 30820 tablet_replica.cc:333] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01: stopping tablet replica
I20260709 12:23:18.059208 30820 raft_consensus.cc:2243] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.060040 30820 raft_consensus.cc:2272] T 1e0995365e4a4b878cb2b16803b7f0c5 P a958846336274bb386baea0d38600e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.102644 30820 tablet_server.cc:196] TabletServer@127.30.25.2:0 shutdown complete.
W20260709 12:23:18.130254 12109 consensus_peers.cc:597] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a -> Peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Couldn't send request to peer a958846336274bb386baea0d38600e01. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:44995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.145218 30820 tablet_server.cc:179] TabletServer@127.30.25.3:0 shutting down...
I20260709 12:23:18.178203 30820 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:18.179018 30820 tablet_replica.cc:333] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.179926 30820 raft_consensus.cc:2243] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.180588 30820 raft_consensus.cc:2272] T 024a5510f1da4a24ab6c12a9e78ae4ef P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.183350 30820 tablet_replica.cc:333] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.184394 30820 raft_consensus.cc:2243] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.185204 30820 raft_consensus.cc:2272] T 719c67c8a50d48b1ae3d8b28c3892a24 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:18.187494 12109 consensus_peers.cc:597] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a -> Peer a958846336274bb386baea0d38600e01 (127.30.25.2:44995): Couldn't send request to peer a958846336274bb386baea0d38600e01. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.2:44995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.188606 30820 tablet_replica.cc:333] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.189381 30820 raft_consensus.cc:2243] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.190382 30820 raft_consensus.cc:2272] T 8ef9f6c9adea439b83bd628a79f6ba9a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.192925 30820 tablet_replica.cc:333] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.193573 30820 raft_consensus.cc:2243] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.194121 30820 raft_consensus.cc:2272] T d4cfc047f3204be0aee3cc87b04c89f4 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.196683 30820 tablet_replica.cc:333] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.197350 30820 raft_consensus.cc:2243] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.197974 30820 raft_consensus.cc:2272] T 2dfc9ec5c0a24b0d94657880b1bd7fb1 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.201440 30820 tablet_replica.cc:333] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.202102 30820 raft_consensus.cc:2243] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:23:18.202514 12109 consensus_peers.cc:597] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.202809 30820 raft_consensus.cc:2272] T 6e9f2ad0c2c14e26bb0db426ff70af3f P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.205284 30820 tablet_replica.cc:333] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.205860 30820 raft_consensus.cc:2243] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.206370 30820 raft_consensus.cc:2272] T b9967187d4ff4971985685c439a92590 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.208540 30820 tablet_replica.cc:333] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.209079 30820 raft_consensus.cc:2243] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.209539 30820 raft_consensus.cc:2272] T 8682452fd61349e49782e24ea9c59a77 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.211835 30820 tablet_replica.cc:333] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.212359 30820 raft_consensus.cc:2243] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.213249 30820 raft_consensus.cc:2272] T c5ee1ff107b34cae9b04c12178909492 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.216569 30820 tablet_replica.cc:333] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.217419 30820 raft_consensus.cc:2243] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.218035 30820 raft_consensus.cc:2272] T ae0d804930c3497da9c71c272591e6b8 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:18.218415 12109 consensus_peers.cc:597] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a -> Peer 5ff0cd0b3e2b4c159ae5fae75c894a05 (127.30.25.1:36969): Couldn't send request to peer 5ff0cd0b3e2b4c159ae5fae75c894a05. Status: Network error: Client connection negotiation failed: client connection to 127.30.25.1:36969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.220650 30820 tablet_replica.cc:333] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.221308 30820 raft_consensus.cc:2243] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.221729 30820 raft_consensus.cc:2272] T 215c4dd47fc64dd4ae6dfad696c471da P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.224543 30820 tablet_replica.cc:333] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.225080 30820 raft_consensus.cc:2243] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.225498 30820 raft_consensus.cc:2272] T 516cbcf53f6f4a288926bbfb38ee34b0 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.227955 30820 tablet_replica.cc:333] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.228451 30820 raft_consensus.cc:2243] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.229372 30820 raft_consensus.cc:2272] T 543a0c698b8644dd945607edb988a1d1 P 91885dd1fdd54c928ae934cbe377559a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.231668 30820 tablet_replica.cc:333] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.232201 30820 raft_consensus.cc:2243] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.233052 30820 raft_consensus.cc:2272] T 9795bed32fab40ff939e0c11afb1f0ff P 91885dd1fdd54c928ae934cbe377559a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.235498 30820 tablet_replica.cc:333] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.236021 30820 raft_consensus.cc:2243] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.236471 30820 raft_consensus.cc:2272] T a8d39c2cef1e4d8dbe8b750de8c7f9e2 P 91885dd1fdd54c928ae934cbe377559a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.238842 30820 tablet_replica.cc:333] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.239339 30820 raft_consensus.cc:2243] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.239816 30820 raft_consensus.cc:2272] T ede410d0ce4646889e6f17c8c742cc46 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.241897 30820 tablet_replica.cc:333] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.242444 30820 raft_consensus.cc:2243] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.242908 30820 raft_consensus.cc:2272] T 37e636804216407ab1bcfd35f444d34a P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.245072 30820 tablet_replica.cc:333] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a: stopping tablet replica
I20260709 12:23:18.245522 30820 raft_consensus.cc:2243] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:18.245993 30820 raft_consensus.cc:2272] T 1e0995365e4a4b878cb2b16803b7f0c5 P 91885dd1fdd54c928ae934cbe377559a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.291406 30820 tablet_server.cc:196] TabletServer@127.30.25.3:0 shutdown complete.
I20260709 12:23:18.330346 30820 master.cc:562] Master@127.30.25.62:40149 shutting down...
I20260709 12:23:18.354802 30820 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:18.355444 30820 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:18.355782 30820 tablet_replica.cc:333] T 00000000000000000000000000000000 P 20b8427267024bf089c30070b05a671b: stopping tablet replica
I20260709 12:23:18.393553 30820 master.cc:584] Master@127.30.25.62:40149 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20905 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (48886 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (459576 ms total)
[  PASSED  ] 8 tests.
I20260709 12:23:18.452414 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2538 messages since previous log ~39 seconds ago
I20260709 12:23:18.452757 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2544 messages since previous log ~39 seconds ago
I20260709 12:23:18.453033 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2544 messages since previous log ~39 seconds ago
I20260709 12:23:18.453346 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~57 seconds ago
I20260709 12:23:18.453616 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~57 seconds ago
I20260709 12:23:18.453846 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 4 messages since previous log ~185 seconds ago
I20260709 12:23:18.454046 30820 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 8 messages since previous log ~266 seconds ago
I20260709 12:23:18.454252 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 8 messages since previous log ~266 seconds ago
I20260709 12:23:18.454502 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 1 messages since previous log ~318 seconds ago
I20260709 12:23:18.454785 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20260709 12:23:18.455087 30820 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
